Document ID: EPA-HQ-OA-2018-0259-22467
Agency: epa
Document Type: Supporting & Related Material
Title: 
Posted Date: 2021-01-06T05:00Z

Strengthening Transparency in Regulatory Science
            (Part  30 - Transparency in Regulatory Decision Making)
                                       
                                       
                        Response to Public Comments on 
                                        
 Proposed Rule [83 FR 18768, April 30, 2018; 85 FR 15396, March 18, 2020]  
                                        
                                        
                                        
                                        
                                        
                                        
                                        
                                        
                                        
                                        
                                        
                                        
                                        
                                        
 

                                 Dexember 2020
                                       
 
 
Comments, letters, and transcripts of the public hearings are also available electronically through http://www.regulations.gov by searching Docket ID EPA-HQ-OA-2018-0259 

FOREWORD 
This document provides the EPA's responses to public comments on the EPA's Proposed Strengthening Transparency in Regulatory Science. The EPA published a Notice of Proposed Rulemaking (NPRM) in the Federal Register on April 30, 2018, at 83 FR 18768. In addition to the NPRM, the EPA published a Supplemental Notice of Proposed Rulemaking (SNPRM) on this action in the Federal Register on March 18, 2020. The EPA received comments on the NPRM and SNPRM via mail, e-mail, facsimile, and a public hearing held in Washington, DC on July 17, 2018. Copies of all comments and transcripts for the public hearing are available at the EPA Docket Center Public Reading Room. Out of an abundance of caution for members of the public and our staff, the EPA Docket Center and Reading Room was closed to public visitors on March 31, 2020, to reduce the risk of transmitting COVID-19. Our Docket Center staff will continue to provide remote customer service via email, phone, and webform while the EPA Docket Center and Reading Room are closed. Comments and transcripts of the public hearing are also available electronically through http://www.regulations.gov by searching Docket ID EPA-HQ-OA-2018-0259. For further information and updates on EPA Docket Center services and the current status, please visit us online at https://www.epa.gov/dockets. 
Just over 950,000 public comments were received on the NPRM and SNPRM. The EPA Docket Center consolidated mass mail campaigns and petitions into single document control numbers (DCNs), resulting in over 22,000 unique comments. Each of these comments was reviewed and significant comments relevant to this action and submitted within the comment period have been summarized and included in this document. 
It is possible some responses in the Response to Comments Document may not reflect the language in the preamble and final rule in every respect. Where the response conflicts with the preamble or the final rule, the language in the final preamble and rule controls and should be used for purposes of understanding the scope, requirements, and basis of the final rule. The responses presented in this document are intended to augment the responses to comments that appear in the preamble to the final rule or to address comments not discussed in that preamble. Although portions of the preamble to the final rule are paraphrased in this document where useful to add clarity to responses, the preamble itself remains the definitive statement of the rationale for the revisions adopted in the final rule. In many instances, responses presented in the Response to Comments Document include cross references to responses on related issues that are located either in the preamble or elsewhere in the Response to Comments Document. Accordingly, the Response to Comments Document, together with the preamble and final rule, and the rest of the administrative record should be considered collectively as the Agency's response to all the significant comments submitted on the proposed rule. 

Table of Contents
Chapter 1: Legal Authority	1
1.1 General Comments	2
1.1.1 General Legal Authority (Notice of Proposed Rulemaking (NPRM) Comments)	2
1.1.2 General Legal Authority (Supplemental Notice of Proposed Rulemaking (SPNRM) Comments)	24
1.2 Housekeeping Statue	64
1.3 Specific Environmental Statues	152
1.3.1 Clean Air Act (CAA) (NPRM and SNPRM Comments).	152
1.3.2 Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA) (NPRM and SNPRM Comments).	252
1.3.3 Safe Drinking Water Act (SDWA) (NPRM and SNPRM Comments).	282
1.3.4 Toxic Substances and Control Act (TSCA) (NPRM and SNPRM Comments)	315
1.3.5 Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA)/ Resource Conservation and Recovery Act (RCRA) (NPRM and SNPRM Comments).	365
1.3.6 Emergency Planning and Community Right-to-Know Act (EPCRA) (NPRM and SNPRM Comments).	398
1.3.7 Clean Water Act (CWA) (NPRM and SNPRM comments).	407
1.3.8 National Environmental Policy Act (NEPA) (SNPRM Comments)	436
1.3.9 Other Environmental Statutes (NPRM Comments)	438
1.4 Evidence Act (SNPRM Comments)	469
1.5 Consistency with the Peer Review Processes Required Under Environmental Statue (NPRM Comments)	470
1.6 Administrative Procedures Act (APA)	482
1.6.1 APA (NPRM and SNPRM Comments).	482
1.7 Other Authorities (NPRM Comments)	622
1.8 Trying to Achieve Administratively What Could Not be Done Legislatively	645
1.9 Other Comments (NPRM Comments)	652
Chapter 2: Purpose and Effects	663
2.1 Purpose and Effects (NPRM Comments)	664
2.1.1 Change in Culture	664
2.1.2 What Problem is the Agency Intending to Fix?	668
2.1.3 Proposed Rule is Biased	703
2.1.4 Environmental and Health Protections Would be Decreased	748
2.1.5 EPA Would be at Odds with Other Parts of the Federal Government	755
2.1.6 EPA Would be at Odds with Other National and International Bodies	763
2.1.7 Proposed Rule is Flawed and Needs to Be Withdrawn/Redone with Feedback from the Scientific Community/Stakeholders	766
2.1.8 Other Effects/Impacts	831
2.2 Purpose/Basis and Effects (SNPRM Comments)	848
2.2.1 Purpose/Basis for Rule	848
2.2.2 Effects	878
Chapter 3: Capable of Being Substantially Reproduced, Independent Validation, and Reanalyze	994
3.1 NPRM Replicability, Reproducibility and Reanalysis	995
3.1.1 Proposed Rule Requirement to Make Available Information Necessary for the Public to Replicate Findings (30.5)	995
3.1.2 Clarification of the Meaning of Terms Used in the Proposal	998
3.1.3 Replication versus Reproducible versus Reanalysis	1002
3.1.4 Whether the Ability to Reanalyze, Replicate and Reproduce a Study Should Determine the Quality of the Study	1005
3.1.5 Replication-Specific Comments	1011
3.1.6 Reproducibility  -  Specific Comments	1018
3.1.7 Replication (or Reproduction) Crisis	1030
3.1.8 Reanalysis  -  Specific Comments	1042
3.1.9 Recommends Further Consultation to Develop New Rule Addressing Reproducibility and Replicability Concerns	1047
3.2 SNPRM Capable of Being Substantially Reproduced; Replication and Reproduction	1047
3.2.1 Capable of Being Substantially Reproduced; Replication and Reproduction	1047
3.2.2 Reanalyze	1072
3.2.3 Independent Validation	1088
Chapter 4: Data and Models	1098
4.1 NPRM Data and Models  -  Related Comments	1100
4.1.1 Data and Models; Dose response data and models (§30.2), requirements to the use of dose response data and models underlying pivotal regulatory science (§30.5), and additional requirements (§30.6)	1100
4.1.2 Definitions of Research Data (§30.2) and Data	1282
4.2 SNPRM Models, Variability/Uncertainty, and Assumptions/Defaults	1285
4.2.1 Modeling Definitions	1285
4.2.2 Models (General) and Best Science Practices	1286
4.2.3 Public Availability of Models/Data	1289
4.2.4 Weighting of Models	1291
4.2.5 Scope of Models Included	1293
4.2.6 Assumptions/Defaults-Variability/Uncertainty	1294
4.2.7 Impacts on Older/Existing Models	1297
4.2.8 Linear vs Nonlinear Modeling	1300
4.2.9 Impacts on Specific Models	1301
4.3 SNPRM Data, Variability/Uncertainty, and Assumptions/Defaults	1303
4.3.1. Clarity of Proposed Definitions	1303
4.3.2 Application of the Rule in Respect to Data	1314
Chapter 5: Influential Scientific Information; Expansion of Proposal; Determining and Weighting ISI (Supplemental Notice of Proposed Rulemaking (SNPRM))	1329
5.1 Definition of Influential Scientific Information	1330
5.1.1 General Support/Dissent for Definition	1330
5.1.2 What is Covered by the Definition of Influential Scientific Information	1331
5.2 Expansion of Scope to Include Influential Scientific Information	1334
5.2.1 General Support/Dissent for Expansion	1334
5.2.2 Justification/Necessity for Expanding Scope	1336
5.2.3 Application Guidance for EPA Programs	1337
5.2.4 Public Comment on Expanded Scope	1338
5.3 Determining What Constitutes Influential Scientific Information	1339
5.3.1 Limits to EPA's Application of the Rule	1345
Chapter 6: Pivotal Science	1346
6.1 NPRM Pivotal Regulatory Science	1347
6.1.1 How Should the Proposed Rule Apply to Dose Response Data and Models Underlying Pivotal Regulatory Science Developed Prior to the Effective Date?	1347
6.1.2 Could Prospective or Retrospective Application Introduce Bias Regarding the Timeliness and Quality of Scientific Information Available?	1350
6.1.3 How Does What this Rule Would Require Compare to Requirements of Environmental Statutes?	1371
6.1.4 Recommended Changes to the Definition of Pivotal Regulatory Science at §30.2	1374
6.2 SNPRM Pivotal Science	1387
6.2.1 Need for Additional Clarity/Specificity	1387
6.2.2 Scope of Pivotal Science	1390
Chapter 7: Publicly Available	1392
7.1 Publicly Available, Tiered Access	1393
7.1.1 40 CFR 30.2 Definition of Publicly Available	1393
7.1.2 40 CFR 30.5: Broad Issues	1403
7.1.3 40 CFR 30.5: Option 1	1450
7.1.4 40 CFR 30.5: Option 2	1505
7.1.5 Ensure Over Time More Availability of Data	1528
Chapter 8: Research Data	1536
8.1 Uniform Administrative Requirements	1536
8.2 Applicability of the Rule to Research Laboratory Samples.	1538
Chapter 9: Science That Serves as the Basis for Informing a Final Significant Regulatory Actions	1539
9.1 NPRM Regulatory Science (Section 30.2)	1540
9.1.1 Ensure that the Regulatory Science is Publicly Available in a Manner Sufficient for Independent Validation (§30.1)	1540
9.2 Regulatory Science Means Scientific Information that Provides the Basis for EPA Final Significant Regulatory Decisions (§30.2)	1589
9.3 EPA's Regulatory Science Should Be Consistent with the OMB's Final Information Quality Bulletin for Peer Review (83 FR 18770)	1592
9.4 EPA's Proposed Regulatory Science Requirements Consistency with OMB's Guidelines for Ensuring and Maximizing Quality, Objectivity, Utility and Integrity of Information Disseminated by Federal Agencies (OMB Guidelines)	1592
9.5 Regulatory Science that Cannot Meet the Requirements of §§30.5 and 30.6	1596
9.6 How Should EPA Address a Circumstance in which EPA Has a Statutory Requirement to Make a Determination for Which Scientific Information Publicly Available Sufficient for Independent Validation Does Not Exist?	1598
9.7 How Does What this Rule Would Require Compare to Requirements of Environmental Statutes? (Relates to Chapter 1)	1600
9.7.1 Best Available Science	1600
9.7.2 Data Quality Act	1604
9.7.3 Other	1606
Chapter 10: Applicability of the Rule	1608
10.1 Limiting Requirements to "Significant Regulatory Actions"	1609
10.2 Expanding the Scope of Applicability of the Rule	1611
10.2.1 Application of the Rule to Other Regulatory Actions	1612
10.2.2 Application of the Rule to Agency Guidance	1614
10.2.3 Application of the Rule to Proposed Rules, Advanced Notice of Proposed Rulemaking and All Aspects of the Rulemaking Process	1615
10.2.4 Individual Party Adjudications, Enforcement Activities, or Permit Proceedings	1619
10.2.5 Application of the Rule to Actions Relying on Records from Previous Reviews	1624
10.2.6 Application of the Rule to Specific EPA Programs	1631
10.2.7 Application of the Rule to a Limited and Special Class of Laboratory Samples	1645
10.3 Narrowing the Scope of Applicability of the Rule	1646
10.3.1 Narrowing the Scope of the Applicability of the Rule	1646
10.3.2 Limiting to "Major" Issues Under the Congressional Review Act or "Economically Signification" Actions Under Executive Order 12866	1647
10.3.3 CAA	1648
10.3.4 Other Comments on Narrowing the Scope of Applicability of the Rule	1649
10.4 Changes in the Scope of the Science Subject to the Rule	1649
10.5 Scope  -  Phase-In	1651
10.5.1 Proposed Rule Does Not Provide for Any Phase-In Time	1651
10.5.2 Support Phasing-In of Transparency Requirements	1651
10.5.3 Conditional Phase-In Implementation Recommendations	1652
10.5.4 Implement Existing Guidance More Fully Before Considering Phasing-In of Rule	1654
10.5.5 Phase-In for Use of Past Studies Lacking in Sufficient Transparency	1655
10.5.6 States Involvement in Identifying Priorities	1655
10.5.7 Transparency Rule Should Never Be Implemented	1655
10.6 Scope  -  Effects on Individual EPA Programs and Exceptions	1656
10.6.1 Proposed Rule Would Have Effects on All/Several EPA Programs	1656
10.6.2 Evaluating Effects of Proposal on Environmental Programs and their Implementation is Unnecessary	1657
10.7 40 CFR 30.3 - How Do the Provisions of this Subpart Apply?	1657
10.7.1 Unlimited Discretion and Authority to Decide What "Any Other Type of Agency Action" is or is Not	1657
10.7.2 Applicability Covers Studies, Models and Analyses Integral to the Functioning of EPA Regulatory Programs	1658
10.7.3 Drafted Exclusions from Proposal's Prohibitions	1660
10.7.4 Applicability to Government-Funded and/or Beyond Government-Funded Scientific Research	1660
10.7.5 Data Pivotal to the Basis of a Regulatory Decision	1662
10.7.6 No Clear Information on How the Rule will Apply	1663
10.7.7 Physical Objects, Drafts and Preliminary Analyses Exclusion	1664
10.8 "Regulatory Decisions" Definition at 40 CFR 30.2	1664
10.8.1 Amend Definition to Remove Limitation of Applicability to Only Final Regulations	1665
10.8.2 Proposal is Inconsistent with Regards to What Types of Regulatory Actions Would be Covered	1665
10.8.3 Regulatory Decisions Should Also Apply to Permitting Activities	1666
10.8.4 Recommends that the Regulatory Decisions Definition Include EPA-Focused Criteria	1666
10.8.5 Basis for Regulatory Decisions	1667
Chapter 11: Availability of Data and Models	1669
11.1. NPRM Transparency and Integrity of Regulatory Decision Making vs. Availability of Raw Data	1669
11.1.1 Basis for EPA Decision-Making Regarding Use/Non-Use of Studies Not Required to be Transparent	1669
11.1.2 EPA Actions in Other Areas	1671
11.2 SNPRM Data Availability-Related Comments	1673
11.2.1 Confidential Business Information, Proprietary Data, or Personally Identifiable Information	1673
Chapter 12: Administrator's Exemptions	1701
12.1 Exemptions (40 CFR 30.9) (NPRM Comments)	1702
12.1.1 Legality of Proposed Exemption Provision in 40 CFR 30.9 of the Rule	1702
12.1.2 Case-by-Case Administrator Exemption Authority	1705
12.1.3 Criteria for Granting Case-by-Case Exemptions	1717
12.1.4 Requests for Specific Rule Exemptions	1726
12.1.5 Define "Reasonable Effort"	1727
12.1.6 Exemption Process Costs	1728
12.1.7 Notice-and-Comment Principles Should Inform EPA's Implementation of the Exemption Process	1728
12.2. Administrator Exemptions (SNPRM Comments)	1728
12.2.1 Granting of Exemptions is Subject to Bias	1729
12.2.2 Rule Would Preclude Consideration of Best Available Science	1740
12.2.3 Areas Needing Additional Clarity/Specificity	1743
12.2.4 Transparency	1743
12.2.5 Retroactivity	1746
12.2.6 Support for Exemptions	1750
12.2.7 Other Comments	1751
Chapter 13: Peer Review	1754
13.1 NPRM Peer Review-Related Comments	1755
13.1.1 Office of Management and Budget (OMB) Final Information Quality Bulletin for Peer Review	1755
13.1.2 Independent Peer Review Requirements (40 CFR 30.7)	1763
13.1.3 Comparison of the Approach in the Proposed Rule to Existing Scientific Review Processes	1804
13.2 SNPRM Independent Peer Review	1886
13.2.1 Existing Peer Review Processes and Protocols Are Sufficient	1886
13.2.2 Concerns Related to Transparency in EPA's Peer Reviews	1891
13.2.3 Supplemental Proposal Violates the Principles of the OMB Final Information Quality Bulletin for Peer Review	1893
13.2.4 Resource Burdens of Peer Review Requirements	1895
13.2.5 Considerations of Bias and Independence of Peer Reviewers	1896
Chapter 14: Implementation (Including Implementation Benefits and Costs)	1900
14.1 NPRM Implementation	1901
14.1.1 Lack of Clarity How the Rule Would be Implemented	1901
14.1.2 Incorporating Stronger Data and Model Access Requirements in Grants and Cooperative Agreements	1908
14.1.3 Building Upon Other Federal Agencies' Policies Regarding Grantee and Cooperator Requirements	1917
14.1.4 Platform for Increasing Public Access to EPA-Funded Data	1918
14.1.5 Methodologies and Technologies Designed to Provide Protected Access to Data	1924
14.1.6 Other Implementation Issues that May Require Special Consideration	1952
14.1.7 Timing of Implementation	1953
14.1.8 Impacts on other Federal Agencies, States and Tribes	1955
14.1.9 Communications	1956
14.1.10 Implementation Costs	1957
14.2 SNPRM Implementation	1983
14.2.1 Inter-organizational Efforts	1984
14.2.2 The EPA's Readiness to Implement the Rule	1985
14.2.3 Implementation Timing/Process	1985
14.2.4 Implementation Costs and Benefits	1987
14.2.5 Implementation Bias	2001
14.2.6 Rule Includes Insufficient Implementation Detail/Clarity	2002
Chapter 15: Statutory and Executive Order Reviews	2038
15.1 NPRM Executive Orders	2038
15.1.1 Executive Order 12291: Federal Regulation	2038
15.1.2 Executive Orders 12866: Regulatory Planning and Review, and 13563: Improving Regulation and Regulatory Review	2039
15.1.3 Executive Order 12898: Federal Actions to Address Environmental Justice in Minority Populations and Low-Income Populations	2043
15.1.4 Executive Order 13045: Protection of Children	2048
15.1.5 Executive Order 13132: Federalism	2051
15.1.6 Executive Order 13175: Consultation and Coordination with Indian Tribal Government	2051
15.1.7 Executive Order 13556: Controlled Unclassified Information	2052
15.1.8 Executive Order 13771: Reducing Regulation and Controlling Regulatory Costs	2053
15.1.9 Executive Order 13777: Enforcing the Regulatory Reform Agenda	2053
15.1.10 Executive Order 13783: Promoting Energy Independence and Economic Growth	2055
15.2 SNPRM Statutory and Executive Order Requirements	2058
15.2.1 Executive Orders	2058
15.2.2 Paperwork Reduction Act	2075
15.2.3 Additional Related Comments	2076
Chapter 16: Federal Transparency and Data Integrity Laws	2077
16.1 Federal Laws and Law-Related Policies Already Address Data Access Concerns	2078
16.2 Information Quality Act (IQA) and Implementation Guidelines	2080
16.2.1 Information Quality Act	2080
16.2.2 Proposed Rule Consistency with OMB Information Quality Guidelines	2080
16.2.3 EPA Information Quality Guidelines-Specific Comments	2082
16.3 OMB Circular A-110, Freedom of Information Act	2087
16.4 Open Data Policy	2088
Chapter 17: Consistency with Recommendations of Third-Party Organizations Who Advocated for Open Science and Policies Adopted by Major Scientific Journals	2090
17.1 Clarification of "takes into consideration the policies or recommendations"	2090
17.2 Misrepresentation of Third-Party and Major Scientific Journal Recommendations	2092
17.2.1 General Comment Positions on Misrepresentation	2092
17.2.2 Administrative Conference of the United States' Study	2093
17.2.3 National Academies of Science (NAS) Reports	2095
17.2.4 National Research Council Reports	2097
17.2.5 Health Effects Institute (HEI)	2098
17.2.6 Center for Open Science	2099
17.2.7 Risk Assessment Specialty Section of the Society of Toxicology and Dose-Response Specialty Group (DRSG) of the Society for Risk Analysis (SRA)	2101
17.2.8 Bipartisan Policy Center Report	2102
17.2.9 EPA's Proposal is inconsistent with policies adopted by major scientific journals	2104
17.2.10 Not Representative of Scientific Community Recommendations	2109
17.2.11 Cited Economist Article Does Not Support EPA's Proposal	2112
Chapter 18: Alternative Approaches to the Proposed Rule	2113
18.1 Base Policies and Requirements on Existing Guidance	2113
18.1.1 National Academy Guidance	2114
18.1.2 U.S. Commission on Evidence-Based Policymaking Recommendations	2116
18.1.3 Good Laboratory Practice	2117
18.1.4 Transparency and Openness Promotion Guidelines	2119
18.1.5 Radiation Effects Research Foundation	2120
18.1.6 Federal Data Strategy	2120
18.1.7 National Center for Health Statistics Research Data Centers	2121
18.2 Approaches Specific to EPA-Funded Research	2121
18.2.1 Enforce Existing Requirements	2121
18.2.2 Update Grant Awards/Cooperative Agreements	2122
18.2.3 Use EPA Funding to Enact Change	2122
18.3 Suggested Changes to the Rule/Approach	2123
18.3.1 Make the Rule More General	2123
18.3.2 Applicability of Science Covered by the Rule	2124
18.3.3 Additions to the Proposed Rule	2126
18.3.4 Subtractions from the Proposed Rule	2129
18.3.5 No Action Necessary	2130
18.3.6 Delay Rulemaking	2131
18.4 Alternative Policies to Increase Transparency in EPA Rulemaking without Releasing Private Data to Public	2132
18.4.1 Other Effective Ways to Vet Scientific Results that Do Not Require Public Access to Research Data	2133
18.4.2 Weight Studies According to Data Transparency	2135
18.4.3 Encourage Scientific Community-Led Efforts to Self-Improve Transparency	2136
18.4.4 Use Independent Third-Party to Evaluate Study Components, Results, Rules, Data Disclosure Issues	2137
18.4.5 Increase Financial Incentive Transparency	2140
18.4.6 Other	2141
18.5 Facilitate Data Publication/Accessibility	2145
18.5.1 Public Accessibility to Environmental Databases and Websites that are Easy to Navigate	2145
18.5.2 Improve Public Accessibility	2146
18.5.3 Make Use of Existing Data Repository (e.g., Rulemaking Docket)	2147
18.5.4 Transparency of Published Data	2148
18.6 Implement, Emulate or Work with Federal Programs	2148
18.6.1 Environmental Protection Agency Policy and Guidance	2149
18.6.2 Collaborate with Other Agencies/Departments	2151
Chapter 19: Comparison of Approach to Hazard, Exposure and Risk Assessment Guidance	2155
19.1 Science and Judgement in Risk Assessment	2155
19.2 Request for Inclusion of Discussion on How EPA Proposes to Address Exposure Assessments and Risk Characterization Data and Models in the Future	2156
19.3 Request Guidance on How to Use Human Data in Risk Assessments Under the Proposed Rule	2157
19.4 Industry Toxicology Study and Epidemiological Study Review and Acceptance Differ	2157
19.5 Public's Interest Best Served When Science is Replicable, Transparent and Explainable with Other Toxicology Information	2158
Chapter 20: Chemical/Pollutant Specific Comments	2160
20.1 Chlorpyrifos	2160
20.2 Formaldehyde	2161
20.3 Lead	2161
20.4 Particulate Matter	2162
20.5 Trichloroethylene (TCE)	2165
Chapter 21: Miscellaneous and Out of Scope Comments	2168
21.1 NPRM Miscellaneous Comments and Out of Scope Comments	2168
21.1.1 Miscellaneous Comments	2168
21.1.2 Out of Scope Comments	2172
21.2 SNPRM Miscellaneous Comments and Out of Scope Comments	2188
21.2.1 Miscellaneous Comments	2188
21.2.2 Out of Scope Comments	2237

Appendix A. Index of Commenters
Appendix B. Overview of Less-Detailed Comments
 Chapter 1: Legal Authority
In the 2018 Proposed Rule, EPA proposed to take the action under several statutes it administers, including Clean Air Act (CAA) sections 103, 301(a), 42 U.S.C. 7403, 7601(a); Clean Water Act (CWA) sections 104, 501, 33 U.S.C. 1254, 1361; Safe Drinking Water Act (SDWA) sections 1442, 1450(a)(1), 42 U.S.C. 300j-1, 300j-9(a)(1); Resource Conservation and Recovery Act (RCRA) sections 2002(a)(1), 7009, 42 U.S.C. 6912(a)(1), 6979; Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA) (as delegated to the Administrator via Executive Order 12580) sections 115, 311, 42 U.S.C. 9616, 9660; Emergency Planning and Community Right-To-Know Act (EPCRA) section 328, 42 U.S.C. 11048; Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA) sections 25(a)(1), 136r(a), 7 U.S.C. 136r(a), 136w; and Toxic Substances Control Act (TSCA), as amended, section 10, 15 U.S.C. 2609. The Agency also states that the action is consistent with requirements in the Administrative Procedure Act (APA) to ensure public participation in the rulemaking process. Additionally, the notice of proposed rulemaking (NPRM) solicits comment on whether additional or alternative sources of authority are appropriate bases for this proposed regulation.
The supplemental notice of proposed rulemaking (SNPRM) maintains its authority under the statues provided in the original proposal. The Environmental Protection Agency (EPA) also notes its 2018 document extending the comment period and announcing a public hearing identifies 5 U.S.C 301 as a source of authority in addition to the statutes cited in the NPRM. In addition, the Agency clarifies that the NPRM's reference to RCRA section 7009, 42 U.S.C 6979 should be to RCRA section 8001, 42 U.S.C. 6981; the citation to CERCLA section 116, 42 U.S.C. 9616, should be to CERCLA section 115, 42 U.S.C. 9615; and includes CWA section 501, 33 U.S.C. 1361.
The Agency also states that it is authorized to promulgate this regulation under its housekeeping authority. While EPA acknowledges it is not one of the 15 "Executive Departments" listed at 5 U.S.C 101, the Agency provides that it gained housekeeping authority through the Reorganization Plan No. 3 of 1970, 84 Stat. 2086 (July 9,1970). According to the Agency, the proposed rule is an internal rule of agency procedure that is intended to be consistent with the statutes that EPA administers and EPA plans to implement this procedural rulemaking in accordance with all applicable statutory and regulatory requirements. While this rule is of internal agency procedure, EPA solicits comment on whether to use its housekeeping authority independently or in conjunction with the clarified environmental statutory provisions.
Several comments were received regarding the legal authority of the NPRM and SNPRM, including the applicability of the cited environmental statutes and EPA's ability to invoke housekeeping authority for the proposed rule. In this section the public comments are organized according to the following topics:
 General Comments
 Housekeeping Statute
 Specific Environmental Statutes
 Evidence Act (SNPRM Comments)
 Consistency with the Peer Review Processes Required Under Environmental Statute (NPRM Comments)
 Administrative Procedures Act (APA) (NPRM and SNPRM Comments)
 Other Authorities (NPRM Comments)
 Trying to Achieve Administratively What Could Not be Done Legislatively
 Other Comments (NPRM Comments)
The specific comments provided in this Chapter reflect the comment letters that provide arguments supported by detailed rationale, examples, and references/documentation. The Docket ID commenter ID numbers (and specific comment excerpt IDs - indicated in brackets) for these commenters are cited throughout this section. In addition, some general comment letters also include comments related to the legality of the NPRM and SNPRM (the Proposal). These comments are summarized in Appendix B. Note that the points and issues raised in these general comment letters may also be raised by the more detailed comments summarized in this Chapter. Therefore, the responses provided in this Chapter adequately address the more general comments received regarding the legal authority and other issues related to the legality of the rule.
1.1 General Comments
1.1.1 General Legal Authority (Notice of Proposed Rulemaking (NPRM) Comments)
Comment [2772-2261]: Commenter (2787) contends that the EPA fails to cite any specific language providing authority for the rule and asks commenters where the authority may be found.
Response: In both the 2018 proposal and the 2020 supplemental proposal, EPA cited several sources of authority and requested comment on those sources or any others that commenters might think appropriate for the final rule. EPA has identified its housekeeping authority, conveyed through Reorganization Plan No. 3 of 1970, as the legal authority for the final rule.
Comment [4531-5099]: Commenter (4541) applauds this proposed rule and find that governing statutes and Executive Orders, not to mention the basics of the scientific method, authorize the proposed Rule and, indeed, have long required it. In not following the proposed Rule in the past, EPA has been flouting governing statutes and Executive Orders, departing from the scientific method, particularly in matters of toxicology and epidemiology, and abusing its authority by ignoring and discarding valid objections to EPA adopted science and the derivative policy and regulatory actions.
Response: EPA appreciates your comment in support of the final rule but disagrees with your assertion that the agency has flouted the law and ignored or discarded valid objections to agency science. EPA uses systematic review and weight of evidence processes to evaluate scientific studies for use in its regulatory decisions and in the development of influential scientific information.
Comment [4871-354]: Commenter (4882) agrees this proposed rule has incorporated appropriate sources of statutory authority and suggest no further source of authority is required.
Response: Thank you for your comment in support of the final rule.
Comment [6106-875]: Commenter (6116) states that on April 30, 2018, EPA published a notice in the Federal Register (FR) of a proposed rule that would drastically reduce the types of scientific studies that can be used to inform EPA regulations protecting public health under the guise of improving transparency. The commenter expresses that they are troubled by this proposal, as we believe it would vastly undermine the mission of the EPA, which is to protect human health and the environment. The commenter notes that it is curious that this proposal champions "transparency" when it seems certain to serve the opposite purpose. An example of one of the reasons the commenter states that they oppose the proposed rule includes, but is not limited to the following: 
      The EPA itself seems to question whether it has the authority to promulgate this rule. The FR notice asks commenters to provide information on what areas of the CAA they feel may demonstrate this type of authority. The EPA should have a solid foundation before it proposes rules and should not have to seek answers for questions about its own authority.
Response: See response to Comment 2772-2261.
Comment [6112-1489]: Commenter (6137) contends that EPA has no authority to limit what scientific information may be considered in making regulatory decisions. The commenter states that no statute authorizes the proposed rule, and EPA lacks any inherent authority to regulate absent a statutory basis. Thus, that commenter suggest that the EPA should proceed to promulgate the Proposed Rule or otherwise limit what science can be considered in regulatory decisions, it will be acting in violation of the law. See 5 U.S.C. § 706(2)(C) (requiring a reviewing court to "hold unlawful and set aside agency action . . . found to be . . . in excess of statutory jurisdiction, authority, or limitations, or short of statutory right").
The commenter states that the stated statutory provisions upon which EPA relies do not provide authority for the Proposed Rule. The commenter notes that the EPA lists a number of statutes it administers as purported authority for this rule, including "provisions providing general authority to promulgate regulations necessary to carry out the Agency's functions under these statutes and provisions specifically addressing the Agency's conducting of and reliance on scientific activity to inform those functions." See 83 Fed. Reg. 18,769. As discussed in detail below, none of the statutes cited by EPA authorizes this proposed action. EPA thus lacks authority to promulgate this rule under any statutory regime administered by the Agency, rendering the rule invalid.
Response: In both the 2018 proposal and the 2020 supplemental proposal, EPA cited several sources of authority and requested comment on those sources or any others that commenters might think appropriate for the final rule. EPA has identified its housekeeping authority, conveyed through Reorganization Plan No. 3 of 1970, as the legal authority for the final rule.
Also see response to Comment 73-410 in this Response to Comment (RTC) document Section 1.2.2.
Comment [6112-1560]: Commenter (6137) provides that in its notice extending the comment period and adding a public hearing, as an implicit admission that it has not cited sufficient authority for the Proposed Rule, EPA adds a new source of alleged authority, stating that "EPA is proposing this rule under authority of 5 U.S.C. 301, in addition to the authorities listed in the April 30th document." See 83 Fed. Reg. 24,255, 24,256 (May 25, 2018). Just like with the other statutory provisions upon which it relies, EPA is trying to fit a square peg in a round hole.
Section 301 of Title 5 provides "[t]he head of an Executive department or military department" authority to "prescribe regulations for the government of his department, the conduct of its employees, the distribution and performance of its business, and the custody, use, and preservation of its records, papers, and property." This provision governs internal organizational and bureaucratic steps required for operations. It allows all agencies to issue regulations to preserve and use their own papers and property. This section is plainly focused on allowing executive agencies to issue rules necessary to carry out the performance of their agencies' internal workings, not to allow EPA to regulate scientific material in rulemakings.
Indeed, according to the commenter, the "purpose" of this section, "which originated in 1789 as a law `to enable General Washington to get his administration underway by spelling out the authority of Government officers to set up offices and to file Government documents'. . . is to set up merely internal guidelines for a given governmental agency" to perform its job. United States v. Lewis, No. C-CR-89-114-01, 1990 WL 11111, *5 (W.D. N.C. Feb. 5, 1990) (citation omitted). That is why it is known as the "Housekeeping Statute," to literally allow the federal government to set up and keep house. U.S. ex. Rel. O'Keefe v. McDonnell Douglas Corp., 132 F.3d 1252, 1254 (8th Cir. 1998) (citing H.R. Rep. No. 85-1461 (1958), reprinted in 1958 U.S.C.C.A.N. 3352). The Act was amended in 1966 as "codifying the general and permanent laws relating to the organization of the Government of the United States and to its civilian officers and employees." Pub. L. No. 89-554, 80 Stat. 378 (Sept. 6, 1966).
In Chrysler Corp. v. Brown, 441 U.S. 281, 310 (1979), the Supreme Court evaluated the Housekeeping Statute and held that it does not provide statutory authority for substantive regulations. After a brief historical analysis of the provision, the Court wrote:
Given this long and relatively uncontroversial history, and the terms of the statute itself, it seems to be simply a grant of authority to the agency to regulate its own affairs. . . It is indeed a "housekeeping statute," authorizing what the APA terms "rules of agency organization procedure or practice" as opposed to "substantive rules."
Id. at 309 - 10. Multiple courts have agreed and limited rulemaking under this provision to non-substantive rules. See, e.g., McDonnell Douglas Corp., 132 F.3d at 1256 (citing examples).
Based on this long line of authority, EPA's reliance on this authority is sorely misplaced. EPA's attempt to "construe [this provision] as something more" is a "misuse" that "twist[s]" the statute beyond its intended purpose; EPA may not "twist this simple administrative statute into an authorization for the promulgation of substantive rules." Id. at 1255 (citing and quoting Chrysler Corp., 441 U.S. at 310 n.41 (quoting H.R. Rep. No. 85 - 1461 at 7 (1958)). 
The Proposed Rule does not relate to the organization of EPA or how it preserves its papers or keeps house. EPA's exclusion of critical health studies is such a far cry from being necessary to "set up offices" and to "file Government documents," that reliance on this provision hardly passes the laugh test. As discussed extensively in these Comments, the Proposed Rule is by no means necessary for EPA to perform its job but rather is antithetical to the very statutes it is responsible for effectuating. Accordingly, for the same reasons the general rulemaking authority provisions under all of the environmental statutes EPA cites do not authorize this rule, § 301 likewise does not permit EPA to issue a rule that undermines scientific integrity as well as all of the public health and environmental protections EPA is charged with enforcing.
The EPA's lack of statutory authority to propose this rule is fatal, as it has no inherent power to act. Indeed, it is well settled that a federal agency "literally has no power to act . . . unless and until Congress confers power upon it." La. Pub. Serv. Comm'n v. FCC, 476 U.S. 355, 374 (1986); see also Bowen v. Georgetown Univ. Hosp., 488 U.S. 204, 208 (1988) ("It is axiomatic that an administrative agency's power to promulgate legislative regulations is limited to the authority delegated by Congress."); Clean Air Council v. Pruitt, 862 F.3d 1, 9 (D.C. Cir. 2017) ("[I]t is `axiomatic' that `administrative agencies may act only pursuant to authority delegated to them by Congress'" (citation omitted)); Ohio Dep't of Medicaid v. Price, 864 F.3d 469, 476 (6th Cir. 2017) ("Agencies, after all, are creatures of statutory authority." (citation and internal quotations omitted)). This is because, under the Constitution, Congress is the branch of government with lawmaking power. City of Arlington, Tex. v. FCC, 569 U.S. 290, 317 (2013) (noting that an agency has no lawmaking power unless Congress delegates that power to it). "The legislative power of the United States is vested in the Congress, and the exercise of quasi-legislative authority by governmental departments and agencies must be rooted in a grant of such power by the Congress and subject to limitations which that body imposes." Chrysler Corp., 441 U.S. at 302. Thus, EPA only has rulemaking power to the extent that Congress delegated it such power. Lyng v. Payne, 476 U.S. 926, 937 (1986) ("an agency's power is no greater than that delegated to it by Congress.")
Moreover, the fact that Congress has given EPA the authority to regulate in a certain area does not mean that it has general authority to make any rule within that area. This argument has been squarely rejected:
The [agency's] position in this case amounts to the bare suggestion that it possesses plenary authority to act within a given area simply because Congress has endowed it with some authority to act in that area. We categorically reject that suggestion. Agencies owe their capacity to act to the delegation of authority from Congress. 
Am. Library Ass'n. v. FCC, 406 F.3d 689, 708 (D.C. Cir. 2005) (quoting Ry. Labor Executives' Ass'n v. Nat'l Mediation Bd., 29 F.3d 655, 670 (D.C. Cir. 1994)) (internal quotations omitted; emphasis in original); see also Liberty Mut. Ins. Co. v. Friedman, 639 F.2d 164, 169 (4th Cir. 1981) ("a court must reasonably be able to conclude that the grant of authority contemplates the regulations issued." (citation and internal quotation marks omitted)). "[T]he power to issue regulations is not the power to issue any regulations." Nat'l Mining Ass'n v. U.S. Dep't of the Interior, 105 F.3d 691, 694 (D.C. Cir. 1997). In light of the statutory limitation on EPA's authority to restrict science in the way it proposes to do (see infra, Section IV of the commenters' comment letter), any general rulemaking authority on which it might otherwise try to rely does not authorize the Proposed Rule. See, e.g., Nat. Res. Def. Council, Inc. v. Reilly, 976 F.2d 36, 40-41 (D.C. Cir. 1992) (refusing to allow EPA to rely on general rulemaking authority to trump specific limitations on its authority because a vague "open-ended power" does not "trump the specific provisions of the [Clean Air] Act"; and "EPA's construction of the statute is condemned by the general rule that when a statute lists several specific exceptions to the general purpose, others should not be implied." (citation and internal quotation marks omitted).
Accordingly, EPA has no general or inherent authority permitting it to lawfully adopt the Proposed Rule or otherwise limit what science may be considered in the rulemaking process. It, therefore, must be acting pursuant to some grant of authority by Congress for the Proposed Rule to be lawful. Yet none of the stated authorities upon which EPA relies provides the necessary authority to promulgate this rule. Given that EPA has no statutory authority to issue the Proposed Rule, its action is ultra vires. See, e.g., McDonnell Douglas Corp., 132 F.3d at 1257 ("An agency's promulgation of rules without valid statutory authority implicates core notions of the separation of powers, and we are required by Congress to set these regulations aside." (citing cases finding ultra vires agency action)). The Proposed Rule is therefore unlawful.
Response: See responses to Comments 31-135, 32-144 and 73-410 in RTC document Section 1.2.2.
Comment [6116-1117]: Commenter (6141) asserts that the Proposal for scientific transparency is consistent with the objectives and requirements of existing federal environmental statutes and administrative policies. Most importantly, the CAA and other major federal environmental statutes provide EPA with broad authority to establish procedures, requirements, and criteria for adopting rules, regulations, and policies for how the Agency adopts new regulations, rules, and guidance. For example, section 301(a) of the CAA provides the EPA Administrator with broad authority to establish regulations "as are necessary to carry out... the functions" under the CAA, while section 307(d) provides extensive authority on the manner in which EPA may conducting notice and comment rulemakings, including provisions for ensuring a robust rulemaking record and adequate opportunities for public participation. The other relevant federal environmental statutes also provide EPA broad authority for the establishment of procedures, requirements, and criteria for how it establishes implementing rules, regulations, and policies. The commenter supports EPA's proposal to use this authority to establish new rules for ensuring the transparency and public availability of the underlying scientific data and other important supporting documentation used by EPA in its regulatory actions.
Response: Thank you for your comment in support of the rulemaking. See responses to Comments 31-135 and 73-410 in Section 1.2.2 of the RTC document.
Comment [6125-1849]: Commenter (6150) agrees that EPA has sufficient legal authority under existing environmental statutes to issue this rulemaking to strengthen the transparency of the science supporting regulatory activity. EPA lays out substantive legal support for this action in the Proposed Rule. Further, EPA's obligation under the Data Quality Act is to "maximiz[e] the quality, objectivity, utility, and integrity of information" and, as importantly, "establish administrative mechanisms allowing affected persons to seek and obtain correction of information maintained and disseminated by the agency." These provisions can only be met if the public has access to the data, models, and methodologies used by EPA and underlying research used by EPA to justify new and existing regulations.
Response: Thank you for your comment in support of the rulemaking.
Comment [6149-2754]: Commenter (6174) provides that there is no basis in existing bedrock environmental laws that authorizes EPA to limit science considered in rulemaking processes. EPA cites several key laws as a justification for this current proposal to limit or constrain the science it can consider. Nowhere in the cited statutes is there any basis for demanding access to raw data, nor does this relate sensibly to any definition of best available science. Rather it undermines the use of best available science as called for in numerous environmental statutes including the CAA. Further, there is no basis in the statutes for politically appointed administrators to choose which science will be considered and which may not be.
The CAA, for example, has requirements to update pollution standards that provide for an adequate margin of safety for public health. This determination can only be reliably made using the best available science. However, this proposal would prevent EPA from using the best available information to set science-based pollution standards that would provide for an adequate margin of safety. EPA should not cite the CAA as an appropriate source of statutory authority.
In sum, there is no statutory authority for EPA to rely on to censor or constrain science. Indeed, quite the opposite. Further, such an effort would only serve to undermine EPA's essential role.
Response: EPA is not relying on the substantive environmental statutes as authority for the final rule. See responses to Comments 32-144, 48-381 and 73-410 in RTC document Section 1.2.2.
Comment [6155-586]: Commenter (6125) states that EPA lists 14 statutory provisions as support for its proposal. On examination, these are all either general rulemaking authorities or general research authorities; none of them has anything to do with how to use science in decision-making. Moreover, the proposal does not expressly say--as a well-drafted and genuinely transparent proposal should-- that it will only apply to the programs under the cited statutes (although that may be implicit), much less discuss in any way the policies those programs embody or why the proposed per se rule would be a proper exercise of discretion under each statute. Indeed, the cited provisions, virtually the only statutory authority the proposal cites as support, are irrelevant to the per se issue.
Response: EPA is not relying on the substantive environmental statutes as authority for the final rule. See response to Comment 73-410 in RTC document Section 1.2.2.
Comment [6155-588]: Commenter (6125) contends that any rule must have a lawful basis. The proposal does not even attempt to show that it does. It claims "discretionary authority" for its proposed approach but identifies no source for such authority. It evades that question by purporting to list "statutes and provisions ​specifically addressing the agency's conducting of and reliance on scientific activity to inform those functions" FR 18769 (emphasis added). It does not explain how those provisions authorize the wholesale redefinition of science. All the provisions cited either authorize research, or general rulemaking not tied to any specific purpose.
EPA's proposal would, by operation of law, change the standards for using science in issuing covered rules under ​all of the listed substantive regulatory authorities. It would thereby make in advance a critical part of the regulatory decision in any covered proceeding, namely the decision how to weigh available studies to best evaluate scientific evidence. This is a decision that has previously been made by detailed case-by-case review in the rulemaking itself. Until today, such decisions have been made after considering the substantive goals of the particular statutory provision involved, guided by the attitude toward scientific evidence embodied in that provision, and striving for conformity with any applicable procedural requirement. The affected provisions are the relevant statutory authorities that EPA's proposal should have cited and discussed, in order to show its consistency with their mandates. EPA's citations to general provisions are insufficient because it is axiomatic that a "general grant of authority cannot trump specific statutory provisions". NRDC v. EPA, 749 F. 3d 1055, 1064 (D.C. Cir. 2014); API v. EPA, 52 F. 3d 1113, 1119 (D.C. Cir. 1995) (same).
The agency's failure to do this is by itself a fatal flaw in this proposal. It is EPA's duty, not EPN's, to identify and discuss these provisions. However, it is clear from even a brief look that these provisions require EPA to consider all the best available science, without authorizing, or even mentioning any of the per se restrictions based on "transparency" that EPA has proposed to use as a universal overriding rule. For these reasons, EPA must justify its proposal as a proper exercise of its authority under each of the statutory provisions whose operation it would affect - provisions that the agency does not even cite. It is the job of the proposer, not the commenters, to survey these provisions and provide the needed justification.
In several of the listed statutes Congress has explicitly told EPA how to use science in decision-making.
 SDWA 1412(b)(3)(A) addresses EPA's "use of science in decision-making" by providing that risk assessments "shall" use "the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices."
 The EPCRA, 42 USC 11023(d)(2)(C) mandates that a determination whether to add a substance to list of "extremely hazardous substances" "shall be based on generally accepted scientific principles or laboratory tests or appropriately designed and conducted epidemiological or population studies available to the Administrator."
 The CWA, 33 USC 1314(a)(1) mandates that the Administrator "shall develop and publish ... criteria for water quality accurately reflecting the latest scientific knowledge" "on the kind and extent of all identifiable effects on health or welfare" in setting water quality standards;
 CAA 108(a)(2) specifies that "air quality criteria for an air pollutant ​shall accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air."
 The TSCA 15 USC Section 2605 (h) provides that "the Administrator shall use scientific information, technical procedures, measures, methods, protocols, methodologies, or models, employed in a manner consistent with the best available science."
These provisions all use mandatory language directing EPA to use reliable and relevant scientific information -- and further specify how to do so. They all direct that decisions regarding protection of public health be based on the full range of relevant scientific information, in some cases specifying how to identify what information to use. None of them give EPA discretion to ignore such information for any of the reasons set forth in the proposal.
Response: EPA is not relying on the substantive environmental statutes as authority for the final rule. See response to Comment 73-410 in RTC document Section 1.2.2.
Comment [6157-898]: Commenter (6127) provides that there is no legal or environmental basis for the agency to add restrictions and ignore available information about the risks of pollution.
Response: The final rule does not require EPA to ignore available information on pollution risks. Under the rule, EPA will give greater consideration to pivotal science based on an evaluation of study quality factors and data availability. See responses to Comments 32-144, 34-155 and 48-381 in RTC document Section 1.2.2.
Comment [6163-827]: Commenter (6133) states that EPA has unsurprisingly failed to cite a single statute that provides any basis for the Proposal. What statutes EPA does cite conflict with the Proposal, because they require EPA either to consider the best available science (which may be based on data that cannot be made public) or to regulate to protect public health and the environment (which cannot be done if critical science is ignored simply because the underlying data cannot be made public). Similarly, none of the other sources EPA cites provide legal or logical support for the Proposal.
Response: EPA is not relying on any of the substantive environmental statutes cited in the 2018 and 2020 proposals as authority for the final rule. See response to Comments 32-144, 48-381 and 73-410 in RTC document Section 1.2.2.
Comment [6163-1046]: Commenter (6133) asserts that it is clear from the above that the Proposal violates the law and must be withdrawn. There is no support for the Proposal in any of the statutes EPA cites, and in fact, those statutes conflict with the Proposal, as do other statutes that EPA failed to mention at all. Further, none of the other sources cited provide legal or logical support for the Proposal. The Proposal also suffers from a host of other problems: its definitions are vague; it is an unexplained reversal from prior agency policy; it handles Confidential Business Information (CBI) in a capricious manner; it treats other types of agency actions inconsistently; and it fails to analyze disproportionate impacts on communities of color, low-income communities, and children.
In the alternative, if EPA decides to move ahead with this reckless, unjustified, and unlawful effort to censor the science that EPA may consider, and must consider, to protect Americans' health and environment, the agency must first issue a supplemental proposal and actual administrative record to cover the multitude of issues, evidence, and specific regulatory text for which EPA fails to provide fair notice. The Proposal fails to provide fair notice or justifications addressing numerous issues that our comments detail -- from an absence of any statutory authority, to failures to address statutory authorities that the Proposal squarely contravenes, to failures to provide reasoned explanations, including basic justifications for EPA's numerous departures from past practices. The Proposal fails to propose specific regulatory text addressing numerous implementation elements, as well as issues that are touched upon only in passing in the preamble (e.g., non-linearity, and linear, non-threshold (LNT)). Apart from all of the significant substantive and procedural defects from which the Proposal suffers, it still manages to be a shockingly shoddy effort missing actual regulatory text and supporting legal, factual, scientific, and technical information that would provide fair notice to the public.
Response: EPA disagrees that the final rule violates the law. Further, EPA addressed many of the significant comments on the 2018 proposal in the 2020 supplemental proposal. Both notices included regulatory text and provide fair notice to facilitate public comment. Also see responses to Comments 31-135 and 73-410 in RTC document Section 1.2.2.
Comment [6163-1101]: Commenter (6133) states that the law is clear that EPA may adopt rules only if those rules are based on statutory authority delegated by Congress. EPA may not invent statutory authority where none exists, nor adopt regulations lacking statutory authority merely because EPA believes that to be better policy. See, e.g., Massachusetts v. EPA, 549 U.S. 497, 535, 127 S. Ct. 1438, 1463 (2007) ("EPA must ground its reasons for action or inaction in the statute."); La. Pub. Serv. Comm'n v. FCC, 476 U.S. 355, 374 (1986) (stating "agency power to act" is shaped by how "Congress confers power upon it"). Agencies need especially clear congressional delegations of authority to create regulatory exemptions. See New York v. U.S. EPA, 413 F.3d 3, 41 (D.C. Cir. 2005) (stating that the agency needs "clear congressional delegation" to support an exemption). EPA identifies no such delegations, certainly not the clear delegations required by law, for the Proposal.
EPA lists seven statutes as the basis for the Proposal. But none of the various statutes cited provides support for the rule's provisions, definitions, requirements, or exemptions. Rather, EPA invents statutory authority where none exists, and creates proposed regulatory text out of thin air. In most cases, EPA simply cites its general authority for rulemaking under the statutes. But that general authority alone cannot provide a basis for the rule, especially when, as explained in section V, the rule would conflict with the requirements of each of the statutes. See New York v. U.S. EPA, 413 F.3d 3, 40 - 42 (D.C. Cir. 2005). In other instances, it appears that EPA just searched the statutes for the word "research" and then cited those sections without any further analysis.
Response: EPA is not relying on any of the environmental statutes cited in the 2018 and 2020 proposals as authority for the final rule. See response to Comment 73-410 in RTC document Section 1.2.2.
Comment [6163-5113]: Commenter (6133) contends that there is no statutory authority for EPA to "conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the Office of Management and Budget (OMB) Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein." The federal statutes that EPA lists as putative authority for the Proposal provide no authority for proposed § 30.7. See 83 Fed. Reg. at 18,769/2 (citing CAA sections 103, 301(a), 42 U.S.C. 7403, 7601(a); CWA sections 104, 501, 33 U.S.C. 1254, 1361; SDWA sections 1442, 1450(a)(1), 42 U.S.C. 300j - 1, 300j -  9(a)(1); RCRA sections 2002(a)(1), 7009, 42 U.S.C. 6912(a)(1), 6979; CERCLA (as delegated to the Administrator via Executive Order 12580) sections 115, 311, 42 U.S.C. 9616, 9660; EPCRA section 328, 42 U.S.C. 11048; FIFRA sections 25(a)(1), 136r(a), 7 U.S.C. 136r(a), 136w; and TSCA, as amended, section 10, 15 U.S.C. 2609, and the APA). The claimed authorities that EPA lists do not mention peer review or even allude to the concept. Id. Neither the Proposal nor accompanying docket materials identify any provision of any federal statute that authorizes EPA to promulgate proposed § 30.7. Our own research revealed no provision of any federal statute that authorizes EPA to promulgate proposed § 30.7. 
Response: None of the statutory provisions cited by the commenter prohibits EPA from conducting additional peer review of studies the agency evaluates as support for significant regulatory decisions and ISI. That said, the final rule provides that if EPA determines that studies have already undergone adequate peer review, EPA will not conduct de novo independent peer review. Adequate peer review is peer review that is consistent with both OMB's and EPA's peer review guidance. The intent of the rule is for EPA to go beyond current OMB/EPA guidance by recognizing that peer review that actually examined the underlying data will lead to even more confidence in the results of a peer reviewed study and that pivotal science that has had this deeper peer review will be given greater consideration in study quality weighting. 
The final rule states in § 30.3 that "[i]n the event the procedures outlined in this rule conflict with statutes that EPA administers, or their implementing regulations, the statutes and regulations will control." Additionally, EPA intends to promulgate future science transparency rules under its substantive environmental statutes that may address peer review requirements consistent with those authorities. 
Comment [6428-1351]: Commenter (6450) states that the proposed rule is not merely a disastrous policy choice, given its threats to public health and the integrity of EPA decision-making, but it is plainly unlawful. EPA clearly lacks the authority to issue the proposal. EPA cites to various provisions in federal law to justify its authority but, in virtually every case, the agency refers to sections that authorize or mandate EPA to undertake research, not to impose arbitrary limitations on the scientific data that informs public health decisions. EPA also cites to statutory provisions authorizing it to promulgate rules "necessary" to achieving the goals of relevant statutes but restricting the use of sound science is neither necessary nor consistent with the statutory goals of protecting clean air, clean water, and a safe environment. Indeed, the proposed rule directly contradicts the specific mandate of numerous federal laws, such as the SDWA and the TSCA, which require EPA to use the "best available" science or all "reasonably available" science and information in its decision-making.
Response: EPA is not relying on any of the substantive environmental statutes cited in the 2018 and 2020 proposals as authority for the final rule. See response to Comment 73-410 in RTC document Section 1.2.2.
Comment [6894-3629]: Commenter (6916) notes that the APA further requires that a notice of proposed rulemaking contain "reference to the legal authority under which the rule is proposed." 5 U.S.C. § 553(b)(2). "`[T]he required specification of legal authority must be done with particularity,'" and "`must be sufficiently precise to apprise interested persons of the agency's legal authority to issue the proposed rule.'" Global Van Lines, 714 F.2d at 1298 (quoting H.R. Rep. No. 1980, at 24 (1946) and U.S. Dept. of Justice, Attorney General's Manual on the APA 29 (1947)). EPA has also failed to meet this requirement.
In both the April 30, 2018 notice of proposed rulemaking and the May 25, 2018 notice extending the comment period, EPA discusses statutory authority for the proposed rule, citing to a number of provisions, largely from statutes it implements. 83 Fed. Reg. at 18,769; 83 Fed. Reg. 24,255, 24,256 (May 25, 2018). In particular, EPA invokes the CAA, CWA, SDWA, EPCRA, FIFRA, TSCA, CERCLA, and RCRA. 83 Fed. Reg. at 18,769.
But rather than identify legal authority with particularity, the cited statutory provisions mainly set forth EPA's broader authorities to conduct research and promulgate regulations. Few of the cited provisions actually address EPA's ability to pick and choose amongst valid scientific information, studies, and techniques in its formation of environmental standards and modeling, and none authorize the wholesale preclusion of probative, relevant studies, as EPA proposes here. Tellingly, in the proposal itself, EPA requests assistance to determine "whether additional or alternative sources of authority are appropriate bases for the proposed regulation." Id. at 18,771. EPA's inability to identify specific statutory authority for its proposed action falls far short of the APA's standard for notice and comment rulemaking, as would any ultimate reliance on statutory authority EPA has failed to cite. See Global Van Lines, 714 F.2d at 1297-99.
Response: See response to Comment 25-96 in the APA Section 1.6.1.2 of the RTC document.
Comment [6895-3186]: Commenter (6919) contends that EPA, like all federal agencies, is a creature of statute, and "may act only pursuant to authority delegated ... by Congress." But the Proposal fails to state clearly under what authority EPA is proposing these rules, providing only a list of statutes that require the Agency affirmatively to take scientific information into account, but not explaining why they would offer the Agency authority to disregard certain relevant studies. Indeed, EPA seems unsure of its authority, requesting comment on "additional or alternative sources of authority."
Nor does the EPA Administrator's authority "to prescribe such regulations...as are necessary to carry out the Agency's functions" present an opening to disregard relevant information, as EPA suggests. To the contrary, courts have held that the Administrator must take into account all relevant studies in the record. Other CAA sections, not cited by the Agency, lay out and define some of the Administrator's rulemaking "functions under the Act" as requiring for example, the use of "the latest scientific knowledge useful in indicating the kinds and extent of all identifiable effects on public health or welfare..." So, the Proposal to permit him to disregard some studies is contrary not only to his general rulemaking authority, but to his specific authority to, inter alia, set the National Ambient Air Quality Standards (NAAQS) based on the "latest scientific knowledge."
The seminal CAA air quality standard-setting provisions have twice been held not to require the release of data underlying studies relied on by the Agency, as discussed above. Indeed, in both situations, the court has described such release as "unnecessary and unjustified."
The Proposal also is inconsistent with the overall precautionary nature of the CAA. Congress did not intend EPA to limit itself to considering only `sure things' but asks the Administrator to protect the public against uncertain dangers due to exposure to air pollution. Primary NAAQS are to be set with an "adequate margin of safety," meaning that "the Administrator need not regulate only the known dangers to health but may `err' on the side of overprotection." This precautionary approach is reflected also in the various technology-forcing sections of the Act, which Senator Muskie said "may [require] that people and industries will be asked to what seems to be impossible at the present time." Under this scheme, the Administrator must consider all relevant studies placed in the record, and "may apply his expertise to draw conclusions from suspected, but not completely substantiated, relationships between facts, from theoretical projections from imperfect data, from probative preliminary data not yet certifiable as `fact' and the like," in order to protect public health as Congress intended. This context simply does not authorize the Administrator to decline to consider any studies  -  but particularly not those representing the best available science  -  because the underlying data is confidential.
Response: EPA is not relying on any of the substantive environmental statutes cited in the 2018 and 2020 proposals as authority for the final rule. See response to Comment 73-410 in RTC document Section 1.2.2.
Comment [6896-3231]: Commenter (6916) provides that the Proposal claims to be "consistent with the principles underlying the APA". But, the Proposal does not clearly identify the authority under which it is promulgated, is so vague as to undermine the ability to meaningfully comment, and in its substance contradicts the APA requirement that agencies must consider all the relevant matter presented in a rulemaking record. 
As discussed above, the CAA does not authorize the Proposal, which is clearly aimed at limiting the studies EPA can rely on and consider in CAA rulemaking. Additionally, APA section 553(b)(2) requires a proposed rule to provide a "reference to the legal authority under which the rule is proposed." While the Proposal's preamble lists various statutes the Agency claims support its action, a closer look reveals that none of those sections actually authorizes EPA to disregard scientific studies. Certainly that is the case for the CAA sections listed by the Agency in the Proposal, including the general rulemaking authority under the Act which requires the evaluation of all material in the record. EPA's suggestion that it has "general authority" to make rules to comply with its obligations is similarly unavailing here, as the general authority does not exist to make rules that are outside EPA's statutory obligations, or in violation of them  -  and a rule that would enable EPA to disregard relevant studies in a CAA rulemaking record is clearly outside the Agency's authority. Nowhere does the Act permit EPA to limit its own access to relevant science. 
Significant issues are presented in the Proposal only in the broadest terms, despite an APA requirement that a proposed rule includes "the terms and substance of the proposed rule or a description of the subjects and issues involved," 5 U.S.C. § 553(b)(3), and with enough specificity "to permit interested parties to comment meaningfully." As noted supra [in their comment letter], we assume (as we must, given no other information from the Agency), that the Proposal is aimed at precluding EPA's reliance on studies for which the data are not made public. That is the natural reading of the proposed regulatory language, which is the most choate information offered in the Proposal. But even that reading is not certain, given the very broad discretionary nature of the exemptions provided for in the rule text (see, e.g., 83 Fed. Reg. 18,774, proposed 40 C.F.R. § 30.9, giving the Administrator the ability to grant an exemption to the data publication requirement if that is "impracticable," which is undefined in the Proposal). As a result, it is unclear how this rule would apply to EPA decision making, and how it would be different than the Agency's current protocol for considering studies relevant to specific rulemaking proposals. 
As one example, EPA alludes to "transparency" multiple times, but without defining what that concept means, or explaining why it believes existing statutes, regulations, guidelines and protocols fail to meet it. While asserting an interest in "ensur[ing] that the data and models underlying science is [sic] publicly available in a manner sufficient for validation and analysis," there is no more specific statement explaining how that would be implemented or why releasing confidential health and business information is necessary. Nor is the Agency's rationale completely clear in the Proposal. When commenters are left to guess, that is not lawfully sufficient notice of the subjects and issues involved. 
Section 553(c) of the APA requires that when finalizing a proposed rule, and after taking comment, an agency must "consider[] the relevant matter presented" in the record. To the extent that the Proposal envisions authorizing EPA to ignore or disregard or not to consider some studies submitted during a rulemaking comment period, such Agency action would be taken in direct violation of this provision of the APA. Nor is there any basis in the APA for EPA to decide to exclude or prohibit the consideration of otherwise relevant matter because it is based on confidential information not available to the general public.
Response: See response to comment 25-96 in APA Section 1.6.1.2 of this document. 
Comment [6900-3485]: Commenter (6921) contends that while EPA has sufficient legal authority across all of its environmental statutes to issue this important rulemaking, the Forum believes EPA can strengthen the foundation of the rule by including statute-specific provisions or by supplementing the rule with statute specific rulemakings. This will allow EPA to link the transparency requirements to the specific regulatory authorities in each environmental statute and to tailor the data access requirements, if necessary, to the specific rulemaking and data gathering provisions.
Response: Thank you for your comment in support of the final rule. EPA intends to promulgate future transparency rules under the substantive environmental statutes that it administers. Those rules will tailor transparency requirements as appropriate for each statute. See response to Comment 73-410 in RTC document Section 1.2.2.
Comment [8750-1430]: Commenter (9227) asserts that agencies are creatures of Congress; "an agency literally has no power to act . . . unless and until Congress confers power upon it." Louisiana Pub. Serv. Comm'n v. FCC, 476 U.S. 355, 374 (1986); see Am. Library Ass'n v. FCC, 406 F.3d 689, 691 (D.C. Cir. 2005) ("It is axiomatic that administrative agencies may issue regulations only pursuant to authority delegated to them by Congress."). EPA points to a smattering of statutes as allegedly authorizing the Proposal. None of these authorities, however, authorize EPA to promulgate a one-size-fits-all regulation governing how the agency will consider science under its various statutory authorities, which is perhaps why EPA solicits comment on whether additional authorities might exist to authorize its Proposal. The varied statutes that the Proposal cites have different requirements as to the agency's obligations when considering science. Compare CAA § 108(a) (standards must "reflect the latest scientific knowledge useful in indicating" health and welfare effects) with TSCA § 4(f) (Administrator must consider "any other information available") with SDWA § 1412(b)(1)(B)(ii)(II) (Administrator must consider "the best available public health information"). The Proposal gives no explanation of how any of the provisions it cites provide authority for the Proposal, much less how all of them authorize identical requirements.
For example, EPA cites the CAA, § 301, 42 U.S.C. § 7601, as purportedly granting authority for the Proposal. The authority granted by section 301(a), however, applies only to the CAA and, in any event, is not broad enough to encompass this Proposal. Section 301 provides that "[t]he Administrator is authorized to prescribe such regulations subject to section 307(d) as are necessary to carry out his [or her] functions under this Act." The courts have consistently "decline[d] to read ... open-ended power into section 301," and instead have required that regulations promulgated under section 301 be both necessary and appropriate. As discussed in more detail below, EPA's Proposal here is not necessary, and instead directly conflicts with several other provisions of the CAA. It is axiomatic that a "general grant of authority cannot trump specific statutory provisions."
Nor does Congressional authorization to conduct or fund research authorize EPA to ignore research in regulatory decision-making. Accordingly, provisions like TSCA § 10, which directs that the "Administrator shall ... conduct such research, development, and monitoring as is necessary to carry out the purposes of this [Act]," and CAA § 103, which authorizes the agency to conduct and support research, plainly do not authorize the Proposal. 
Response: EPA is not relying on any of the substantive environmental statutes cited as authority in the 2018 and 2020 proposal. See response to Comment 73-410 in RTC document Section 1.2.2.
Comment [9066-1894]: Commenter (6188) notes that EPA cites numerous statutory provisions to support this rulemaking, but none provides authority for the agency to promulgate the Proposed Rule.
Generally, the statutory provisions EPA invokes fall into one of two categories: (1) provisions authorizing EPA to conduct research in furtherance of statutory objectives and (2) general provisions authorizing the EPA Administrator to promulgate regulations as necessary to achieve the purposes of a given statute. None of these statutory references provides the requisite authority for adoption and implementation of the Proposed Rule.
As a basis for authority to promulgate the Proposed Rule, EPA points to provisions authorizing the establishment of research and development programs pursuant to each of the federal environmental laws EPA administers. But these statutory references are unavailing. Each statute directs EPA to set up research programs and to undertake specific activities attendant to the administration of those programs, but none governs -- or even references -- the extent to which research should be used in regulatory decision-making by EPA. Further, any regulatory authority EPA may have under the referenced provisions is limited to the individual research and development programs in question and does not extend to unrelated research by outside parties. In other words, the cited provisions allow EPA to set up its own research programs, but do not create authority to place limitations on how research, whether conducted or financed by EPA or produced by an outside party, is used to set regulatory standards.
For example, EPA purports to derive authority for the Proposed Rule from CAA § 103. That provision simply authorizes EPA to establish a national research and development program for the prevention and control of air pollution and, as part of that program, to conduct and promote the coordination of research and studies relating to the causes, effects, extent, prevention, and control of air pollution. 42 U.S.C. § 7403(a)(1). The section authorizes specific activities of the Administrator in establishing such a program, none of which includes limiting the scope of reviewable data, research, or studies when undertaking regulatory action. The section has no bearing on how or to what extent EPA utilizes research in regulatory decision-making processes.
Response: EPA is not relying on any of the substantive environmental statutes cited as authority in the 2018 and 2020 proposal. See response to Comment 73-410 in RTC document Section 1.2.2.
Comment [9066-1895]: Commenter (6188) contends that the rulemaking authority provided by provisions authorizing the EPA Administrator to promulgate regulations "as necessary to carry out his functions" under various environmental statutes does not extend to actions that would undermine, rather than further, the relevant acts' directives. See 42 U.S.C. § 7601(a) (CAA); 33 U.S.C. § 1361 (CWA); 42 U.S.C. § 300j-9(a)(1) (SDWA); 42 U.S.C. § 6912(a)(1) (RCRA); 42 U.S.C. § 9615 (CERCLA); 42 U.S.C. 11048 (EPCRA); 7 U.S.C. § 136w (FIFRA). As discussed in greater detail below, both the intent and the language of the Proposed Rule are in direct opposition to the statutory requirements of these environmental statutes, which seek to protect public health and the environment.
The Proposed Rule, apparently mistakenly, also cites as authority 42 U.S.C. § 6979, a labor standards provision within the RCRA requiring certain prevailing wage standards to be met on construction projects receiving EPA grants, and 42 U.S.C. § 9616, a section of the CERCLA requiring listing and evaluation of facilities and commencement of remedial activities for listed facilities. Neither one of these sections is relevant to the Proposed Rule or to research activities more generally. Inclusion of these mistaken cites in the Proposed Rule's laundry list of "authorizing" provisions highlights the absence from that list of any statutory language that provides clear authority for a rule of this unprecedented breadth and import and the hasty nature of this rulemaking process.
In sum, EPA offers no legal authority upon which to base a rulemaking of this significance.
Response: EPA is not relying on any of the substantive environmental statutes cited as authority in the 2018 and 2020 proposal. See response to Comment 73-410 in RTC document Section 1.2.2.
Comment [9078-3508]: Commenter (6194) provides that "[I]t is `axiomatic' that `administrative agencies may act only pursuant to authority delegated to them by Congress." Clean Air Council v. EPA, 862 F.3d 1, 9 (D.C. Cir. 2017). Multiple statutes administered by EPA direct the agency to consider and utilize science. While the type of science to be considered, e.g., best science or best available science, and the process for considering science, e.g., as one of multiple, discrete factors or as part of broader balancing analyses, varies across statutes, nowhere do these statutes suggest that "best" science is limited to that for which associated raw data is available to the public. Courts have concurred. See, e.g., American Trucking Ass'n v. EPA, 283 F.3d 355, 372 (D.C. Cir. 2002) (finding that the CAA imposes no requirement for EPA to "obtain and publicize the data underlying published studies on which the Agency relies" and that "requiring agencies to obtain and publicize the data underlying all studies on which they rely `would be impractical and unnecessary.'") (internal citations omitted). The EPA may not read into a statute a requirement that makes its implementation impractical.
Nor do the statutes administered by EPA have a concept akin to the Proposal's "pivotal regulatory science." EPA is now attempting to re-interpret multiple statutes with virtually no analysis. Such reinterpretation would read new requirements into well-established statutes decades after their passage in a manner inconsistent with EPA's historic positions. If Congress had intended to create additional subcategories of regulations or science as suggested by the Proposal, it would have done so.
The Proposal's creation of a new category of regulation, "regulatory decisions," and new categories of science --  "pivotal regulatory science" and "regulatory science" -- rely on factors that Congress did not authorize or intend EPA to consider. When Congress wanted to place additional emphasis on certain types of or access to science it did so; not establishing subcategories of regulations or science tied to the availability of underlying data was a deliberate decision by Congress, not an oversight that EPA can write-in to the statutes. Hence, the Proposal would change EPA's interpretation of multiple statutes that govern the agency's consideration and use of science by creating new subcategories of science and regulations. None of the statutes referenced by EPA support the Proposal's suggestion to redefine regulations or science, much less require EPA to pursue the course it has proposed.
Response: See response to Comment 73-410 in RTC document Section 1.2.2.
Comment [9078-3548]: Commenter (6194) contends that the Proposal would prevent EPA from relying on the best available information in discharging its duties. EPA is charged with developing regulations that provide societal benefits by reducing harm to human health and the environment from the presence of chemicals in air, soil, drinking water, food, and consumer products. The Proposal implicates research that is central to making such determinations; creating an obstacle to the consideration of such science is contrary to both primary statutory directives and requirements to conduct cost benefit analyses. Likewise, according to the commenter, the EPA fails to address how it could implement a regulation that is inconsistent with federal requirements and standards governing the use of science across agencies.
The commenter notes that the statutory requirements to develop health-based standards and conduct cost-benefit analyses EPA acknowledges that it must use the "best available science" in all of its regulatory actions. 
This directive is embodied in multiple statutes implemented by EPA and is reflected in other environmental statutes, further illustrating Congress' conceptualization of "best" available science. Broadly speaking, there are three basic types of statutory requirements that the EPA and other agencies consider scientific information. First, there are requirements that the agency consider the "best available science." Second, there are requirements that the agency consider particular factors, including scientific factors. Third, there are requirements that the agency balance economic costs or technological feasibility against public health, environmental, or other benefits that must be scientifically assessed. The precise terminology regarding the required use of science may, as illustrated below, vary across statutes, but what these provisions have in common are requirements for EPA and other agencies to use scientific information that is "best," not simply adequate, and "available." The commenter provides that following examples of statutory directives regarding the use of science by the EPA and other Agencies.
 SDWA [42 U.S.C § 300g1(b)(3)(A)]: EPA must use "[t]he best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices," and "[d]ata collected by accepted methods or best available methods (if the reliability of the method and the nature of the excision justifies use of the data)."
 [42 U.S.C. § 300g1(b)(1)(b)(ii)]: EPA must use "best available public health information" in determining whether to regulate contaminants.
 CWA [33 U.S.C. § 1321(a)(27)]: Some provisions dealing with oil and hazardous substances on the Gulf Coast require "best available science."
 TSCA [15 U.S.C. § 2625(h)]: "In carrying out sections 2603, 2604, and 2605 of this title, to the extent that the Administrator makes a decision based on science, the Administrator shall use scientific information, technical procedures, measures, methods, protocols, methodologies, or models, employed in a manner consistent with the best available science". 
 Magnuson -  Stevens Fishery Conservation and Management Act (MSA) [16 U.S.C. § 1851(a)(2)]: "Conservation and management measures shall be based upon the best scientific information available." 
 Endangered Species Act (ESA) [16 U.S.C. § 1536(a)(2)]: In consultation to ensure no jeopardy to endangered or threatened species, "each agency shall use the best scientific and commercial data available." 
 [16 U.S.C. § 1536(c)(1)]: Advice that an endangered or threatened species may be present in a location should be "based on the best scientific and commercial data available." 
 [16 U.S.C. § 1536(h)(2)(b)(i)16 U.S.C. § 1533(b)(1)(A)]: Permanent exceptions shall be given to prevent extinction of species "based on the best scientific and commercial data available." The Secretary must determine whether a species is a threatened or endangered based upon "the best scientific and commercial data available."
 Marine Mammal Protection Act (MMPA) [16 U.S.C. § 1371(a)(4)(C)]: "If the Secretary determines, using the best scientific information available, that certain forms of deterrence have a significant adverse effect on marine mammals, the Secretary may prohibit such deterrent methods, after notice and opportunity for public comment, through regulation under this chapter." 
 [16 U.S.C. § 1371(a)(3)(A)]: The decision to waive the ban on taking must be based on the "best scientific evidence available and in consultation with the Marine Mammal Commission."
None of these statutory provisions link the concept of the "best available science" to the availability of underlying raw data, nor can such a limitation be read into the statutes. Where Congress wanted to direct EPA's decision regarding what science to consider or how -- it did so. When Congress wanted to link "best available science" to data being publicly available -- it did so. Thus, if Congress had wanted to put a blanket limitation on EPA's consideration of science for which underlying data is not publicly available, it clearly knew how to do so -- and it chose not to.
As EPA has previously explained, the agency would be unable to fulfill statutory mandates if it could not consider studies that follow federal laws and ethical standards regarding the privacy of patient and individual data: 
      (If EPA and other governmental agencies could not rely on published studies without conducting an independent analysis of the enormous volume of raw data underlying them, then much plainly relevant scientific information would become unavailable to EPA for use in setting standards to protect public health and the environment. . . . [S]uch data are often the property of scientific investigators and are often not readily available because of the proprietary interests of the investigators or because of arrangements made to maintain confidentiality regarding personal health status and lifestyle information of individuals included in such data. Without provisions of confidentiality, the possibility of conducting such studies could be severely compromised.)
Examples of EPA statutory mandates that could be impeded by the Proposal include (i) the development of standards to control emissions from burning hazardous waste fuels under the RCRA "as may be necessary to protect human health and the environment," 42 U.S.C. § 6924(q)(1); and (ii) determinations of whether hazardous sites need to be placed on the CERCLA National Priorities List (NPL). With respect to the latter, CERCLA provides that, if a health assessment indicates that a release or threatened release may pose a serious threat to human health, the Administrator of the Agency for Toxic Substances and Disease Registry (ATSDR) shall "notify the Administrator of EPA who shall promptly evaluate such release or threatened release in accordance with the hazard ranking system . . . to determine whether the site shall be placed on the National Priorities List." 42 U.S.C. § 9604 (i)(6)(H). The commenter note that health assessments provided by ATSDR to EPA on a scheduled basis are exempt from peer review as a prerequisite to submission. 
The Proposal would also cut EPA off from information needed to complete the cost-benefit analysis of regulations required under certain statutes and government-wide requirements.
Not only would the Proposal impede EPA's fulfillment of its statutory duties, it would also be contrary to judicial precedent, which holds that "best available science" includes "all existing scientific evidence relevant to the [agency] decision [in question]." Ecology Ctr., Inc. v. U.S. Forest Serv., 451 F.3d 1183, 1194 n.4 (10th Cir. 2006) (quoting Heartwood, Inc. v. U.S. Forest Serv., 380 F.3d 428, 436 (8th Cir. 2004)). While agencies need not produce new evidence, they must seek out information and "`cannot ignore existing data.'" Id. (emphasis added). Even if an agency is anticipating the arrival of better evidence, it must act on the basis of the evidence it currently has. Many of the fundamental public health studies on which EPA has based key rules and standards under the statutes are studies for which the raw data was not or could not have been released. Nonetheless these studies have been examined and validated, as will future studies. Arbitrarily cutting off consideration of a category of science is inconsistent with EPA's statutory obligations.
Response: See response to Comments 32-144, 34-155, 48-381 and 73-410 in RTC document Section 1.2.2.
Comment [9217-3544]: Commenter (9225) asserts that equally problematic, the proposed rule is not authorized by any authority delegated to EPA by Congress and is contrary to a number of statutes under EPA's authority. This includes, but is not limited to, the CAA; CWA; TSCA; Frank R. Lautenberg Chemical Safety for the 21st Century Act (LCSA); SDWA; FIFRA; and more. Substantively, the rule violates numerous public health and environmental provisions contained in these laws, as well as requirements to use the best available science or to consider all available information, while procedurally, it violates the APA and a number of other laws that set forth specific procedures EPA must follow during its rulemaking process. It also lacks an environmental justice analysis even though the rule will have the greatest impact on low-income and minority communities that benefit from protections based on the very studies the rule restricts from consideration when setting exposure limitations for pollution and toxic chemicals. Simply put, the proposal cannot withstand legal scrutiny.
Response: See responses to Comments 32-144, 48-381 and 73-410 in RTC document Section 1.2.2. Also see response 48-502 in the APA Section 1.6.1.2 of the RTC document. 
Comment [6092-476]: Commenter (6102) notes that as EPA takes strides to increase transparency, it is imperative that the agency assure full compliance with all statutory requirements for the protection of privacy, the confidentiality of research participants, the protection of proprietary data and the confidentiality of business information. The EPA should continue to explore and develop protocols to code or de-identify data sets to make them available while fully protecting privacy. Given these challenges, the NAM recommends the EPA explore ways to tailor requirements on a statute-by-statute basis as appropriate.
Response: EPA intends to promulgate future transparency rules under the substantive environmental statutes that it administers. Those rules will tailor transparency requirements as appropriate for each statute. See response to Comment 73-410 in RTC document Section 1.2.2.
Comment [6894-3619]: Commenter (6915) notes that EPA asks, "whether alternative or additional regulatory or other policy vehicles are appropriate to establish and implement these policies, and whether further regulatory or other policy vehicles on the programmatic or statutory level would be appropriate as alternative or additional steps." (83 FR 18771). It is EPA's job to identify and describe these alternatives, and to explain why it has put forward its particular proposal: it may not, at this late stage, ask amorphous questions on policy design.
Response: EPA disagrees that it was inappropriate or impermissible for the agency to seek comment on alternative methods to implement this policy besides rulemaking.
Comment [6425-2389]: Commenter (6447) asserts that EPA has not identified any legitimate justification for this proposal, has failed to support the proposal with any kind of administrative record, and has failed to obtain support for the proposal from its own Science Advisory Board (SAB), rendering it arbitrary and capricious. Moreover, the proposal violates existing law. For these reasons, and because it is clear that the proposal would undermine EPA's mission and public health, we are strongly opposed. EPA should not create any new restrictions on the use of scientific information.
Response: EPA provided a rationale for the rule in the 2018 proposal and expanded and clarified its justification for the rule in the 2020 supplemental proposal. EPA also sought and received input on the rule from the SAB. The rule does not limit the studies the agency may consider in actions to satisfy its mission to protect human health and the environment.
1.1.2 General Legal Authority (Supplemental Notice of Proposed Rulemaking (SPNRM) Comments)
Comment [29-121]: Commenter (11184) writes that EPA lacks authority for the proposed rule and SNPRM.
The SNPRM changes the authority EPA claims to rely on for the proposal, however either way, the proposal remains unlawful. The text of the SNPRM proposes to rely solely on 5 U.S.C.§ 301. 85 Fed. Reg. at 15404. The preamble says EPA is taking comment on whether to use 5 U.S.C. § 301, which the SNPRM calls EPA's housekeeping authority, independently or in conjunction with statutory provisions cited in the proposed rule and corrected in the SNPRM's preamble. 85 Fed. Reg. at 15398. The fact that EPA is still considering and asking what authority it can rely on for the proposed rule, twenty-two months after the initial proposed rule, shows that EPA does not have such authority. The SNPRM's statements on authority continues EPA's pattern of making rules first and searching for a justification later (which it still has not provided for this rulemaking). As the NTAA explained in its 2018 comments: In order to protect human health and environment, EPA must have a solid foundation before it proposes rules and rulemaking as such portrays the agency's naivete in seeking answers for questions about its own authority.
First, none of the initial or corrected environmental statutes provide EPA authority for the proposed rulemaking. In most cases, EPA simply cites its general authority for rulemaking under the statutes. Still that general authority alone cannot provide a basis for the rule, especially when, as here, the rule would conflict with the requirements of each of the statutes. Courts have made clear that "EPA cannot rely on its gap-filling authority to supplement [a statute's] provisions when Congress has not left the agency a gap to fill." NRDC v. EPA, 749 F.3d 1055, 1064 (D.C. Cir. 2014); see also New York v. US EPA, 413 F.3d 3, 40-42 (D.C. Cir. 2005). EPA otherwise cites to statutes describing research programs established by the agency, with no relation to or authorization for the requirements the SNPRM seeks to create. The KBIC also disagrees with EPA's reliance on 5 U.S.C. § 301 because 1) it is questionable whether§ 301 applies to EPA as EPA is not one of the fifteen "Executive Departments" listed at 5 U.S.C. § 101, and 2) the proposed rule and SNPRM are not merely internal rules of agency procedure-they would significantly impact every aspect of EPA's scientific and regulatory work, the individuals who contribute to that work, and the Tribes and communities who rely on it.
The KBIC appreciates the SNPRM's recognition that "in the event the procedures outlined in the proposed rulemaking conflict with the statutes that EPA administers, or their implementing regulations, the statutes and regulations will control." 85 Fed. Reg. at 15398. This statement alone should lead EPA to withdraw the proposed rule and SNPRM; the requirements conflict with every statute EPA administers and would cause EPA to unlawfully exclude valid scientific research from its work.
Response: As discussed in the final rule preamble at Section I. C, EPA is not claiming authority under the Federal Housekeeping Statute, 5 U.S.C. § 301. Rather, EPA maintains that the Reorganization Plan No. 3 of 1970, which established EPA, transferred authority to EPA that is equivalent to Federal housekeeping authority. 
This is a procedural rule because it exclusively governs EPA's internal affairs and does not regulate any person or entity outside of EPA. See Chrysler Corp. v. Brown, 441 U.S. 281, 302 (1979). EPA is not interpreting, applying, or citing as authority any of the substantive environmental statutes that it administers. EPA plans to issue subsequent rulemakings implementing this procedural rule, some of which may be substantive and therefore subject to judicial review. If implementing this procedural rule will result in a conflict with an environmental statute that EPA administers, then this procedural rule will yield under § 30.3.
Comment [48-441]: Commenter (12727) writes that as they explained in their comments on the 2018 proposal, none of the environmental statutes EPA identifies in the original proposal authorize EPA to issue the proposed rule or otherwise promulgate a one-size-fits-all regulation governing how the agency will consider science under its various statutory authorities. EPA gives no explanation of how any of the provisions it cites provide authority for the 2018 proposal and Supplemental Notice, much less how all of them authorize identical requirements despite their varied obligations for considering science. Not only is there still no authority for EPA's pan-statutory proposal, but the 2018 proposal as supplemented would force EPA to more broadly violate a diversity of obligations to consider science under the different environmental laws. While the specifics of the requirements vary, Congress has often commanded EPA to utilize the "best available science" for its policies, which necessitates consideration of the full range of available science. The proposal would restrict consideration of science and cause EPA to violate both its general statutory obligations to consider all available data when undertaking rulemakings as well as the specific requirements to consider science under the respective environmental statutes.
The Supplemental Notice multiplies this defect, broadly expanding the regulation to cover "data and models underlying pivotal regulatory science and pivotal science used to support significant regulatory decisions and influential scientific information, respectively, not simply dose-response data and dose-response models" -- effectively hamstringing the agency from considering the best available science across its full portfolio of actions to protect the environment and public health, and impeding its ability to gain a better understanding of the threats to public health and the environment. Nothing in the Supplemental Notice remedies this expansive defect or even attempts to reduce it.
Response: See response to comment 29-121.
Comment [48-479]: Commenter (12727) notes that several environmental statutes require EPA to ground its decision-making in scientific evidence. The CAA, for example, mandates that "[a]ir quality criteria . . . accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare," 42 U.S.C. § 7408(a)(2), and the Toxic Substances Control Act requires the Administrator to "make decisions . . . based on the weight of the scientific evidence," 15 U.S.C. § 2625(i)."
The originally proposed regulatory language (§§ 30.3, 30.5) violated these statutory commands by preventing EPA from relying on a study as "pivotal regulatory science . . . used to justify significant regulatory decisions" if dose-response data or models underlying the study were not publicly available, without regard to whether the study had been validated by other means. The Supplemental Notice exacerbates these violations by expanding the proposed regulation's scope (§§ 30.3, 30.5) to not just pivotal regulatory science used to justify significant regulatory decisions, but also "pivotal science supporting influential scientific information."
Response: See response to comment 29-121.
Comment [48-490]: Commenter (12727) writes that just as EPA is required to consider all available science when making its regulatory decisions, governing environmental and public health statutes likewise require EPA to consider all available science when releasing ISI, which often forms the basis for the agency's regulatory decisions. For example:
CAA section 108 instructs EPA to establish air quality criteria that "accurately reflect the latest scientific knowledge," which criteria, in turn, must inform NAAQS.
The TSCA directs that, in implementing the Act's testing requirements, manufacturing and processing notice requirements, and requirements for the prioritization, risk evaluation, and regulation of chemical substances and mixtures, EPA must make its decisions "based on the weight of the scientific evidence." Likewise, in carrying out EPA's responsibilities under TSCA, "the Administrator shall use scientific information . . . employed in a manner consistent with the best available science." Numerous other TSCA provisions emphasize EPA's obligation to consider all reasonably available scientific information when implementing TSCA's requirements.
The CWA instructs that EPA's water quality criteria must "accurately reflect[] the latest scientific knowledge" on a variety of factors.
The SDWA generally requires EPA to use "the best available, peer-reviewed science," and to use the "best available public health information" when deciding whether to regulate a particular contaminant.
A rule that prohibits EPA from considering (or that downgrades) valid, high quality scientific studies when generating ISI such as the products identified above would contravene the above- noted statutory directives regarding EPA's obligation to consider all available science.
Response: See response to comment 29-121.
Comment [48-495]: Commenter (12727) writes that there are also many circumstances under which EPA develops [ISI] with broad applicability to multiple programs or varying governmental actions that have yet to be identified. Such products include Integrated Risk Information System (IRIS) chemical reviews by EPA's Office of Research and Development (ORD) (e.g., the 2019 "IRIS Toxicological Review of Ethyl Tertiary Butyl Ether (ETBE)" and "IRIS Toxicological Review of Tert-Butyl Alcohol (TBA)"), and reports by EPA's Office of Chemical Safety and Pollution Prevention (e.g., "Continuing Development of Alternative High- Throughput Screens to Determine Endocrine Disruption, Focusing on Androgen Receptor, Steroidogenesis, and Thyroid Pathways"). EPA's application of the proposed public disclosure regulations to such products is especially pernicious because it would likely be impossible -- or at least, immensely burdensome -- to determine at the time that the ISI is utilized in agency decision- making whether, in developing the ISI, EPA omitted from consideration (or gave lesser weight to) valid scientific studies that EPA is obligated to consider fully under the relevant statutory or regulatory provisions.
For example, the purpose of ORD's IRIS chemical reviews is to support EPA's mission of protecting public health and the environment by identifying and characterizing the health hazards of chemicals found in the environment. [...] Among other things, EPA uses this information to help set national standards under TSCA, the CAA, the SDWA, and the EPCRA, and to clean up hazardous sites under CERCLA. But none of these statutes allow EPA to ignore valid, pivotal scientific studies based solely on the fact that underlying data or models are not publicly available. If ORD is forced to exclude consideration of such studies when developing its IRIS chemical reviews, EPA's reliance on these reviews when taking actions under these statutes would be unlawful and arbitrary unless EPA combs through the report and available science to confirm that all valid studies have been considered. The need to undertake such a comprehensive review of ISI before relying upon it would fundamentally undermine the purpose of preparing ISI for general use by EPA's various programs, offices, and regions. Thus, beyond the numerous statutory violations described above. 
Response: See response to comment 29-121.
Comment [48-503]: Commenter (12727) writes that EPA must withdraw the Supplemental Notice as inconsistent with its substantive obligations to use the best available science. Apparently anticipating these conflicts, EPA concedes that:
"This internal agency procedure is intended to be consistent with the statutes that EPA administers and EPA plans to implement this procedural rulemaking in accordance with all applicable statutory and regulatory requirements. . . . Nonetheless, in the event the procedures outlined in the proposed rulemaking conflict with the statutes that EPA administers, or their implementing regulations, the statutes and regulations will control."
This rationale is vague and meaningless. As an initial matter, EPA has not explained how its rule is "intended to be consistent with" the statutes it administers, nor has it attempted to identify the areas of conflict it anticipates, which renders its promise to yield to their requirements arbitrary. Even if it had offered some indication of the conflicts it foresees, its protean response is ineffectual: to the extent that the agency seeks to inoculate its rule against legal challenges based on these statutory violations, any invalid application -- of the many that would proliferate upon implementation -- would doom the rule. EPA cannot insulate its unlawful policy from challenges by promising to yield in the future to statutory and regulatory requirements.
Nor does EPA enhance its rule with the mere suggestion that its options for considering studies in some circumstances "improve consistency" with statutes requiring use of the best available science. Aside from the inherent flaws in these approaches, discussed in greater detail occasional compliance with statutory requirements cannot render its rule lawful. To correct the numerous statutory violations documented above, EPA must conform its actions to the directives Congress imposed; partial alignment will not suffice.
Response: See response to comment 29-121. The purpose of the savings clause is to clarify the scope of the rule and how it will be implemented by EPA. Specific standards set by the various environmental statutes will be discussed as appropriate when EPA develops subsequent rulemakings implementing this final procedural rule. As explained above, EPA is not relying on the substantive environmental statutes as authority for this rule. See the final rule preamble discussion at Section I. C.
Comment [48-600]: Commenter (12727) writes that EPA's failure to characterize costs and benefits necessarily precludes the agency from assessing whether the costs are disproportionate to the benefits, a finding that could render the proposal arbitrary and capricious. As described below, this failure extends to harms that the proposal would inflict upon the public at large, vulnerable populations, the research community, and the agency. EPA does not identify any statutory authority that could justify its failure to assess the costs and benefits of the proposed rule, and it is certain that the agency could not rely on a radical use of a generally applicable statute like the Housekeeping Act in order to evade well-established requirements of administrative law.
Response: Because this final rule is internal and purely procedural, it will not impose any direct costs or benefits outside of EPA.
Comment [54-249]: Commenter (11361) writes that in addition to asserting that the EPA can introduce these changes under its housekeeping authority, the EPA also lays the groundwork in this SNPRM to subvert public comment in the future. The EPA states that it is "considering how to proceed, apart from this supplemental proposal, to establish regulations interpreting provisions of, and/or exercising substantive rulemaking authority delegated to it by programmatic statutes." Creating regulations specifically about how the EPA interprets or carries out its statutes could result in a process whereby many, perhaps all, new environmental regulations pursuant to the EPA's statutes, including the CAA and CWA, would be considered internal procedures and thus give the EPA leeway to circumvent the APA's required notice-and-comment rulemaking. The EPA has been acting with an aggressive deregulatory agenda for the last three years. This action not only shuts the public out of federal environmental decisions but could lay the groundwork for discarding all of the EPA's regulatory accountability. They firmly oppose the establishment of regulations that could allow the EPA to circumvent notice-and-comment rulemaking.
Response: Any substantive regulations issued under the environmental statutes that EPA administers will be subject to notice and comment rulemaking as required in the APA.
Comment [73-412]: Commenter (11576) writes that the environmental statutes cited in the Supplemental Notice offer no support for the proposed rule.
In its original notice, of course, the EPA made no attempt to rely on any kind of "housekeeping" power. Instead, the agency said it was "propos[ing] to take this action under authority of the statutes it administers, including provisions providing general authority to promulgate regulations necessary to carry out the Agency's functions under these statutes and provisions specifically addressing the Agency's conducting of and reliance on scientific activity to inform those functions[.]" The EPA went so far as to list the specific statutory sections that it believed its rule could rest on, citing:
CAA sections 103, 301(a), 42 U.S.C. 7403, 7601(a); CWA sections 104, 501, 33 U.S.C. 1254, 1361; SDWA sections 1442, 1450(a)(1), 42 U.S.C. 300j - 1, 300j - 9(a)(1); RCRA sections 2002(a)(1), 7009, 42 U.S.C. 6912(a)(1), 6979; CERCLA (as delegated to the Administrator via Executive Order 12580) sections 115, 311, 42 U.S.C. 9616, 9660; EPCRA section 328, 42 U.S.C. 11048; FIFRA sections 25(a)(1), 136r(a), 7 U.S.C. 136r(a), 136w; and TSCA, as amended, section 10, 15 U.S.C. 2609.
None of these provisions offered any support for the proposed rule -- and many were at odds with it. As noted during the prior comment period, "restricting sound science is neither necessary nor consistent with the ... goals" of the referenced environmental statutes. A few of the citations, moreover, appeared to be entirely irrelevant.
With its supplemental notice, the EPA appears to abandon its misguided effort to rely on the laws it had previously cited, stating that it no longer "propose[s] to interpret provisions of a particular statute or statutes that it administers[.]" At the same time, however, the notice declares that the agency is still "taking comment on whether to use its housekeeping authority independently as authority or in conjunction with the environmental statutory provisions cited as authority in the 2018 proposed rulemaking[.]" And it also corrects two of the citations included in the agency's earlier list, "clarifying that the citation to the Resource Conservation and Recovery Act ... section 7009, 42 U.S.C. 6979, should be to ... section 8001, 42 U.S.C. 6981[,]" and "the citation to the CERCLA ... section 116, 42 U.S.C. 9616, should be to ... section 115, 42 U.S.C. 9615[.]"
Given the EPA's inconsistent statements regarding the relevance of the statutory provisions it has cited, it is not at all clear what the public is being asked to comment on. What is clear, however, is that the agency's two new citations do nothing to support the proposed rule. As none of the environmental statutes the agency has cited provide a legal basis for the proposed action, the EPA would get nowhere in trying to use them "in conjunction" with the agency's asserted and inapposite housekeeping authority.
Response: See response to comment 29-121.
Comment [79-527]: Commenter (12427) writes that there is no statutory requirement or deadline for the proposed rule for "Strengthening Transparency in Regulatory Science" or its supplemental notice. In fact, there is no statutory mandate for this proposed rule, at all. Thus, there is no requirement or need for this rule or its supplemental notice. The proposed rule and supplemental notice should be withdrawn.
Response: EPA identified a need to standardize its internal process for considering the availability of data and models underlying pivotal science used in its significant regulatory decisions and ISI so that EPA can promulgate future regulations in a more efficient and transparent manner. EPA decided to meet that need by issuing a housekeeping regulation. As explained in the final rule preamble at Section I. C, EPA's Reorganization Plan transferred the authority to issue housekeeping regulations to EPA, and EPA is issuing this procedural rule in accordance with that authority.
Comment [81-584]: Commenter (12715) writes that the proposed rule conflicts with fundamental statutory requirements to use the best available science and citation to "housekeeping" authority cannot cure that defect.
Whether EPA is arbitrarily barring consideration of scientific studies or information with nonpublic underlying data or data that are not able to be independently validated, or arbitrarily ascribing less weight to such studies or information, neither approach constitutes "housekeeping," and neither comports with statutory requirements that EPA use the best available science. To reiterate from the 2018 Comments, EPA's obligation with respect to the use of scientific information is clear: it must base its decisions on such criteria as the latest scientific knowledge, the best available, peer-reviewed science, and/or generally accepted scientific principles or laboratory tests. To cite just a few examples, in performing its duties, EPA must rely on: "the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices," SDWA of 1996, 42 U.S.C. § 300g-1(b)(3)(A)(i); the "best available science," TSCA, 15 U.S.C. § 2625(h); "the latest scientific knowledge," CWA of 1977, 33 U.S.C. § 1314(a)(1), and CAA of 1970, 42 U.S.C. § 7408(a)(2); and "generally accepted scientific principles or laboratory tests, or appropriately designed and conducted epidemiological or other population studies," EPCRA, 42 U.S.C. § 11023(d)(2). EPA has repeatedly emphasized the importance of this fundamental precept, including in its 2018-2022 strategic plan, which states that one of the agency's priorities is to "identify, assess, conduct, and apply the best available science to address current and future environmental hazards."
These statutory requirements apply to all aspects of EPA's decision-making, including its scientific evaluations, adoption of regulatory standards, and development of policies. They do not permit what EPA proposes here: to either exclude or give less weight to scientific studies or information based on criteria -- availability of the underlying data and ability to independently validate underlying data -- that are not determinative of whether the studies or information constitute the best available science. Agencies may not adopt or implement regulations that conflict with the statutes under which they are promulgated, and an agency's interpretation of those statutes must always at least be reasonable. See Chevron, U.S.A., Inc. v. Nat. Res. Def. Council, Inc., 467 U.S. 837, 842-44 (1984). Furthermore, neither 5 U.S.C. § 301 nor any other purported housekeeping authority allows EPA to negate these statutes' fundamental, substantive, and common-sense commands. And even assuming EPA has some inherent housekeeping authority, it cannot rely on a general grant of rulemaking authority to promulgate regulations or develop policies that are otherwise inconsistent with more specific statutory directives. Glob. Van Lines, Inc. v. Interstate Commerce Comm'n, 714 F.2d 1290, 1293-97 (5th Cir. 1983).
Finally, EPA acknowledges that the statutes it administers would control in the event of a conflict between the proposed rule and those statutes, see 85 Fed. Reg. at 15,398, but that caveat cannot save the proposed rule. First, the core principle of the proposed rule is facially invalid, i.e., there is no circumstance in which it would be acceptable for EPA to exclude or give less weight to relevant, probative scientific studies, models, or other information that have been validated through peer review, on the sole basis that the underlying data are not publicly available or are not able to be independently validated. EPA cannot save an invalid rule by including what amounts to a waiver procedure, because the "essence of waiver is the assumed validity of the general rule." Alltel Corp. v. Fed. Commc'n Comm'n, 838 F.2d 551, 561 (D.C. Cir. 1988). Second, the agency's essential duty in rulemaking is to exercise its subject matter expertise to enact rules that implement and further the purposes of the statutes Congress has assigned it to administer. Chevron, 467 U.S. at 842-44. This does not include proposing rules without first determining the source of legal authority but instead asking commenters to supply that information. Third, the agency must propose its rules with specificity and be clear as to the programs, regulations, and information to which they will apply. 5 U.S.C. § 553(b)(3); Home Box Office, Inc., supra, 567 F.2d at 35-36.
Response: See response to comment 29-121. EPA identified applicable authority in the proposal (83 FR 18768), the supplemental (85 FR 15396), and the final rule Federal Register notice. [See Docket No. EPA-HQ-OA-2018-0259].
EPA also requested public comment on whether the authority EPA identified was appropriate. Requesting public insight on a proposed path forward is part and parcel of the notice and comment process.
Comment [82-695]: Commenter (12464) writes that both alternatives are inconsistent with the statutes directing which data EPA must consider in certain contexts. As they explained in a previous comment letter, many statutes require EPA to consider science in its decision-making, without any reference to whether such science is based on studies with raw data that is publicly available. For example, under the CAA, EPA must set and review the NAAQS for six common pollutants. In setting these standards, the CAA instructs EPA to use "the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects [of air pollution] on public health or welfare." Thus, a conflict would arise between the statutory language and the reduced-weight alternative if EPA gave less weight to a study that was "the latest scientific knowledge" on a relevant question merely because the raw data from that study is not publicly available. The same conflict would exist with the proposed tiered-access system, if the "latest scientific knowledge" came from a study with raw data to which researchers have not or could not grant restricted access.
Response: See response to comment 29-121.
Comment [82-834]: Commenter (12464) writes that the Proposal is similarly substantive as it "relocates the metes and bounds" of "best available science" requirements under the multiple environmental statutes that EPA implements. Specifically, the Proposal not only changes "how" EPA performs environmental decision-making under these statutes, but it also changes "what" studies EPA must consider as "best available science" when issuing ISI and making significant regulatory decisions. Indeed, the Supplemental Notice acknowledges that the purpose of the proposed rule is "to establish an agency-wide approach to ensure that the data and models underlying EPA's significant regulatory decisions are publicly available." 85 Fed. Reg. at 15,398.
Response: See response to comment 29-121.
Comment [82-865]: Commenter (12464) writes that the Proposal is inconsistent with EPA's duties under multiple statutes and executive orders. They pointed out many of these conflicts in comments on the Initial Proposal, but the Supplemental Notice continues to make the same errors. Moreover, some changes introduced in the Supplemental Notice exacerbate the problems.
Response: See response to comment 29-121. EPA issued this rulemaking consistent with its obligations under all applicable executive orders. 
Comment [84-1403]: Commenter (11390) writes that the proposed rule falls outside EPA's rulemaking authority.
Despite EPA's assertion that the amended Proposed Rule governs "internal agency procedures" and "does not regulate any entity outside the Federal government," the Proposed Rule is, in fact, a substantive regulation. It will profoundly impact how EPA interacts with regulated parties by changing how the agency performs its core functions.
"It is axiomatic that an administrative agency's power to promulgate legislative regulations is limited to the authority delegated by Congress." Bowen v. Georgetown Univ. Hosp., 488 U.S. 204, 208 (1988). EPA steps far beyond that authority in the SNPRM, proposing a regulation that would change the way the agency engages in virtually any decision-making or scientific process -- outside of agency adjudications and enforcement activities -- whether or not that process ultimately leads to the adoption of regulations. EPA has revised the Proposed Rule to cite the agency's housekeeping authority as its basis to regulate. But EPA has no statutory authority to issue the Proposed Rule, either through the Housekeeping Statute or via other environmental statutes EPA has previously cited.
EPA also continues to cite to numerous provisions in environmental statutes to support this rulemaking, but none provides authority for the agency to promulgate the Proposed Rule. Generally, these provisions fall into one of two categories: (1) provisions authorizing EPA to conduct research in furtherance of statutory objectives and (2) general provisions authorizing the EPA Administrator to promulgate regulations as necessary to achieve the purposes of a given statute. None of these statutory references provides the requisite authority for adoption and implementation of the Proposed Rule.
EPA points to provisions authorizing the establishment of research and development programs pursuant to each of the federal environmental laws EPA administers. But these statutory references are unavailing. Each statute directs EPA to set up research programs and to undertake specific activities attendant to the administration of those programs, but none governs -- or even references -- the extent to which research should be used in regulatory decision-making by EPA. Further, any regulatory authority EPA may have under the referenced provisions is limited to the individual research and development programs in question and does not extend to unrelated research by outside parties. In other words, the cited provisions allow EPA to set up its own research programs, but do not create authority to place limitations on how research, whether conducted or financed by EPA or produced by an outside party, is used to set regulatory standards.
For example, EPA purports to derive authority for the Proposed Rule from CAA § 103. That provision simply authorizes EPA to establish a national research and development program for the prevention and control of air pollution and, as part of that program, to conduct and promote the coordination of research and studies relating to the causes, effects, extent, prevention, and control of air pollution. 42 U.S.C. § 7403(a)(1). The section authorizes specific activities of the Administrator in establishing such a program, none of which includes limiting the scope of reviewable data, research, or studies when undertaking regulatory action. The section has no bearing on how or to what extent EPA utilizes research in regulatory decision-making processes.
The rulemaking authority provided by provisions authorizing the EPA Administrator to promulgate regulations "as necessary to carry out his functions" under various environmental statutes does not extend to actions that would undermine, rather than further, the relevant acts' directives. See 42 U.S.C. § 7601(a) (CAA); 33 U.S.C. § 1361 (CWA); 42 U.S.C. § 300j-9(a)(1) (SDWA); 42 U.S.C. § 6912(a)(1) (RCRA); 42 U.S.C. § 9615 (CERCLA); 42 U.S.C. 11048 (EPCRA); 7 U.S.C. § 136w (FIFRA). As discussed in greater detail below, both the intent and the language of the Proposed Rule are in direct opposition to the statutory requirements of these environmental statutes, which seek to protect public health and the environment.
In sum, EPA offers no legal authority upon which it may lawfully base a rulemaking of this significance. 
Response: See response to comment 29-121. As discussed in the final rule preamble at Section I. C, this is a procedural rule governing EPA's internal affairs. It does not regulate or affect the legal rights and obligations of parties outside of the agency, and therefore is not a substantive rule. An internal rule of agency procedure is not substantive merely because it may indirectly affect the voluntary behavior of parties outside of the agency. See American Hosp. Ass'n v. Bowen, 834 F.2d 1037, 1051 (D.C. Cir. 1987) ("[A]gency rules that impose `derivative,' `incidental,' or `mechanical' burdens upon regulated individuals are considered procedural, rather than substantive"). 
This procedural rule does not impose requirements on external scientists. Instead, this rule exclusively regulates EPA employees. It is not intended to impose burdens on external scientists who want EPA to consider their research when making significant regulatory decisions or developing ISI, particularly since the final rule does not require EPA to categorically exclude studies for which the data is not available. EPA has explained that this procedural rule has limitations on its scope in terms of conflicts with other substantive requirements and that, in situations where this rule applies, EPA will give greater consideration to those studies where the underlying data and models are available for independent validation. That means that the agency will consider all relevant studies when taking covered actions.
While it is entirely possible that this procedural rule may influence how scientists choose to conduct their studies or present their research to EPA, that potential incidental effect does not render the rule substantive. See James A. Hurson Assocs., Inc. v. Glickman, 229 F.3d 277, 280 (D.C. Cir. 2000) (noting that a procedural rule "covers agency actions that do not themselves alter the rights or interests of parties, although it may alter the manner in which the parties present themselves or their viewpoints to the agency.") (internal quotation marks and citations omitted).
Comment [89-799]: Commenter (11403) writes that the proposed rule is not based on any valid statutory authority and is arbitrary and capricious.
EPA's 2018 version of this rule cited a hodgepodge of statutory authority, none of it valid. EPA at least corrected the obvious typographical errors from the previous version. However, those edits still do not grant EPA the authority to restrict science from all regulatory decision-making processes within the agency. EPA is attempting to use general grants of authority to create a completely novel approach to assessing science. However, as the Supreme Court has stated, "Congress . . . does not alter the fundamental details of a regulatory scheme in vague terms or ancillary provisions -- it does not, one might say, hide elephants in mouseholes." EPA does not have the authority to alter the fundamental details of all of its regulatory schemes based on such vague provisions of law.
Section 103 of the CAA directs EPA to "conduct, and promote the coordination and acceleration of, research, investigations, experiments, demonstrations, surveys and studies" on air pollution. Curtailing the use of science based on arbitrary notions of transparency does the opposite of what Section 103 requires.
Section 301(a) of the CAA gives the Administrator authority to "prescribe such regulations as are necessary to carry out his functions" relating to the CAA. This catch-all provision certainly provides the authority to prescribe regulations, but EPA has not provided a single rationale explaining how the proposed rule is "necessary" to the carrying out of functions under the CAA.
Section 1442 of the SDWA generally authorizes the Administrator to "conduct research, studies, and demonstrations relating to the causes, diagnosis, . . . and other impairments of man resulting directly or indirectly from contaminants in water, or to the provision of dependably safe supply of drinking water . . ." This section does not authorize EPA to exclude science based on a perceived lack of transparency or support any other aspect of the proposed rule.
Section 1450(a)(1) of the SDWA states, "The Administrator is authorized to prescribe such regulations as are necessary or appropriate to carry out his functions under this subchapter." This catch-all provision provides the authority to prescribe regulations, but EPA has not provided a single rationale explaining how the proposed rule is "necessary" to carrying out the functions under the SDWA.
Section 2002(a)(1) of the RCRA authorizes the Administrator to "prescribe, in consultation with Federal, State, and regional authorities, such regulations as are necessary to carry out his functions under this chapter. . ." This catch-all provision provides the authority to prescribe regulations, but EPA has not provided a single rationale explaining how the proposed rule is "necessary" to carrying out functions under the RCRA.
Section 8001 of the RCRA provides EPA with authority to conduct and assist with research relating to the handling of waste management and recovery. The section includes a provision stating, "the Administrator shall establish a management program or system to insure the coordination of all such activities and to facilitate the process of development of sound new technology (or other discoveries) from the research phase, through development, and into the demonstration phase." Not only does section 8001 not provide EPA with authority to promulgate this regulation, the requirement to facilitate the development of new science contradicts the proposed rule's restrictions on science. Limiting scientific research only serves to hinder development of new science, not facilitate it.
Section 115 of the CERCLA and Executive Order 12580 authorize the President to delegate duties under CERCLA and designate EPA as the statute's administrator. This delegation provides the authority to prescribe regulations, but EPA has not provided a single rationale explaining how the proposed rule is necessary to carrying out the functions under CERCLA.
Section 311 of the CERCLA directs the Administrator in the use and development of research and training tools. The section does not mention transparency of data or give any other indication that it authorizes EPA to promulgate the proposed rule.
Section 328 of the EPCRA states, "The Administrator may prescribe such regulations as may be necessary to carry out this chapter." This catch-all provision provides the authority to prescribe regulations, but EPA has not provided a single rationale explaining how the proposed rule is "necessary" to carrying out functions under the EPCRA.
Section 25(a)(1) of the FIFRA authorizes the Administrator "to prescribe regulations to carry out the provisions of this subchapter." This general authorization section of FIFRA differs from that of the other statutes listed because it actually does specify some considerations the Administrator must give when promulgating regulations. For example, it states, "Such regulations shall take into account the difference in concept and usage between various classes of pesticides, including public health pesticides, and differences in environmental risk and the appropriate data for evaluating such risk between agricultural, nonagricultural, and public health pesticides." Unlike the other general grants of rulemaking authority, this provision clearly intends to offer detail and guidance on the nature of the rulemaking authority that the provision grants. As it does not mention restriction of scientific data, there is no indication that EPA has authority to undertake the proposed rule under this section.
Section 136r(a) of the FIFRA directs the Administrator to conduct research to carry out the purposes of the statute. The section states, "The Administrator shall also take care to ensure that such research does not duplicate research being undertaken by any other Federal agency." The section does not state that EPA can ignore sound science if the raw data is not publicly available.
Section 10 of the TSCA grants the Administrator authority on research, development, collection, dissemination, and utilization of data. There is no mention of any type of research or data that warrants exclusion in order to carry out the purposes of the statute.
Section 501 of the CWA states, "The Administrator is authorized to prescribe such regulations as are necessary to carry out his functions under this chapter." This catch-all provision provides the authority to prescribe regulations, but EPA has not provided a single rationale explaining how the proposed rule is "necessary" to carrying out functions under the CWA.
In short, none of the statutory provisions cited by EPA provide authority for the proposed rule. The proposed rule fails at step one under Chevron v. Natural Resources Defense Council since it is unreasonable to believe that any statutory general grant of authority or any mention of EPA's oversight of other science or research programs would enable EPA to conduct rulemaking in a way that is altogether inconsistent with the rule of law. None of the cited provisions provide the authority sought by EPA, and as such the proposed rule should be withdrawn.
Response: See response to comment 29-121.
Comment [93-854]: Commenter (11395) writes that the requirements in the proposed regulation are inconsistent with specifications in several of the environmental statues cited in the SNRPM. In addressing this issue, the SNRPM says, "in the event the procedures outlined in the proposed rulemaking conflict with the statutes that EPA administers, or their implementing regulations, the statutes and regulations will control." However, there is no attempt to delineate those conflicts or to explain how an environmental statute can serve as legal authority for a regulation that conflicts with important provisions of that statute. NESCAUM also strongly disagrees with the application of the Housekeeping Statue as an authority for a regulation with significant, far-reaching implications.
Response: See response to comment 29-121.
Comment [97-900]: Commenter (11479) writes that the proposed rule, as modified in the supplemental notice, would clearly interfere with the agency's use of the best available science. For example, the policy violates laws like the TSCA and the SDWA because of the statutory requirement to use the "best available science." That requirement does not make exception for the agency to effectively ignore certain scientific information of high quality simply because the unanalyzed data cannot be made public for reasons of privacy, confidentiality, or intellectual property. According to judicial precedent, "best available science" is defined as "all existing scientific evidence relevant to the decision" and a body of evidence that "cannot ignore existing data." In addition to the "best available science" requirements, TSCA also requires EPA to take into consideration information "that is reasonably available to the Administrator." EPA's own regulations define that term as "information that EPA possesses or can reasonably generate, obtain, and synthesize . . . , whether or not that information is confidential business information, [sic] that is protected from public disclosure..." Not only would this proposed rule conflict with TSCA's mandate to consider the best available science, it would challenge the method by which EPA can review the weight of the scientific evidence, which in rulemaking for TSCA is defined by a systematic review method. These reviews consider the entire body of scientific evidence, so the exclusion of certain studies based on whether the raw data and models are publicly available would render these reviews, as mandated, impossible.
Further, the CAA has requirements to update pollution standards that provide for an "adequate margin of safety" to protect public health. This determination can only be reliably made using the best available science. This proposal would, however, prevent EPA from using the best available science to set pollution standards that would provide for an adequate margin of safety, because the proposal excludes studies where the raw data and other underlying information cannot be made public. Because of this, EPA should not cite the CAA as a source of statutory authority. Further, the CAA encourages a close partnership with the National Academy of Sciences (NAS) and other independent scientific institutions. When EPA is considering a proposal to reinterpret how it uses science in regulatory decisions, it should first and foremost consider comments from and engage with NAS and similar bodies, which, for this rule, EPA did not.
In fact, virtually all of EPA's statutory mandates require the use of the best available scientific evidence. Congress did not state in any of these laws that the agency shall only consider scientific evidence derived from studies where the underlying unanalyzed data is publicly available, nor that the evidence considered should be solely at the Administrator's discretion.
Response: See response to comment 29-121.
Comment [101-962]: Commenter (11482) writes that the proposed rule likely violates federal laws that require the EPA to use the best scientific information available to it. EPA has a duty to use the best scientific information available under several laws. For example, the TSCA requires EPA to operate "in a manner consistent with the best available science" and make regulatory decisions based on the weight of the scientific evidence. Numerous court cases have weighed in on the best available science standard in the context of numerous different federal laws. For example, courts have found that the agency must use the best science that is available to it at the time of the decision, not the best that might be available to it at some time in the future.
Importantly, the critical actor under the best available science standard is the agency  --  the standard requires the use of the best science that is available to the agency, not necessarily available to the public. Excluding science that is available to the agency but not available to the public would thus be a violation of "best available science" provisions.
Under federal law, the agency must find a way to continue using the best available scientific data and models available to it, regardless of whether those data and models are publicly available. Excluding science whose raw data are not in the public domain runs counter to the "best available science" doctrine and is thus illegal.
Response: See response to comment 29-121.
Comment [114-1002]: Commenter (11392) notes that EPA is proposing that the Agency will only use pivotal science if the data is available in a manner sufficient for independent validation, including, for studies with restricted data and models, if there is tiered access to these data and models. 
EPA addresses the limitations to publishing CBI and personally identifiable information (PII) by proposing "tiered access" to the underlying data and models deemed too sensitive to publish. The SNPRM describes tiered access as "creating multiple versions of a single dataset with varying levels of specificity and protection." This proposal is vague and impracticable. If EPA expects researchers to implement this system, then the Agency is creating enormous new requirements for the research community. 
First, this undermines EPA's claim that the rule "would not regulate the conduct or determine the rights of any entity outside the federal government." This would undercut the already dubious notion that this rule falls under the Agency's authority under the Federal Housekeeping Statute. Furthermore, describing such requirements simply as "multiple versions of a single dataset with varying levels of specificity and protection" is excessively vague for such a monumental overhaul of how the Agency considers science in its rulemaking. If the intent is for the researcher to determine the appropriate level of access, who would determine whether a researcher's determined access levels appropriately met the requirements of the rule? Would researchers be expected to review the applications for reanalysis in perpetuity? Has a cost analysis been done to assess the burden this rule would place on researchers and on the institutions hosting the data? What happens if a research team has since disbanded? Does EPA expect researchers in other countries to work with the Agency to manage such a data access system so their studies can be used by the United States government? If EPA intends for the scientific community to adopt  --  and potentially implement  --  these dataset and access requirements, it is not offering researchers a meaningful description of the parameters on which they can comment. 
The SNPRM references a pilot project with the Centers for Disease Control (CDC) involving a secure data enclave for sensitive EPA datasets. If EPA intends to use CDC resources to implement this rule, the Agency's argument that it is merely an internal procedure matter is far more complicated than stated. Was CDC involved in writing this rule? Do they have the resources to host EPA's data and models and moderate tiered access? Will CDC be making the decision as to who can access the data and at what level? Will CDC be reviewing applications for access? The SNPRM does not address any of these questions. If EPA intends to make its arrangement with the CDC permanent, and establish the CDC as the host of a secure data enclave for studies considered in Agency rulemaking, EPA must clearly articulate its rationale for partnering with a separate federal agency rather than performing the work internally. EPA must also describe how such an unusual arrangement would be structured. It must be clear that the CDC is involved in this rulemaking.
If, instead, the Agency plans on instituting and maintaining a "tiered access" system of its own, a host of implementation questions are unanswered in the text of the supplemental. Where will such a system be housed within the Agency? EPA is not a data management agency  --  does it have the capability to manage a large amount of sensitive data? Does EPA have a robust cybersecurity strategy? The Secret Science Reform Act of 2014, a legislative precursor to this SNPRM, was estimated by the Congressional Budget Office (CBO) to cost "$250 million annually over the next few years" should it have passed into law  --  how much will this SNPRM cost, and what money will EPA use to undertake a project of this magnitude that has not been authorized by Congress? Does EPA have staff who are capable of running this system? How many people will EPA need to hire? How will members of the public apply to access the sensitive data? Who at EPA will determine external access to the data, and at what tier? Who at EPA will determine what parts of the datasets belong in each tier of the system? How will this address restrictions put on PII protected by the Health Insurance Portability and Accountability Act, the National Institutes of Health (NIH), and Institutional Review Boards (IRB), which contain no applicable "tiered access" exemptions from privacy requirements? The Agency's solicitation for public comments on this matter is insufficient given the paucity of details on this proposed solution to the publication limitations of sensitive data. 
Response: This procedural rule does not require researchers to implement the tiered data access system or provide that EPA will use CDC resources to implement that system. See responses to comments 29-121 and 84-1403.
Comment [114-1427]: Commenter (11392) writes in regard to the environmental statutory provisions cited in the proposed rule and the SNPRM, they echo the point made by many commenters in response to the proposed rule: Congress has repeatedly mandated that EPA must use the best available science to inform its regulatory process. This is a legal obligation, a clear-cut directive from Congress, and that supersedes any "general authority... to carry out the Agency's functions" under the various environmental statutes.
Response: See response to comment 29-121.
Comment [123-1046]: Commenter (11491) notes that EPA has requested comment on what legal authority it should rely on for this rulemaking. As EPA acknowledges, it lacks authority to supersede the requirements of its substantive statutes. Nor does the Agency purport to amend its prior regulations interpreting its substantive authorities. While EPA cites its housekeeping authority in Reorganization Plan 3 of 1970, this authority is for internal management purposes only. Accordingly, the Supplemental Notice acknowledges that, where the Proposed Rule conflicts with law or implementing regulations, the law or implementing regulations will control. It does not appear that EPA's substantive statutes or its housekeeping authority authorize the blanket approach to scientific transparency set forth in the Proposed Rule and Supplemental Notice. Therefore, we anticipate that the Proposed Rule will cause confusion and uncertainty in application, which will in fact undermine regulatory transparency and business certainty, a purported goal of this Agency. For this reason, as well, we urge EPA not to finalize this rulemaking. 
Response: See response to comments 29-121 and 84-1403. 
Comment [125-1089]: Commenter (11495) writes that EPA's Supplemental Notice is at odds with provisions of multiple environmental statutes which require the Agency to consider certain high-quality scientific information. For example, the SDWA requires that the issuance of national drinking water regulations be based on the "best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices." TSCA requires the EPA Administrator to ensure that evaluation and regulation of toxic substances is "consistent with the best available science" and to use all scientific information that is "reasonably available" when making related decisions. The CWA and the CAA require that water and air quality criteria determinations be based on the "latest scientific knowledge." Lastly, EPCRA requires that some actions "be based on generally accepted scientific principles or laboratory tests, or appropriately designed and conducted epidemiological or other population studies."
Each of these provisions directs EPA to incorporate a certain quality of scientific information into its rulemaking decisions made pursuant to the statute. In particular, these requirements are designed to ensure the scientific validity of the information upon which EPA bases its actions. Mandating a high-quality scientific basis for regulatory actions is necessary to guarantee the best possible protections for public health and welfare and to minimize partisan influence in scientific decision-making.
The Supplemental Notice, however, would impermissibly prohibit the agency from considering of relevant and high-quality scientific studies whenever the data underlying the study are not publicly available -- in direct contravention of the statutory directives that mandate, for example, consideration of the "best available science." None of the provisions requires, or even suggests, that EPA only consider information that is publicly available. It strains the plain meaning of words like "best," "latest," and even "appropriate" to suggest that they permit the unscientific requirement that all data be publicly available.
EPA "cannot ignore available . . . information." Courts have held that "best available data" requirements "prohibit[] [an agency] from disregarding available scientific evidence that is in some way better than the evidence [it] relies on." Furthermore, a plaintiff may establish a violation of a "best available science" requirement by "point[ing] to any scientific evidence that the agency failed to consider." The Supplemental Notice would undoubtedly preclude EPA from considering certain studies and scientific information that is "better" or the "best available," simply because the study's underlying data is not publicly available, thereby failing to satisfy Congressional directives and undermining the quality of regulatory decisions.
Likewise, EPA cannot avoid the statutory directives to consider high-quality scientific information simply by assigning a lesser weight to such information when the underlying data are not publicly available, as the Agency alternatively proposes. The Supplemental Notice's alternative proposal is designed to have largely the same pernicious effects as flatly prohibiting consideration of studies lacking publicly available data. Just as with the primary proposal, EPA's alternative proposal would limit the agency's consideration of the best available science in making important regulatory decisions, and is, therefore, arbitrary, and capricious.
The effect of the Proposed Rule is inconsistent with EPA's view that the Supplemental Notice is "intended to be consistent with the statutes that EPA administers and EPA plans to implement this procedural rulemaking in accordance with all applicable statutory and regulatory requirements." EPA attempts to resolve this inconsistency by stating that "in the event the procedures outlined in the proposed rulemaking conflict with the statutes that EPA administers, or their implementing regulations, the statutes and regulations will control." However, as discussed below, EPA lacks the statutory authority to issue a rule this broad, even when it claims that it will avoid statutory conflicts. 
Response: See response to comment 29-121.
Comment [125-2205]: Commenter (11495) writes that neither the environmental statutes nor the federal Housekeeping Statute suggests that EPA, in deciding how it will handle scientific information, can reject or attribute lesser weight to scientific evidence that otherwise meets those criteria solely because the underlying data are not publicly available. EPA's Proposed Rule both lacks sufficient legal authority and unlawfully contradicts the environmental statutes that the Agency is tasked with implementing.
Response: See responses to comments 29-121 and 84-1403.
Comment [127-1209]: Commenter (11497) notes that the EPA has an obligation to adhere to the fundamental mandates of its authorizing statutes, which require the agency to promulgate regulatory decisions based on the "best available science." On behalf of the 1 in 8 women who will be diagnosed in their lifetime and the 40,000 lives that are lost each year in the U.S., the EPA must ensure that the best available science informs implementation of the national laws so as to prevent this devastating disease. This proposal seriously undermines EPA's inclusion, consideration and use of the best available science and thus takes a hard step away from that goal.
Response: See response to comment 29-121.
Comment [130-1137]: Commenter (11884) writes that presumably, EPA hopes to evade the important environmental laws (including NEPA provisions) by styling the Trio as mere housekeeping provisions, but nothing in the history of the housekeeping statute suggests that Congress intended it to permit agencies to undercut statutes that enshrine important policies like CERCLA, FIFRA, NEPA, RCRA, and TSCA. Accordingly, the SNPRM and Trio are neither authorized by the Housekeeping statute nor, if they were, could be given effect if given the Trio's conflict with such important environmental laws.
Response: See responses to comments 29-121 and 84-1403.
Comment [131-1884]: Commenter (11890) writes that this rule and supplement constitute a dramatic change to the data the agency will consider as part of its science assessment and regulatory work, and as such would be inappropriate and illegal to adopt under internal housekeeping rules. We believe the proposed rule could have serious adverse impacts across the wide range of environmental laws that are critical to protecting public health.
Response: A rule is substantive if it has legal force and effect on individual rights and obligations. See Chrysler Corp. v. Brown, 441 U.S. 281, 302 (1979). As discussed in the final rule preamble at Section I. C, this is a procedural rule governing EPA's internal affairs. It does not regulate or affect the legal rights and obligations of parties outside of the agency, and therefore is not a substantive rule. 
A rule is not substantive solely because it is a fundamental change of agency policy. The cases so holding are distinguishable from these circumstances. In Mendoza v. Perez, 754 F.3d 1002, 1021 (D.C. Cir. 2014), plaintiffs challenged a series of letters issued by the Department of Labor (DOL) which set forth special procedures for certifying foreign agricultural workers as temporary workers under the H-2A visa program. The court stated that "[a] rule is [substantive] if it supplements a statute, adopts a new position inconsistent with existing regulations, or otherwise effects a substantive change in existing law or policy." However, that statement was in response to the defendants' claim that the letters merely interpret existing regulations. The court's point there was that the letters do much more than that. 
The court also cited Shalala v. Guernsey Mem'l Hospital, 514 U.S. 87, 100 (1995) for the proposition that rules that represent a major shift in law or policy are substantive. In that case, the Supreme Court agreed with defendants that the rule at issue was an interpretive rule but acknowledged that notice and comment would have been required had the rule adopted a position inconsistent with the agency's existing regulations.
Thereafter, the court turned to defendants' alternative argument that the letters are procedural rules. As part of that discussion, the court pointed out that "[p]rocedural rules `do not themselves alter the rights or interests of parties, although [they] may alter the manner in which the parties present themselves or their viewpoints to the agency.'" Id. at 1023 (quoting Batterton v. Marshall, 648 F.2d 694, 707 (D.C. Cir. 1980) (second alteration in original). The court then observed that the letters "do not merely describe how the Department will evaluate H-2A applications, but they set the bar for what employers must do to obtain approval." The court ultimately held that in so doing, the letters substantially affect the rights of interested private parties. In other words, the court did not hold that the letters were substantive rules because they changed Departmental policy, but rather because they imposed requirements on persons outside of the agency.
The 10th Circuit stated in Rocky Mountain Helicopters, Inc. v. F.A.A., 971 F.2d 544, 547 (10th Cir. 1992), that a rule is substantive if it "changes existing law, policy, or practice" out of context. In that case, the Federal Aviation Administration (FAA) informed the plaintiff that its pilots would no longer be allowed to wear night vision goggles. However, in that case, the FAA claimed that the letter was an interpretive rule, as opposed to a substantive rule, and the court ultimately agreed on grounds that the letter did not change existing law, policy, or practice. So again, the broad statement that commenters rely on is in the context of a court determining whether a rule is interpretive, not procedural. It makes sense for courts to point out where an allegedly interpretive rule does more than simply articulate the agency's interpretation of existing regulations.
Finally, substantive rules are not the only means for agencies to effect a change in existing policy. Agencies can also issue procedural and interpretive rules that articulate a policy shift. Nevertheless, the APA clearly differentiates procedural and interpretive rules and statements of policy from substantive rules. See 5 U.S.C. § 553 (exempting procedural and interpretive rules from notice and comment procedures). 
Comment [133-1160]: Commenter (11894) writes that EPA now claims authority to base its data restriction proposal not just on the provisions of specific EPA laws that we addressed in our opening comments, but also on the general federal Housekeeping Statute, 5 U.S.C. § 301. This provision, enacted by the first Congress, provides that the head of an executive department or military department may prescribe regulations for the government of his department, the conduct of his employees, the distribution and performance of its business, and the use and preservation of its records, papers, and property. As the Supreme Court has recognized, this language only authorizes rules that govern internal agency operations and cannot be used to justify a legislative rule. In an attempt to bring the proposed science restrictions within that purpose, EPA's proposal states that it does not regulate any entity outside the federal government. Rather, they claim that the proposed requirements would modify the EPA's internal procedures regarding the transparency of science underlying regulatory decisions and that this rule would not regulate the conduct or determine the rights of any entity outside the federal government. Their position is that it exclusively pertains to the internal practices of the EPA. (85 FR 15398).
EPA specifically states that its proposal does not interpret or rely on the specific regulatory provisions of the various EPA statutes, despite the flood of comments it received demonstrating that these were indisputably the governing statutory authorities. But in a backhanded acknowledgment of a possible problem, the proposal states that this internal agency procedure is intended to be consistent with the statutes that EPA administers and EPA plans to implement this procedural rulemaking in accordance with all applicable statutory and regulatory requirements. Nonetheless, it says that in the event the procedures outlined in the proposed rulemaking conflict with the statutes that EPA administers, or their implementing regulations, the statutes and regulations will control. Id.
Response: See response to comment 29-121.
Comment [133-1171]: Commenter (11894) writes that EPA's proposal does not discuss any substantive provisions of regulatory statutes. However, it attempts to insulate itself against conflict with them by providing that:
[I]n the event the procedures outlined in the proposed rulemaking conflict with the statutes that EPA administers, or their implementing regulations, the statutes and regulations will control.
EPA provides neither substantive legal standards nor procedures for making this shell game of a determination. Rather, it is an entirely lawless and cynical ploy. It calls on EPA to "balance" between an approach to data evaluation that is illegal because it rests neither on statutory authority nor on responsible analysis and the approach the law prescribes. The next step will be to argue that because EPA has given itself authority to sometimes not do the illegal thing, its authority to do the illegal thing should be sustained.
All this shows that EPA is well aware of the probability that the authorities on which it rests its proposal do not support it, and that any rule remotely like its proposal could only rest on the substantive provisions of the various regulatory statutes describing the types of data that can be used as rulemaking support.
It also makes clear how EPA will try to defuse this issue without ever actually confronting it. If such an "in case of conflict" provision were to take effect, EPA quite predictably would use it to uphold its data bar in most cases, hoping to prevail not on the merits, but because of the presumption of regularity given to government action and lack of opposition due to the expense and uncertainty of differing with the government. In extreme cases, EPA could back down. And if it should lose in an individual case, that could always be blamed on misapplication of the "in case of conflict" provision, not on any defect in its structure. This gambit does nothing to preserve the rule's legality, failing for the same reason that the initially proposed case-by-exception fails: a subsequent promise to cure an arbitrary result does nothing to cure that illegality. See Ameren Services v FERC 880 F. 3d 571, 584 (DC Cir 2018):
We once described an agency's effort to offer future rulemaking as a response to a claim of agency illegality as an "administrative law shell game," Am. Tel. & Tel. Co. v. FCC, 978 F.2d 727, 732 (D.C. Cir. 1992), a phrase the Supreme Court thought apt. See MCI Telecomm. v. Am. Tel. & Tel. Co., 512 U.S. 218, 222, 114 S.Ct. 2223, 129 L.Ed.2d 182 (1994).
Compounding this illegality is the absence of any discussion of either the substantive criteria or the procedural mechanisms by which EPA would determine whether this censored-science rule would be inconsistent with its actual statutory authorities. An agency's empty reference to adherence to statutory factors is not reasoned decision-making. See Philadelphia Gas Works v. FERC, 989 F. 2d 1246, 1251 (D.C. Cir. 1993 ) (for FERC to utter the words 'unique facts and circumstances' and 'equity,' ... as a wand waved over an undifferentiated porridge of facts, leaves regulated parties and a reviewing court completely in the dark as to the core of FERC's reasoning and its relationship to past precedent"); Verizon Cos. v. FCC , 570 F. 3d 294, 304 (D.C. Cir. 2009) ("These conclusory statements that such factors are being considered cannot substitute for the reasoned explanation that is wanting in this decision"); American Gas Ass'n, v. FERC, 593 F. 3d 14, 19 (D.C. Cir. 2010) ("A passing reference to relevant factors, however, is not sufficient to satisfy the Commission's obligation to carry out reasoned and principled decision-making. The Commission must fully articulate the basis for its decision.")
Response: See response to comment 48-503.
Comment [133-1176]: Commenter (11894) asks, what are the legal authorities under which EPA prepares the studies that would be affected, and how do they support this step? All government actions must be authorized by law. Yet this proposal does not even mention that topic, much less discuss how the statutory language (whatever it might be) supports such restrictions on studies prepared under its authority
Response: EPA discussed its legal authority for this action in the preamble of the proposal at Section I. C and modified that authority as discussed in the preamble of the supplemental notice at Section I. B, and the preamble of the final rule at Section I. C. 
Comment [141-1273]: Commenter (11911) states that there's no doubt the finalization of the Strengthening Transparency in Regulatory Science (STRS) Rule would have a substantive impact by severely impairing the EPA's ability to protect public health and the environment. 
If finalized, the STRS Rule would substantively violate a number of environmental statutes. Both the ESA and, for example, call on the EPA to use the "best scientific... data available" when issuing regulations; the TSCA requires the "best available science." This mandate means the agency cannot "disregard[] available scientific evidence that is in some way better than the evidence [it] relies on." The agency must not "ignore available studies, even if it disagrees with or discredits them." In another example, NAAQS under the CAA air must be set at a level "requisite to protect public health," and must "accurately reflect the latest scientific knowledge" including "all identifiable effects [of the pollutant] on public health or welfare." 
 
Further, under the APA, an agency must not "fail[] to consider an important aspect of the problem" or neglect to "consider[] relevant factors" when issuing regulation. If the EPA were to ignore whole categories of scientific studies in reaching a regulatory decision, the agency would surely run afoul of longstanding Supreme Court precedent. The Proposed STRS Rule and accompanying SNPRM cannot constitute an end-run around the procedural requirements of the APA or the Congressional mandate of environmental statutes. 
Response: Any impacts on third parties that result from this final rule are not enough to render it substantive. See responses to comments 29-121 and 84-1403. This final rule will enhance EPA's ability to consider the pivotal science underlying its significant regulatory decisions and ISI in the most transparent way possible. As discussed at Section III. E of the final rule preamble, EPA will apply a weighting approach when considering pivotal science, not a categorical exclusion. EPA has explained that this procedural rule has limitations on its scope in terms of conflicts with other substantive requirements and that EPA will give greater consideration to those studies where the underlying data and models are available for independent validation. That means that the agency will consider all relevant studies when taking covered actions. 
Comment [147-1315]: Commenter (12399) notes that the supplemental states that the rule would not apply when its provisions "conflict" with statutory law. Obviously, laws prevail over agency rules, so this stated exclusion is not necessary. In any case, it is not clear that any law on the books would prevent the agency from ensuring that any science it uses meets basic scientific transparency standards. And in the unlikely situation that questions arise, this issue should remain within the purview of the courts to determine whether a conflict exists. Including this provision appears to do nothing more than give the Administrator an excuse to waive the rule for rather arbitrary reasons.
Response: See response to comment 48-503. Here, the Administrator's discretion to exempt certain studies on an ad hoc basis from the final rule's requirements comes from section 30.9 of the final rule, not the savings clause. That section clearly outlines the circumstances under which the Administrator may grant exemptions, enumerates several factors that should inform that determination, and requires the Administrator to document the rationale for the exemption.
Comment [159-1404]: Commenter (12430) writes that U.S. EPA administers and enforces most of the country's foundational environmental laws, including the CAA, CWA, SDWA, RCRA, CERCLA, EPCRA, FIFRA, and TSCA. These laws include statutes that expressly require U.S. EPA to consider the best available science. If adopted, the SNPRM would violate these statutes.
Response: See response to comment 29-121.
Comment [159-1406]: Commenter (12430) writes that the SNPRM does not address or even acknowledge the statutes that affirmatively require U.S. EPA to consider the best available science. It instead invokes additional statutes that it claims provide authority for the proposed rule. It argues that the rule may be authorized by the Federal Housekeeping Statute, by certain substantive statutes, or by a combination of the two. But the statutes the SNPRM cites do not address the use of science and certainly do not allow U.S. EPA to disregard its express statutory mandates to use best available science.
Response: See response to comment 29-121.
Comment [175-1574]: Commenter (12471) writes that environmental statutes separately provide the Administrator with substantive policy-making authority to weigh information appropriately in the regulatory process. Increased transparency resulting from the Transparency Rule's procedural framework may be one of the factors that the Administrator considers when using that authority. The Transparency Rule's framework would remain purely procedural. Even when considered as part of the regulatory process specified in an environmental statute, the framework would add no new substantive requirements to that process.
Increasing transparency assessments does not diverge from the requirement that the Administrator use the best science in reaching regulatory decisions. In fact, the opposite is true. Scientific transparency is critical to reproducibility, which is, in turn, a central tenant of the scientific process. Without a common framework to increase transparency, the Administrator's process to factor transparency and reproducibility, core components of the scientific method, in the regulatory process could be arbitrary and capricious.
Response: EPA agrees that this final rule is purely procedural and directs EPA to weigh public availability of the pivotal science underlying EPA's significant regulatory decisions among other factors enumerated in the final rule. Regarding the environmental statutes that EPA administers, see response to comment 29-121.
Comment [181-1622]: Commenter (12699) writes that the EPA should rely on both the substantive environmental statutes and the housekeeping authority.
The EPA would not be implementing these statutes properly if it were using flawed science in the promulgation of its rules. To highlight one example, it is useful to reexamine the U.S. Supreme Court case Michigan v. EPA. In discussing the "appropriate and necessary" language in 112(n)(1)(A) of the CAA, the U.S. Supreme Court argued "[o]ne would not say that it is even rational, never mind "appropriate," to impose billions of dollars in economic costs in return for a few dollars in health or environmental benefits."
Similarly, it would not be rational for the EPA to use science for these very costly policies without even knowing whether the science is credible. Nor would it be rational for the agency to fail to take every possible step to ensure scientific accuracy, including making underlying information available to the public (especially since the EPA is well aware of the problems that pervade the science). Making the information available to the public is not some nice gesture to appease the public. It is a critical step to help the agency itself get necessary feedback to better assess the science.
Response: EPA has elected to rely solely on housekeeping authority equivalent to the Federal Housekeeping authority, which was transferred to EPA in EPA's Reorganization Plan. See response to comment 29-121.
EPA would like to clarify that the final rule does not require EPA or the public to make data and models publicly available. However, EPA agrees that it is important for the pivotal science underlying EPA's significant regulatory decisions and ISI to be as transparent as possible. 
Comment [181-1624]: Commenter (12699) writes that there is also a practical reason for utilizing these environmental statutes as statutory authority for the rule. It should help affected parties to enforce EPA's compliance with the rule. Relying upon the housekeeping authority alone might make this regulation covering internal procedures more difficult to enforce. Regarding this enforcement concern, the EPA should learn some lessons from the Information Quality Act (IQA). It is extremely important that the rule itself have clear and objective requirements, without creating so much flexibility and discretion that there is nothing for a court to enforce. In addition, by having clear requirements, this will limit future administrations from getting around the rule.
To its credit, the EPA is undertaking this extremely important transparency initiative. It would do a disservice to the public and the agency itself if the rule has no "teeth" and is simply ignored whenever it is convenient for future administrations.
Response: See response to comment 29-121.
Comment [183-1822]: Commenter (12702) notes that while the Supplemental Notice is proposing to allow non-public data to be used when public data are unavailable, the notice makes clear that EPA would have wide discretion to determine whether and when to do so. They remain concerned that this discretion could decrease the consistency and certainty of how certain data are treated. Additionally, such treatment could conflict with requirements under existing statutes and Court decisions. This concern applies even if EPA chooses to rely upon its housekeeping authority as proposed in the supplemental notice.
Response: Section 30.9 of the final rule outlines the circumstances under which the Administrator may grant exemptions, enumerates several factors that should inform that determination, and requires the Administrator to document the rationale for the exemption. Conflict analyses will be conducted and discussed as appropriate when EPA develops subsequent rulemakings implementing this final procedural rule. See the final rule preamble discussion at Section III.G.
Comment [184-1660]: Commenter (12707) writes that the proposed structure is arbitrary and capricious and will lead to outcomes that do not fulfill the agency's statutory mandate and will, therefore, be vulnerable to legal challenge. EPA should continue to apply the rigorous standards the agency has used over many decades of issuing lifesaving standards and let stakeholders engage in the process that is full and open with regards to science.
By setting forth unnecessary and confusing restrictions on the kinds of studies it can consider in developing significant rules under critical environmental and public health laws from air pollution standards to water quality issues to chemical regulation in consumer products, EPA is undermining its ability and duty to meet statutory obligations.
Response: As discussed in the final rule preamble at Section III.D, EPA will apply a weighting approach when considering pivotal science, not any categorical exclusions. EPA has explained that this procedural rule has limitations on its scope in terms of conflicts with other substantive requirements and that in situations where this rule applies, EPA will give greater consideration to those studies where the underlying data and models are available for independent validation. That means that the agency will consider all relevant studies when taking covered actions. Regarding EPA's obligations under the statutes it administers, see response to comment 29-121.
Comment [184-1661]: Commenter (12707) writes that the Federal Housekeeping Statute alone or in conjunction with existing bedrock environmental laws does not authorize EPA to limit science considered in rulemaking processes. EPA's supplemental proposal is a second failed attempt at justifying an effort to limit or otherwise constrain its own ability to use the best available public health studies and science from outside EPA. The effort in the initial proposal to ground so-called "transparency" in bedrock environmental laws was exposed as untenable. EPA's addition of the Federal Housekeeping Statute to the mix and recitation of new sections of several environmental laws does not strengthen EPA's case. Whether or not EPA has 301 housekeeping authority is irrelevant. The statute authorizes EPA to control its own internal documents or materials; the statute and its authorities are not applicable to the long-standing public health studies EPA has relied upon to protect public health through the notice and comment rulemaking process.
EPA's justification that this action to constrain science or leave science up to the political whims of the administrator is about internal agency procedure does not hold. This proposal is not about internal paperwork, but about whether or not EPA can fulfill its core mission consistent with bedrock environmental laws.
Response: See response to comment 29-121.
Comment [184-1662]: in addition to the Housekeeping Statute, the supplemental proposal points to specific provisions of several key laws (RCRA, CERCLA and the CWA) as a justification for limiting or constraining the science it can consider. Nowhere in the cited statutes or sections is there any basis for demanding access to raw data or otherwise disqualifying long standing studies, nor does this relate sensibly to any definition of best available science. Rather it undermines the use of best available science as called for in numerous environmental statutes including the CAA.
The CAA, for example, has requirements to update pollution standards that provide for an adequate margin of safety for public health. This determination can only be reliably made using the best available science. However, this proposal would prevent EPA from using that information to set science-based pollution standards that would provide for an adequate margin of safety. EPA should not cite the CAA as an appropriate source of statutory authority. The added confusion of the supplemental proposal, including so-called tiered access or EPA's ability to consider some research over what has long been considered "best available science" based upon the availability of data, does not legitimize the proposal.
In sum, there is no statutory authority, with or without the Housekeeping Statute, for EPA to rely on to censor or constrain science. Further, finalizing this effort will only serve to undermine EPA's essential role and create confusion over which studies will be considered and how they will be valued to set standards that directly impact public health and the quality of our air and water. Many of these uncertainties are recognized in EPA's own Science Advisory Board's April 28, 2020 letter.
Response: See response to comment 29-121.
Comment [187-1701]: Commenter (12720) writes that the law is clear that EPA may adopt rules only if those rules are based on statutory authority delegated by Congress. EPA may not invent statutory authority where none exists, nor adopt regulations lacking statutory authority merely because EPA believes that to be better policy. See, e.g., Massachusetts v. EPA, 549 U.S. 497, 535, 127 S. Ct. 1438, 1463 (2007) ("EPA must ground its reasons for action or inaction in the statute."); La. Pub. Serv. Comm'n v. FCC, 476 U.S. 355, 374 (1986) (stating "agency power to act" is shaped by how "Congress confers power upon it"). Agencies need especially clear congressional delegations of authority to create regulatory exemptions. See New York v. U.S. EPA, 413 F.3d 3, 41 (D.C. Cir. 2005) (stating that the agency needs "clear congressional delegation" to support an exemption). EPA identifies no such delegations, certainly not the clear delegations required by law, for either the 2018 Proposal or the Supplemental Proposal. 
Response: See response to comment 29-121.
Comment [187-1714]: Commenter (12720) writes that EPA attempts to paper over this arbitrary abuse of discretion by noting that the rulemaking is intended to "be consistent with the statutes that EPA Administers and ... all applicable statutory and regulatory requirements." Id. The agency goes on to note that "in the event the procedures outlined in this proposed rulemaking conflict with the statute that EPA administers, or their implementing regulations, the statutes and regulations will control." Id. While it is certainly true that a rulemaking that conflicts with statutory authority would be struck down as arbitrary and capricious, EPA's statements here do nothing to "fix" the Proposal nor render it a valid exercise by the Agency. An administrative action that is arbitrary and capricious and an abuse of discretion does not change character through this type of Agency statement  -  the EPA does not get to pick and choose the statutory authorities it chooses to follow. Rather, Congress makes laws, and those laws contain certain proscriptions and prescriptions that the Agency to whom Congress has delegated rulemaking authority must follow. Implicit in EPA's recognition of the potential for "conflict" with statutes and implementing regulations is the understanding that this rule is arbitrary, capricious, and an abuse of the Agency's discretion. Neither the substantive statutes described above, nor the housekeeping authority, authorize the rulemaking.
Response: See response to comment 48-503.
Comment [187-1717]: Commenter (12720) writes that the Supplemental Proposal and original Proposal lack any statutory authority or other legal authority to adopt any regulation based on the proposed regulatory definition, "Capable of being substantially reproduced." 85 Fed. Reg. at 15,405 (proposed 40 CFR § 30.2). "5 U.S.C. 301; 5 U.S.C. App.; Pub. L. 98 - 80, 84 Stat. 2086" in no way provides that authority. See supra at III.I of commenter's letter. Nor do any of the organic environmental statutes that EPA cites in the Proposal or Supplemental Proposal.
Response: See response to comment 29-121.
Comment [187-1720]: Commenter (12720) writes that the Supplemental Proposal and original Proposal lack any statutory authority or other legal authority to adopt any regulation based on the proposed regulatory definition, "Data." 85 Fed. Reg. at 15,405 (proposed 40 CFR § 30.2). "5 U.S.C. 301; 5 U.S.C. App.; Pub. L. 98 - 80, 84 Stat. 2086" in no way provides that authority. See supra at III.I of commenter's letter. Nor do any of the organic environmental statutes that EPA cites in the Proposal or Supplemental Proposal.
Response: See response to comment 29-121.
Comment [187-1721]: Commenter (12720) writes that the Supplemental Proposal and original Proposal lack any statutory authority or other legal authority to adopt any regulation based on the proposed regulatory definition, "Independent validation." 85 Fed. Reg. at 15,405 (proposed 40 CFR § 30.2). "5 U.S.C. 301; 5 U.S.C. App.; Pub. L. 98 - 80, 84 Stat. 2086" in no way provides that authority. See supra at III.I of the commenters' comment letter. Nor do any of the organic environmental statutes that EPA cites in the Proposal or Supplemental Proposal.
Response: See response to comment 29-121.
Comment [187-1724]: Commenter (12720) writes that the Supplemental Proposal and original Proposal lack any statutory authority or other legal authority to adopt any regulation based on the proposed regulatory definition, ISI. 85 Fed. Reg. at 15,405 (proposed 40 CFR § 30.2). "5 U.S.C. 301; 5 U.S.C. App.; Pub. L. 98 - 80, 84 Stat. 2086" in no way provides that authority. See supra at III.I of the commenters' comment letter. Nor do any of the organic environmental statutes that EPA cites in the Proposal or Supplemental Proposal.
Response: See response to comment 29-121.
Comment [187-1725]: Commenter (12720) writes that the Supplemental Proposal and original Proposal lack any statutory authority or other legal authority to adopt any regulation based on the proposed regulatory definition, "Model." 85 Fed. Reg. at 15,405 (proposed 40 CFR § 30.2). "5 U.S.C. 301; 5 U.S.C. App.; Pub. L. 98 - 80, 84 Stat. 2086" in no way provides that authority. See supra at III.I of the commenters' comment letter. Nor do any of the organic environmental statutes that EPA cites in the Proposal or Supplemental Proposal.
Response: See response to comment 29-121.
Comment [187-1727]: Commenter (12720) writes that the Supplemental Proposal and original Proposal lack any statutory authority or other legal authority to adopt any regulation based on the proposed regulatory definition, "Pivotal science." 85 Fed. Reg. at 15,405 (proposed 40 CFR § 30.2). "5 U.S.C. 301; 5 U.S.C. App.; Pub. L. 98 - 80, 84 Stat. 2086" in no way provides that authority. See supra at III.I of the commenters' comment letter. Nor do any of the organic environmental statutes that EPA cites in the Proposal or Supplemental Proposal.
Response: See response to comment 29-121.
Comment [187-1729]: Commenter (12720) writes that the Supplemental Proposal and original Proposal lack any statutory authority or other legal authority to adopt any regulation for the proposed regulatory definition, "The Supplemental Proposal and original Proposal lack any specific statutory authority or other legal authority to adopt any regulation based on the proposed regulatory definition, "Publicly available." 85 Fed. Reg. at 15,405 (proposed 40 CFR § 30.2). "5 U.S.C. 301; 5 U.S.C. App.; Pub. L. 98 - 80, 84 Stat. 2086" in no way provides that authority." 85 Fed. Reg. at 15,405 (proposed 40 CFR § 30.2). "5 U.S.C. 301; 5 U.S.C. App.; Pub. L. 98 - 80, 84 Stat. 2086" in no way provides that authority. See supra at III.I of the commenters' comment letter. Nor do any of the organic environmental statutes that EPA cites in the Proposal or Supplemental Proposal.
Response: See response to comment 29-121.
Comment [187-1730]: Commenter (12720) writes that the Supplemental Proposal and original Proposal lack any statutory authority or other legal authority to adopt any regulation for the proposed regulatory definition, "The Supplemental Proposal and original Proposal lack any specific statutory authority or other legal authority to adopt any regulation based on the proposed regulatory definition, "Publicly available." 85 Fed. Reg. at 15,405 (proposed 40 CFR § 30.2). "5 U.S.C. 301; 5 U.S.C. App.; Pub. L. 98 - 80, 84 Stat. 2086" in no way provides that authority." 85 Fed. Reg. at 15,405 (proposed 40 CFR § 30.2). "5 U.S.C. 301; 5 U.S.C. App.; Pub. L. 98 - 80, 84 Stat. 2086" in no way provides that authority. See supra at III.I of the commenters' comment letter. Nor do any of the organic environmental statutes that EPA cites in the Proposal or Supplemental Proposal.
Response: See response to comment 29-121.
Comment [187-1731]: Commenter (12720) writes that the Supplemental Proposal and original Proposal lack any statutory authority or other legal authority to adopt any regulation based on the proposed regulatory definition, "Reanalyze." 85 Fed. Reg. at 15,405 (proposed 40 CFR § 30.2). "5 U.S.C. 301; 5 U.S.C. App.; Pub. L. 98 - 80, 84 Stat. 2086" in no way provides that authority. See supra at III.I of the commenters' comment letter. Nor do any of the organic environmental statutes that EPA cites in the Proposal or Supplemental Proposal.
Response: See response to comment 29-121.
Comment [187-1733]: Commenter (12720) writes that the Supplemental Proposal and original Proposal lack any statutory authority or other legal authority to adopt any regulation based on the proposal to apply "the provisions of this part" to "data and models, underlying pivotal science supporting influential scientific information and/or underlying pivotal regulatory science used to justify significant regulatory decisions regardless of the source of funding or identity of the party conducting the science." 85 Fed. Reg. at 15,405 (proposed 40 CFR § 30.3). "5 U.S.C. 301; 5 U.S.C. App.; Pub. L. 98 - 80, 84 Stat. 2086" in no way provides that authority. See supra at III.I of the commenters' comment letter. Nor do any of the organic environmental statutes that EPA cites in the Proposal or Supplemental Proposal.
Response: See response to comment 29-121.
Comment [187-1734]: Commenter (12720) writes that the Supplemental Proposal and original Proposal lack any statutory authority or other legal authority to adopt any regulation based on the proposal to condition "EPA's use of data and models underlying pivotal regulatory science and pivotal science" on any of the criteria, restrictions or provisions in proposed section 30.5, Option 1. 85 Fed. Reg. at 15,405 (proposed 40 CFR § 30.5, Option 1). "5 U.S.C. 301; 5 U.S.C. App.; Pub. L. 98 - 80, 84 Stat. 2086" in no way provides that authority. See supra at III.I of the commenters' comment letter. Nor do any of the organic environmental statutes that EPA cites in the Proposal or Supplemental Proposal.
The Supplemental Proposal and original Proposal lack any statutory authority or other legal authority to adopt any regulation based on the proposal that, "[w]hen promulgating significant regulatory decisions or finalizing influential scientific information, the Agency will only use pivotal regulatory science and/or pivotal science that includes studies with restricted data and models (i.e., those that include CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects) if there is tiered access to these data and models in a manner sufficient for independent validation, and studies that do not include restricted data and models if the data and models are publicly available in a manner sufficient for independent validation." 85 Fed. Reg. at 15,405 (proposed 40 CFR § 30.5, Option 1). "5 U.S.C. 301; 5 U.S.C. App.; Pub. L. 98 -  80, 84 Stat. 2086" in no way provides that authority. See supra at III.I of the commenters' comment letter. Nor do any of the organic environmental statutes that EPA cites in the Proposal or Supplemental Proposal.
The Supplemental Proposal and original Proposal lack any statutory authority or other legal authority to adopt any regulation based on the proposal that, "Information is considered `available in a manner sufficient for independent validation' when it includes the information necessary to understand, assess, and reanalyze findings," including the six listed examples in proposed section 30.5(a) through (f). 85 Fed. Reg. at 15,405 (proposed 40 CFR § 30.5, Option 1). "5 U.S.C. 301; 5 U.S.C. App.; Pub. L. 98 - 80, 84 Stat. 2086" in no way provides that authority. See supra at III.I of the commenters' comment letter. Nor do any of the organic environmental statutes that EPA cites in the Proposal or Supplemental Proposal.
The Supplemental Proposal and original Proposal lack any statutory authority or other legal authority to adopt any regulation based on the proposal to condition "EPA's use of data and models underlying pivotal regulatory science and pivotal science" on any of the criteria, restrictions or provisions in proposed section 30.5, Option 2. 85 Fed. Reg. at 15,405 (proposed 40 CFR § 30.5, Option 2). "5 U.S.C. 301; 5 U.S.C. App.; Pub. L. 98 - 80, 84 Stat. 2086" in no way provides that authority. See supra at III.I of the commenters' comment letter. Nor do any of the organic environmental statutes that EPA cites in the Proposal or Supplemental Proposal.
The Supplemental Proposal and original Proposal lack any statutory authority or other legal authority to adopt any regulation based on the proposal that, "When promulgating significant regulatory decisions or finalizing influential scientific information, the Agency will, other things equal, give greater consideration to studies where the underlying data and models are publicly available in a manner sufficient for independent validation." 85 Fed. Reg. at 15,405 (proposed 40 CFR § 30.5, Option 2). "5 U.S.C. 301; 5 U.S.C. App.; Pub. L. 98 - 80, 84 Stat. 2086" in no way provides that authority. See supra at III.I of the commenters' comment letter. Nor do any of the organic environmental statutes that EPA cites in the Proposal or Supplemental Proposal.
The Supplemental Proposal and original Proposal lack any statutory authority or other legal authority to adopt any regulation based on the proposal that, "[w]hen promulgating significant regulatory decisions or finalizing influential scientific information," "[t]he Agency will also give greater consideration to studies based on data and models that include confidential business information, proprietary information or PII if these data and models were available through restricted access, such as through a secure data enclave, in a manner sufficient for independent validation." 85 Fed. Reg. at 15,405 (proposed 40 CFR § 30.5, Option 2). "5 U.S.C. 301; 5 U.S.C. App.; Pub. L. 98 - 80, 84 Stat. 2086" in no way provides that authority. See supra at III.I of the commenters' comment letter. Nor do any of the organic environmental statutes that EPA cites in the Proposal or Supplemental Proposal.
The Supplemental Proposal and original Proposal lack any statutory authority or other legal authority to adopt any regulation based on the proposal that, "[w]hen promulgating significant regulatory decisions or finalizing influential scientific information," "the Agency will identify those studies that are given greater consideration and provide a short description of why greater consideration was given." 85 Fed. Reg. at 15,405 (proposed 40 CFR § 30.5, Option 2). "5 U.S.C. 301; 5 U.S.C. App.; Pub. L. 98 - 80, 84 Stat. 2086" in no way provides that authority. See supra at III.I of the commenters' comment letter. Nor do any of the organic environmental statutes that EPA cites in the Proposal or Supplemental Proposal.
The Supplemental Proposal and original Proposal lack any statutory authority or other legal authority to adopt any regulation based on the proposal that, "Information is considered `available in a manner sufficient for independent validation' when it includes the information necessary to understand, assess, and reanalyze findings," including the 5 listed examples in proposed section 30.5(a)(1) through (5). 85 Fed. Reg. at 15,405-406 (proposed 40 CFR § 30.5, Option 2). "5 U.S.C. 301; 5 U.S.C. App.; Pub. L. 98 - 80, 84 Stat. 2086" in no way provides that authority. See supra at III.I of the commenters' comment letter. Nor do any of the organic environmental statutes that EPA cites in the Proposal or Supplemental Proposal.
The Supplemental Proposal and original Proposal lack any statutory authority or other legal authority to adopt any regulation based on the proposal that "[t] he provisions of this section apply to data and models underlying pivotal regulatory science or pivotal science regardless of who funded or conducted the underlying data, models, or other regulatory science or pivotal science." 85 Fed. Reg. at 15,406 (proposed 40 CFR § 30.5, Option 2). "5 U.S.C. 301; 5 U.S.C. App.; Pub. L. 98 - 80, 84 Stat. 2086" in no way provides that authority. See supra at III.I of the commenters' comment letter. Nor do any of the organic environmental statutes that EPA cites in the Proposal or Supplemental Proposal.
The Supplemental Proposal and original Proposal lack any statutory authority or other legal authority to adopt any regulation based on the proposal that "[w]hen promulgating significant regulatory decisions or finalizing influential scientific information," and "[w]here data and models are controlled by third parties, EPA may work with those parties to endeavor to make the data and models available in a manner that complies with this section." 85 Fed. Reg. at 15,406 (proposed 40 CFR § 30.5, Option 2) (emphasis added). "5 U.S.C. 301; 5 U.S.C. App.; Pub. L. 98 - 80, 84 Stat. 2086" in no way provides that authority. See supra at III.I of the commenters' comment letter. Nor do any of the organic environmental statutes that EPA cites in the Proposal or Supplemental Proposal.
Response: See response to comment 29-121.
Comment [187-1735]: Commenter (12720) writes that the Supplemental Proposal and original Proposal lack any statutory authority or other legal authority to adopt any regulation based on the proposal to condition "the use of data and models underlying pivotal regulatory science and pivotal science" on any of the criteria, restrictions or provisions in proposed section 30.6. 85 Fed. Reg. at 15,406 (proposed 40 CFR § 30.6). "5 U.S.C. 301; 5 U.S.C. App.; Pub. L. 98 - 80, 84 Stat. 2086" in no way provides that authority. See supra at III.I of the commenters' comment letter. Nor do any of the organic environmental statutes that EPA cites in the Proposal or Supplemental Proposal.
The Supplemental Proposal and original Proposal lack any statutory authority or other legal authority to adopt any regulation based on the proposal to "give explicit consideration to high quality studies, including but not limited to those that explore: A broad class of parametric dose-response or concentration-response models; a robust set of potential confounding variables; nonparametric models that incorporate fewer assumptions; various threshold models across the dose or exposure range; and models that investigate factors that might account for spatial heterogeneity." 85 Fed. Reg. at 15,406 (proposed 40 CFR § 30.6). "5 U.S.C. 301; 5 U.S.C. App.; Pub. L. 98 - 80, 84 Stat. 2086" in no way provides that authority. See supra at III.I of the commenters' comment letter. Nor do any of the organic environmental statutes that EPA cites in the Proposal or Supplemental Proposal.
Response: See response to comment 29-121.
Comment: Comment [187-1736]: Commenter (12720) writes that the Supplemental Proposal and original Proposal lack any statutory authority or other legal authority to adopt any regulation based on the proposal to "conduct independent peer review on all pivotal regulatory science used to justify significant regulatory decisions and on all pivotal science underlying influential scientific information, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review and the exemptions described therein," prior to or instead of considering that science without independent peer review. 85 Fed. Reg. at 15,406 (proposed 40 CFR § 30.7). "5 U.S.C. 301; 5 U.S.C. App.; Pub. L. 98 - 80, 84 Stat. 2086" in no way provides that authority. See supra at III.I of the commenters' comment letter. Nor do any of the organic environmental statutes that EPA cites in the Proposal or Supplemental Proposal.
Response: See response to comment 29-121.
Comment [187-1737]: Commenter (12720) writes that the Supplemental Proposal and original Proposal lack any statutory authority or other legal authority to adopt any of proposed part 30, including any proposal to allow the Administrator to "grant an exemption to this part on a case-by case basis if he or she determines that compliance is impracticable because technological barriers render sharing of the data or models infeasible, the development of the data or model was completed or updated before [EFFECTIVE DATE OF FINAL RULE] or making the data and models publicly available would conflict with laws governing privacy, confidentiality, CBI, or national and homeland security." 85 Fed. Reg. at 15,406 (proposed 40 CFR § 30.9). "5 U.S.C. 301; 5 U.S.C. App.; Pub. L. 98 - 80, 84 Stat. 2086" in no way provides that authority. See supra at III.I of the commenters' comment letter. Nor do any of the organic environmental statutes that EPA cites in the Proposal or Supplemental Proposal.
Response: See response to comment 29-121.
Comment [198-1907]: Commenter (3549) writes that as a baseline, whether working on "significant regulatory actions," "influential scientific information," or anything requiring the consideration of science, EPA must base its decisions on the best available science. Anything less is nonsensical, arbitrary, and in violation of the environmental laws EPA is charged with implementing. The SNPRM does not maintain this baseline, but instead provides for ways EPA would exclude and lessen consideration of the best available science. Therefore, the SNPRM, along with the proposed rule, must be withdrawn. See, e.g., 42 U.S.C. § 300g-1(b)(1)(B)(ii), (3)(A) (SDWA provisions requiring regulation of contaminants based on best available science); 15 U.S.C. § 2625(h) (TSCA provision requiring the use of best available science); see also 83 Fed. Reg. at 18,769 ("The best available science must serve as the foundation of EPA's regulatory actions.").
Response: See response to comment 29-121. The supplemental did not provide that EPA can exclude or lessen consideration of high-quality studies and this final rule provides that EPA will weigh public availability of underlying scientific data among other enumerated factors. 
Comment [198-1928]: Commenter (13549) the SNPRM changes the authority EPA claims to rely on for the proposal, however either way, the proposal remains unlawful. The text of the SNPRM proposes to rely solely on 5 U.S.C. § 301. 85 Fed. Reg. at 15404. The preamble says EPA is taking comment on whether to use 5 U.S.C. § 301, which the SNPRM calls EPA's housekeeping authority, independently or in conjunction with statutory provisions cited in the proposed rule and corrected in the SNPRM's preamble. 85 Fed. Reg. at 15398. The fact that EPA is still considering and asking what authority it can rely on for the proposed rule, twenty-two months after the initial proposed rule, shows that EPA does not have such authority. The SNPRM's statements on authority continues EPA's pattern of making rules first and searching for a justification later (which it still has not provided for this rulemaking). As the NTAA explained in its 2018 comments: In order to protect human health and environment, EPA must have a solid foundation before it proposes rules and rulemaking as such portrays the agency's naiveté in seeking answers for questions about its own authority.
First, none of the initial or corrected environmental statutes provide EPA authority for the proposed rulemaking. In most cases, EPA simply cites its general authority for rulemaking under the statutes. Still that general authority alone cannot provide a basis for the rule, especially when, as here, the rule would conflict with the requirements of each of the statutes. Courts have made clear that "EPA cannot rely on its gap-filling authority to supplement [a statute's] provisions when Congress has not left the agency a gap to fill." NRDC v. EPA, 749 F.3d 1055, 1064 (D.C. Cir. 2014); see also New York v. U.S. EPA, 413 F.3d 3, 40-42 (D.C. Cir. 2005). EPA otherwise cites to statutes describing research programs established by the agency, with no relation to or authorization for the requirements the SNPRM seeks to create. The Band also disagrees with EPA's reliance on 5 U.S.C. § 301 because 1) it is questionable whether § 301 applies to EPA as EPA is not one of the fifteen "Executive Departments" listed at 5 U.S.C. § 101, and 2) the proposed rule and SNPRM are not merely internal rules of agency procedure -- they would significantly impact every aspect of EPA's scientific and regulatory work, the individuals who contribute to that work, and the Tribes and communities who rely on it.
The Band appreciates the SNPRM's recognition that "in the event the procedures outlined in the proposed rulemaking conflict with the statutes that EPA administers, or their implementing regulations, the statutes and regulations will control." 85 Fed. Reg. at 15398. This statement alone should lead EPA to withdraw the proposed rule and SNPRM; the requirements conflict with every statute EPA administers and would cause EPA to unlawfully exclude valid scientific research from its work.
By way of example, in CAA Section 108(a)(2), Congress requires EPA to issue air quality criteria for air pollutants to "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air." 42 U.S.C. § 7408(a)(2). The SNPRM does not claim that studies that have not publicly released all their underlying data, such as those linking air pollution to mortality and disease, do not accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air. If EPA were to exclude such studies from consideration when issuing air quality criteria, it would be in violation of the CAA.
Response: See response to comment 29-121. EPA requested comment on the authorities it identified to solicit public input as part of the notice and comment exercise, not as a signal that the authorities identified were inapposite. Indeed, viewing requests for comment as concessions that the underlying rule is arbitrary and capricious would be contrary to the purpose of providing notice and comment. EPA also notes that EPA discusses the justification for this rule in the final rule preamble at Sections III.D and E.
Comment [200-2215]: Commenter (12412) writes that the SNPRM changes the authority EPA claims to rely on for the proposal, however either way, the proposal remains unlawful. The text of the SNPRM proposes to rely solely on 5 U.S.C. § 301. 85 Fed. Reg. at 15404. The preamble says EPA is taking comment on whether to use 5 U.S.C. § 301, which the SNPRM calls EPA's housekeeping authority, independently or in conjunction with statutory provisions cited in the proposed rule and corrected in the SNPRM's preamble. 85 Fed. Reg. at 15398. The fact that EPA is still considering and asking what authority it can rely on for the proposed rule, twenty-two months after the initial proposed rule, shows that EPA does not have such authority. The SNPRM's statements on authority continues EPA's pattern of making rules first and searching for a justification later (which it still has not provided for this rulemaking). As they explained in their 2018 comments: In order to protect human health and environment, EPA must have a solid foundation before it proposes rules and rulemaking as such portrays the agency's naiveté in seeking answers for questions about its own authority.
First, none of the initial or corrected environmental statutes provide EPA authority for the proposed rulemaking. In most cases, EPA simply cites its general authority for rulemaking under the statutes. Still that general authority alone cannot provide a basis for the rule, especially when, as here, the rule would conflict with the requirements of each of the statutes. Courts have made clear that "EPA cannot rely on its gap-filling authority to supplement [a statute's] provisions when Congress has not left the agency a gap to fill." NRDC v. EPA, 749 F.3d 1055, 1064 (D.C. Cir. 2014); see also New York v. U.S. EPA, 413 F.3d 3, 40-42 (D.C. Cir. 2005). EPA otherwise cites to statutes describing research programs established by the agency, with no relation to or authorization for the requirements the SNPRM seeks to create. Little Traverse Bay Bands of Odawa Indians (LTBB) disagrees with EPA's reliance on 5 U.S.C. § 301 because 1) it is questionable whether § 301 applies to EPA as EPA is not one of the fifteen "Executive Departments" listed at 5 U.S.C. § 101, and 2) the proposed rule and SNPRM are not merely internal rules of agency procedure -- they would significantly impact every aspect of EPA's scientific and regulatory work, the individuals who contribute to that work, and the Tribes and communities who rely on it.
LTBB appreciates the SNPRM's recognition that "in the event the procedures outlined in the proposed rulemaking conflict with the statutes that EPA administers, or their implementing regulations, the statutes and regulations will control." 85 Fed. Reg. at 15398. This statement alone should lead EPA to withdraw the proposed rule and SNPRM; the requirements conflict with every statute EPA administers and would cause EPA to unlawfully exclude valid scientific research from its work.
By way of example, in CAA Section 108(a)(2), Congress requires EPA to issue air quality criteria for air pollutants to "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air." 42 U.S.C. § 7408(a)(2). The SNPRM does not claim that studies that have not publicly released all their underlying data, such as those linking air pollution to mortality and disease, do not accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutants in the ambient air. If EPA were to exclude such studies from consideration when issuing air quality criteria, it would be in violation of the CAA.
Response: See response to comment 198-1928.
Comment [201-1983]: Commenter (12474) writes that the SNPRM changes the authority EPA claims to rely on for the proposal, however either way, the proposal remains unlawful. The text of the SNPRM proposes to rely solely on 5 U.S.C. § 301. 85 Fed. Reg. at 15404. The preamble says EPA is taking comment on whether to use 5 U.S.C. § 301, which the SNPRM calls EPA's housekeeping authority, independently or in conjunction with statutory provisions cited in the proposed rule and corrected in the SNPRM's preamble. 85 Fed. Reg. at 15398. The fact that EPA is still considering and asking what authority it can rely on for the proposed rule, twenty-two months after the initial proposed rule, shows that EPA does not have such authority. The SNPRM's statements on authority continues EPA's pattern of making rules first and searching for a justification later (which it still has not provided for this rulemaking). As they explained in their 2018 comments: In order to protect human health and environment, EPA must have a solid foundation before it proposes rules and rulemaking as such portrays the agency's naiveté in seeking answers for questions about its own authority.
First, none of the initial or corrected environmental statutes provide EPA authority for the proposed rulemaking. In most cases, EPA simply cites its general authority for rulemaking under the statutes. Still that general authority alone cannot provide a basis for the rule, especially when, as here, the rule would conflict with the requirements of each of the statutes. Courts have made clear that "EPA cannot rely on its gap-filling authority to supplement [a statute's] provisions when Congress has not left the agency a gap to fill." NRDC v. EPA, 749 F.3d 1055, 1064 (D.C. Cir. 2014); see also New York v. U.S. EPA, 413 F.3d 3, 40-42 (D.C. Cir. 2005). EPA otherwise cites to statutes describing research programs established by the agency, with no relation to or authorization for the requirements the SNPRM seeks to create.
The NTAA also disagrees with EPA's reliance on 5 U.S.C. § 301 as:
1) It is questionable whether § 301 applies to EPA as EPA is not one of the fifteen "Executive Departments" listed at 5 U.S.C. § 101, and
2) The proposed rule and SNPRM are not merely internal rules of agency procedure --  they would significantly impact every aspect of EPA's scientific and regulatory work, the individuals who contribute to that work, and the Tribes and communities who rely on it.
The NTAA appreciates the SNPRM's recognition that "in the event the procedures outlined in the proposed rulemaking conflict with the statutes that EPA administers, or their implementing regulations, the statutes and regulations will control." 85 Fed. Reg. at 15398. This statement alone should lead EPA to withdraw the proposed rule and SNPRM; the requirements conflict with every statute EPA administers and would cause EPA to unlawfully exclude valid scientific research from its work.
By way of example, in CAA Section 108(a)(2), Congress requires EPA to issue air quality criteria for air pollutants to "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air." 42 U.S.C. § 7408(a)(2). The SNPRM does not claim that studies that have not publicly released all their underlying data, such as those linking air pollution to mortality and disease, do not accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air. If EPA were to exclude such studies from consideration when issuing air quality criteria, it would be in violation of the CAA.
Response: EPA is not relying on 5 U.S.C. § 301 for its authority. See response to comment 29-121. See also responses to comments 84-1403 and 198-1928. 
Comment [205-2037]: Commenter (13788) writes that the Federal Housekeeping Statute cannot be used to authorize rules that limit the scope of EPA's responsibilities under other statutes. The Chrysler Court was clear that "[section] 301 does not authorize regulations limiting the scope of [the Trade Secrets Act]" (the statute under consideration in that case), or of any other substantive statutory authority. Similarly, the Federal Housekeeping Statute cannot authorize regulations limiting the scope of environmental statutes. If finalized, this rule would limit EPA's ability to evaluate and use the best available science to support its regulatory decisions, thereby undermining its ability to fully implement its responsibilities as mandated under environmental statutes. It therefore cannot be a valid exercise of the authority granted by the Federal Housekeeping Statute.
Neither the Federal Housekeeping Statute nor the IQA provide authority or justification for a rule that would significantly impede and disrupt EPA's ability to fulfill its statutory mandates. For example, the air quality criteria EPA is required to issue when reviewing the NAAQS under the CAA takes the form of an integrated science assessment (ISA), in which the Agency evaluates the latest science on public health and environmental effects of criteria pollutants. The ISA document is considered a highly influential scientific assessment (HISA), which is a subset of the ISI governed by the IQA guidance. Subjecting the ISA to the requirements of this rule and SNPRM would result in the exclusion of important, longstanding public health studies from consideration, because the human health data on which they are based, and which makes them of value, is not publicly available as contemplated by the proposed rule. Such a result would directly conflict with the CAA's mandate that the air quality criteria reflect the latest scientific knowledge.
The Agency claims that "[i]n the event the procedures outlined in this part conflict with statutes that EPA administers, or their implementing regulations, the statutes and regulations will control." EPA fails, however, to explain how the Agency will define and identify such conflicts and ensure that staff do not follow the proposed rule. This is a significant question that EPA entirely fails to discuss.
Response: See response to comment 29-121. As discussed in the final rule preamble at Section I.C because this is purely a procedural rule, EPA will discuss the statutes that EPA administers in forthcoming substantive rulemakings under those statutes.
Comment [205-2038]: Commenter (13788) writes that tot one of the other environmental statutes cited as potential authority by EPA actually authorizes this rule. At most, many of the provisions cited by EPA generally allow the agency to promulgate regulations in order to implement various statutes, with many requiring that the regulations be necessary. However, EPA has not shown how this rule would further the implementation of any statute, much less how the rule is necessary to fulfill any statutory responsibilities, and the fact that the Agency has not considered such a rule necessary until now suggests the opposite. Under the CAA, the most important scientific studies and models for use in evaluating the public health effects of air pollution are those that are based on actual human health data -- much of which cannot be publicly available due to the confidentiality requirements imposed by other federal statutes and the agreements under which the data is provided to researchers. For CAA rulemaking then, any of the approaches EPA is proposing actively undermine the Agency's use of the best science in making decisions that promote and protect the public health and welfare -- the fundamental purpose of the CAA.
If Congress had intended EPA to prioritize data transparency over the quality of scientific information in its regulatory decision-making, it could have written that requirement into the statutes. In the absence of such a requirement and as required by law, EPA should prioritize quality in accordance with the various "mandates requiring that its decisions rest on various formulations of the best available science."
Response: See response to comment 29-121.
1.2 Housekeeping Statue
Comment [9-34]: Commenter (10020) writes that the Proposed Notice of Supplemental Rulemaking is new rulemaking, not a revision of any prior proposal. 
As such, it simply does not fall within your agency "housekeeping authority." Moreover, your proposed new rulemaking does not meet the requirements of the APA. Your Proposed Notice of Supplemental Rulemaking recites on its face that you want to add a definition of "publicly available." This in itself is new rulemaking, not a revision of any prior proposal. As such, it simply does not fall within your agency "housekeeping authority" and is not only unauthorized but is invalid at least to that extent and for that reason. Moreover, your proposed new rulemaking is simply not authorized or valid under the requirements of the APA. In short, your proposed new definition of "publicly available" is simply unauthorized as a part of the housekeeping authority you invoke in your subject Notice of Proposed Supplementary Rulemaking. When treated as newly proposed rulemaking, your proposal is of course subject to the requirements of the APA. It does not meet them. It must be withdrawn or rejected for that reason alone. 
Response: This purely internal procedural final rule is not subject to APA notice and comment requirements, including the requirement to publish a notice of proposed rulemaking and request comment on that proposed action. See 5 USC 553(b)(A). Nevertheless, EPA elected to initiate notice and comment on both the procedural rule NPRM and SNPRM. As its title implies, the SNPRM supplements the NPRM by making the changes and clarifications outlined in the SNPRM preamble at Section I.C. It is not a new rulemaking. Regardless the additional round of notice and comment provided on the SNPRM is equivalent to that which would have been available for an NPRM initiating a new rulemaking. 
Comment [13-51]: Commenter (10944) strongly opposes the 2020 supplemental to the proposed rule, Strengthening Transparency in Regulatory Science. EPA housekeeping authority should not be used to unravel existing regulations that have been instrumental in promoting public health in the US. 
The supplemental rule contains new definitions and changes that are dangerous to public health, adding to the already harmful original rule. EPA housekeeping authority should not be used to limit environmental science. I urge you to reconsider the changes proposed by the supplemental rule in the interest of all Americans. 
Response: The rule will apply to future rulemakings. EPA intends to continue relying on the high-quality studies to support its significant regulatory decisions and ISI. EPA has explained that this procedural rule has limitations on its scope in terms of conflicts with other substantive requirements and that, in situations where this rule applies, EPA will give greater consideration to studies where the underlying data and models are available for independent validation. 
Comment [16-182]: Commenter (10048) notes that the SNPRM seeks comment on whether to use "housekeeping" authority for the proposed rule. The proposed rule is substantive and beyond the scope of housekeeping authority. They note, the Supreme Court held, "Section 301 is a "housekeeping statute," authorizing rules of agency organization, procedure, or practice as opposed to "substantive rules." There is nothing in the legislative history to indicate that 301 is a substantive grant of legislative power to promulgate rules authorizing the release of trade secrets or confidential business information." Chrysler Corp. v. Brown, 441 U.S. 281 (1979).
Response: The final rule is a rule of agency procedure and practice. The rule applies only to EPA and does not obligate or affect the rights of any outside parties. EPA does not intend to release trade secrets or CBI when implementing the rule. The rule governs only how EPA will consider studies used to develop significant regulatory decisions and ISI.
Comment [20-75]: Commenter (10753) writes that the EPA misleadingly suggests that the Trio is a minor housekeeping matter. Nothing can be further from the truth. Indeed, even if analyzed separately, each of the three elements of the Trio is far more than a housekeeping matter. Each element is in effect a dramatic procedural change that has dire substantive impacts. Obviously, none of the elements of the Trio can be adopted as a housekeeping matter. Cf. The EPA's `Censored Science' Rule Isn't Just Bad Policy, It's Also Illegal, J. Goodwin, 11-22-2019 https://sensiblesafeguards.org/the-epas-censored-science-rule-isnt-just-bad-policy-its-also-illegal/ (attached).
By restricting sensible scientific inquiry, the EPA effectively is proposing to ignore any and all evidence that undercuts its proposals. While the editors' numerous prior comments made this clear, an example may be helpful. Dr. Frank Kline, formerly professor of the medical schools of both USC and UC Irvine, explained how the Rule will frustrate good science. See Dr. Kline's statement included at https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-8817. 
As an integral part of the editors' objections to each element of the Trio, they incorporate by reference every objection stated in the following comment briefs:
https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-0561 https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-1037 https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-3132 https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-3133 https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-4879 https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-5164 https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-5183 https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-5184 https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-5421 https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-5995 https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-6107 https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-6113 https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-6801 https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-8279 https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-8178 https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-8817 (Collectively, these 16 formal comment briefs are referred to as "the Comments").
Response: EPA will not categorically exclude any studies from consideration when promulgating significant regulatory decisions or developing ISI. EPA has explained that this procedural rule has limitations on its scope in terms of conflicts with other substantive requirements and that, in situations where this rule applies, EPA will give greater consideration to those studies where the underlying data and models are available for independent validation. That means that the agency will consider all relevant studies when taking covered actions. 
Comment [22-77]: Commenter (10975) writes as to "statutory authority"  -  the disingenuously-entitled "Transparency" rule is not appropriately issued under the authority of the internal agency "house-keeping" statute. Housekeeping rules issued without full rulemaking cannot have legal effect upon third parties outside the government. But the effect of this proposed rule would be to prevent scientific researchers outside of the government from meaningfully submitting their highly relevant work essential in federal initiatives to control disease and other public health necessities. In some cases, this provision will frustrate the business and contractual undertakings of non-profit scientific entities providing services to federal agencies. The preclusion, for irrational, harmful partisan purposes, of scientists' effective submission of scientific data, moreover, may violate their First Amendment right to petition the government, by pre-emptively blocking the effectiveness of such submissions without regard to their substance and merit.
Response: Nothing in this rule would prevent scientists from submitting their data and studies to EPA. Scientists can also submit their studies for consideration during the public comment period for a significant regulatory decision or during peer review of ISI. Further, the rule does not address or affect business or contractual undertakings of any outside entities. The final rule addresses how EPA will consider the relevant scientific studies when promulgating significant regulatory decisions or developing ISI subject to the rule. EPA has explained that this procedural rule has limitations on its scope in terms of conflicts with other substantive requirements and that, in situations where this rule applies, EPA will give greater consideration to those studies where the underlying data and models are available for independent validation. 
Comment [25-94]: Commenter (10972) writes that the APA requires that the agency in its notice refer to the legal authority under which a rule is proposed. Section 553(b)(2). This requirement cannot be hedged away by asking for comment on the issue. This SNPRM suggests that it is under the EPA's Housekeeping authority. These rules cannot fit under the housekeeping authority. They are not merely procedural as they encode a substantive value judgment and have great substantive effects on the rights and interests of parties. In the circumstances, the EPA's legal authority ought to have been identified in the applicable substantive provisions. Not only does the EPA implicitly acknowledge the great substantive effects of these proposed rules, but it also explicitly notes substantive provisions that it believes could be applicable. The provisions providing legal authority must be correctly identified as such in order to meet the APA's legal requirements.
Identifying the correct legal authority is also important because provisions for issuance of substantive regulations often carry with them requirements unique to them. Notwithstanding the desire to discuss authority under various substantive statutes, the EPA has clearly not complied with the unique rulemaking requirements of some of these statutes. Whatever the Housekeeping authority may entail, it cannot provide a means to avoid the procedural requirements of the legal provisions authorizing a proposed rule.
Seeking to operate on the housekeeping authority is particularly inappropriate in this case where genuine scientific transparency makes different substantive demands under different laws and in different sectors. The SNPRM indicates that the EPA is aware of conflict between some of the statutes it administers and the proposed rule. A claim to be acting under those statutes cannot be sustained in such circumstances. Nothing is achieved by asserting that the statutes control if there is conflict between such statutes or their regulations and the proposed regulations; that regulations cannot be contrary to applicable statutes is trite law, but it can never empower actively passing rules contrary to the statute allegedly empowering them. The Housekeeping authority cannot allow such an effect. Regulations ought to elucidate statutes and facilitate the achievement of statutory goals, not to obfuscate statutory goals.
Response: The commenter claims that the rule is not procedural because it "encodes a substantive value judgment." The D.C. Circuit held that a rule is required to undergo full notice and comment proceedings if it "also encodes a substantive value judgment or puts a stamp of approval or disapproval on a given type of behavior." American Hosp. Ass'n v. Bowen, 834 F.2d 1037, 1047 (D.C. Cir. 1987); see also Air Transport Ass'n of America v. DOT, 900 F.2d 369, 376 (D.C.Cir.1990). Even if this applies, EPA satisfied any such requirement because the agency promulgated the rule after undergoing notice and comment. Additionally, the final rule does not categorically exclude any relevant studies from consideration. EPA will evaluate and weigh studies determined to be pivotal regulatory science, using the criteria in § 30.5, which will allow all high-quality studies to be factored into the agency's development of significant regulatory decisions and ISI.
EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, EPA does not rely on, interpret, or apply provisions of a particular statute or statutes that it administers. EPA will undertake such efforts in forthcoming substantive actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. 
EPA identified potential sources of legal authority in both the NPRM and SNPRM (Refs 5-7). Therefore, EPA did in fact meet any obligation under the APA to cite a source of authority. Furthermore, this is a purely procedural rule for reasons already discussed and therefore not subject to the APA's notice and comment requirements; EPA elected to nevertheless request public comment on various aspects of the rule, including whether the sources of authority already identified were appropriate. This was not improper under the APA.
Comment [31-135]: Commenter (9493) writes that even in the absence of an unprecedented pandemic, the proposed rule would not be one that could be adequately addressed in only thirty days. This fact is evident in the reach and complexity of the EPA's significantly revised proposal:
The EPA's new proposal appears to rest on a new theory for why the agency is even authorized to promulgate such a rule. According to the notice, the EPA is now attempting to rely on a "housekeeping authority" rooted in the Federal Housekeeping Statute, Reorganization Plan No. 3, or both, maybe. While the EPA mentioned the housekeeping statute when it extended the first comment period, it now seems to be relying on a more amorphous "housekeeping authority." (83 FR 15397-98). It is also unclear whether the EPA is relying on this asserted authority exclusively, or whether it intends to include a host of environmental statutes as additional authority for the rule. As the EPA has acknowledged in its notice, reliance on this new theory of authority calls for additional consideration and comment.
Response: EPA is authorized to promulgate this procedural rule under its authority to promulgate housekeeping regulations governing its internal affairs (hereinafter, "housekeeping authority"). EPA is relying on this housekeeping authority and not the environmental statutes as authority for the final rule. Because EPA proposed this authority in the supplemental notice, no additional notice and comment is necessary. 
The Federal Housekeeping Statute provides that "[t]he head of an Executive department or military department may prescribe regulations for the government of his department, the conduct of its employees, the distribution and performance of its business, and the custody, use, and preservation of its records, papers, and property." 5 U.S.C. 301. As the Supreme Court discussed in Chrysler Corp. v. Brown, the intended purpose of section 301 was to grant early Executive departments the authority "to govern internal departmental affairs." As the Supreme Court further notes, section 301 authorizes "what the [APA] terms `rules of agency organization, procedure or practice' as opposed to substantive rules." 
While EPA is not one of the 15 "Executive departments" listed at 5 U.S.C. 101, EPA gained housekeeping authority equivalent to that granted to Executive departments in section 301 through the Reorganization Plan No. 3 of 1970, 84 Stat. 2086 (July 9, 1970), which created EPA. The Reorganization Plan established the Administrator as "head of the agency," transferred functions and authorities of various agencies and Executive departments to EPA and gave EPA the authority to promulgate regulations to carry out the transferred functions.
Section 2(a)(1)-(8) of the Reorganization Plan transferred to EPA functions previously vested in several agencies and Executive departments including the Departments of the Interior and Agriculture. Section 2(a)(9) also transferred so much of the functions of the transferor officers and agencies "as is incidental to or necessary for the performance by or under the Administrator of the functions transferred" and provided that "[t]he transfers to the Administrator made by this section shall be deemed to include the transfer of [] authority, provided by law, to prescribe regulations relating primarily to the transferred functions." The Federal Housekeeping Statute was existing law at the time the Reorganization Plan was enacted.
The Office of Legal Counsel (OLC) has opined that the Reorganization Plan "convey[s] to the [EPA] Administrator all of the housekeeping authority available to other department heads under section 301" and demonstrates that "Congress has vested the Administrator with the authority to run EPA, to exercise its functions, and to issue regulations incidental to the performance of those functions." 
Courts have considered EPA to be an agency with federal housekeeping authority. The U.S. Court of Appeals for the Second Circuit, in EPA v. General Elec. Co., 197 F.3d 592, 595 (2d Cir. 1999), found that "the Federal Housekeeping Statute, 5 U.S.C. 301, authorizes government agencies such as the EPA to adopt regulations regarding `the custody, use, and preservation of [agency] records, papers, and property.'" The Fourth Circuit Court of Appeals, in Boron Oil Co. v. Downie, 873 F.2d 67, 69 (4th Cir. 1989), held that the district court exceeded its jurisdiction where it compelled testimony contrary to duly promulgated EPA regulations which EPA argued were authorized by section 301. The Second and Fourth Circuits did not directly address whether it was an "Executive department", but rather assumed that EPA has authority to issue housekeeping regulations under section 301. Indeed, if EPA did not possess such authority, whether under section 301 or otherwise, EPA would not be able to carry out its daily functions which would in turn preclude EPA from exercising its duties as a federal regulatory agency. The same would hold true for other regulatory agencies that are not listed as an Executive department under section 101.
Even if the Federal Housekeeping statute does not directly apply to EPA, the agency's Reorganization Plan provides the authority to promulgate housekeeping regulations. As indicated by the case law and the OLC Opinion, it has long been presumed that EPA has been granted the authority to promulgate housekeeping regulations, whether pursuant to section 301 or equivalent housekeeping authority. 
The Reorganization Plan does not expressly state that the authority to promulgate regulations is limited to the transferred functions, but rather that it relates primarily to the transferred functions. To argue that EPA's housekeeping authority is limited to carrying out only those functions explicitly transferred in the Reorganization Plan is to accept the statutory maxim that "the explicit mention of one thing is the exclusion of another." Courts routinely reject this principle because it is based on the faulty premise that the drafters of a statute considered and rejected all possible alternative or supplemental provisions. See, e.g., National Petroleum Refiners Ass'n v. FTC, 482 F.2d 672, 676 (D.C. Cir. 1973) (noting that the maxim is "increasingly considered unreliable"); see also Am. Trucking Ass'ns v. United States, 344 U.S. 298, 309-310 (1953) (rejecting the same principle when evaluating plaintiffs claim that the Interstate Commerce Commission (ICC) did not have the authority to power, control, regulate, or affect leasing practices because that authority was not explicitly named in the ICC's authorization statute). The court in American Trucking rejected the same principle when evaluating plaintiffs claim that the ICC did not have the authority to power, control, regulate, or affect leasing practices because that authority was not explicitly named in the ICC's authorization statute.
Comment [141-1273]: Commenter (11911) writes that EPA is not authorized to promulgate the Strengthening Transparency in Regulating Science (STRS) Rule under its housekeeping authority, as the finalized Rule itself would substantively violate a number of environmental statutes. Further, EPA neglects its mandate under Executive Order 12,866 by failing to account for the social and environmental harms STRS Rule's finalization would generate. 
EPA is not authorized to promulgate the STRS Rule under its housekeeping authority. The Housekeeping Act provides that "[t]he head of an Executive department... may prescribe regulations for the government of his department, the conduct of its employees, the distribution and performance of its business, and the custody use, and property." 5 U.S.C. 301. As the Supreme Court has made clear, the purpose of section 301 is to grant Executive departments "authority to govern internal department affairs.... as opposed to substantive rules." 
Courts have repeatedly rejected agency attempts, including recent ones, "to twist [the Housekeeping Act, a] simple administrative statute[,] into an authorization for the promulgation of substantive rules." 
Response: See response to Comment 31-135.
Comment [32-144]: Commenter (11182) notes that EPA is also taking comments on whether to use its housekeeping authority independently as authority or in conjunction with the environmental statutory provisions cited as authority in the 2018 proposed rulemaking (as clarified in the SNPRM). EPA claims it is not seeking to interpret its own statutory authority and that the changes proposed in the SNPRM are authorized under its housekeeping authority. They strongly disagree. The proposed rule creates arbitrary criteria for exclusion of studies considered pivotal science on the basis of use of confidential medical data. The SNPRM significantly increases the number of studies potentially excluded from consideration or subject to down-weighting by the proposed rule, thus far exceeding simple matters of administrative discretion. That the rule is intended to apply to studies in which the potential financial impacts are anticipated to exceed $500,000,000.00 further demonstrates a level of complexity in excess of that intended to be managed through administrative housekeeping alone.
For these reasons the commenter strongly opposes both the use of the housekeeping provisions to make the changes envisioned in the SNPRM and the proposed ability of the administrator to exempt or not exempt scientific research for consideration by EPA, no matter what aspects of the research are used in the decision-making.
Response: Implementation of the final rule will not result in the categorical exclusion of any relevant studies from consideration. EPA will evaluate studies determined to be pivotal science, using the criteria in § 30.5, which will allow all high-quality studies to be factored into the agency's development of significant regulatory decisions and ISI. The criteria are based on EPA's Framework for Human Health Risk Assessment to Inform Decision Making (Ref. 69). EPA has explained that this procedural rule has limitations on its scope in terms of conflicts with other substantive requirements and that, in situations where this rule applies, EPA will give greater consideration to those studies where the underlying data and models are available for independent validation. Studies identified as pivotal science where the underlying data and models are available for independent validation will be given greater consideration than pivotal science lacking data and model transparency. The EPA will document why a particular study or studies received lesser consideration. An Administrator's exemption can be requested so that pivotal science whose underlying data and models is not, or cannot be, made publicly available can be given greater consideration.
Comment [34-155]: Commenter (11192) writes that the EPA claims to have authority to make extensive changes to how the agency uses science in its decision-making through the Federal Housekeeping Statute, and dwells on this point in the SNPRM for several paragraphs. Remarkably, the EPA suggests that this rule, which ignores established processes for evaluating scientific merit and would impact every regulation under the EPA's authority, is not a "substantive rule." The EPA asserts that the proposed rule "exclusively pertains to the internal practices of the EPA" and fails to mention that the EPA relies extensively on studies conducted by other entities, and indeed, is mandated to consider outside research under both the CAA and CWA. The EPA does not address how the application of the proposed rule could impact its ability to faithfully execute its statutory responsibilities under these acts; it simply makes reference to its intention to be consistent with its other statutes. As the EPA repeatedly asserts that this proposed rule deals with internal EPA processes, the EPA invites public comment about whether to pursue this proposed rule under only its housekeeping authority, or under its housekeeping authority along with its other statutory authorities, and conspicuously does not invite comment about whether or not invocation of the housekeeping authority is appropriate at all. Let it be clear: a rule of this breadth, with impacts extending into all significant agency decisions and actions, should by no means be considered a basic agency procedure and housekeeping.
Response: The final rule describes how EPA will consider studies underlying significant regulatory actions and ISI based on data and model availability and study quality factors. Those factors are found in the agency guidance document entitled Framework for Human Health Risk Assessment to Inform Decision Making. (Ref. 69) and are already used by EPA to evaluate outside research.
Comment [38-185]: Commenter (10751) writes that as they explained their prior comments this week, the EPA has expanded its proposal to stifle good science by publishing a SNPRM. In general, the SNPRM expands the Rule so as to further hamstring good scientific inquiry. For example, the SNPRM improperly purports to hide both scientific data and models as a minor agency housekeeping matter. Obviously, the Rule, the SNPRM, and the Rule to the extent modified by the SNPRM (collectively referred to as the Trio) is far more than an internal agency housekeeping matter.
Response: Nothing in the final rule will lead to the suppression of scientific data and models. EPA has explained that this procedural rule has limitations on its scope in terms of conflicts with other substantive requirements and that, in situations where this rule applies, EPA will give greater consideration to those studies where the underlying data and models are available for independent validation. Additionally, the Administrator's exemption in § 30.9 can be used to allow greater consideration for pivotal science where the underlying data and models are not publicly available. 
Comment [40-193]: Commenter (10751) notes that in the SNPRM, EPA states:
      Seventh, the EPA is proposing the option of using its housekeeping authority independently as authority for taking this action or in conjunction with the environmental statutory provisions cited as authority in the 2018 proposed rulemaking (as clarified in this supplemental proposal). The Agency continues to consider whether it is appropriate to rely on its authority in the above-referenced environmental statutory provisions (potentially in conjunction with its housekeeping authority). The Agency will consider comments on this issue submitted in response to the 2018 proposal and in response to this SNPRM. Section 301 authority as transferred to EPA in Reorganization Plan No. 3 of 1970 provides appropriate authority for EPA to promulgate regulations that govern internal agency procedures. This action establishes internal agency procedures governing how EPA employees will handle studies when the data and models underlying science that is pivotal to EPA's significant regulatory decisions and/or influential scientific information are or are not publicly available.
Unfortunately, neither time nor the space permitted in public comments like this one permit a full discussion of the legal errors in the SNPRM much less in the Trio. However, it is self-evident that the Trio is both legally invalid and improper. 
Congress expressly provided that in the housekeeping provisions of 5 U.S. Code § 301. Departmental regulations.
      The head of an Executive department or military department may prescribe regulations for the government of his department, the conduct of its employees, the distribution and performance of its business, and the custody, use, and preservation of its records, papers, and property. This section does not authorize withholding information from the public or limiting the availability of records to the public." (Pub. L. 89 - 554, Sept. 6, 1966, 80 Stat. 379.) https://www.law.cornell.edu/uscode/text/5/301
While careful analysis of the section's every clause would further demonstrate that the Trio is void ab initio, a plain reading the last sentence suffices. It states: "This section does not authorize withholding information from the public or limiting the availability of records to the public." In the present case, the Trio try to limit the availability of good scientific information of the public. Cf., e.g., Dr. Kline's statement included at https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-8817.
This result directly conflicts with the express directions of Congress.
Nor is consideration of the cannons of statutory construction necessary to conclude that each element of the Trio is invalid. As the editors explained in their prior comment briefs, both NEPA and the relevant environmental statutes require that the EPA disclose good scientific information to the public. This policy of fulsome disclosure is antithetical to the Trio.
      https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-0561
https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-1037
https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-3132
https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-3133
https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-4879
https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-5164
https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-5183
https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-5184
https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-5421
https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-5995
https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-6107
https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-6113
https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-6801
https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-8279
https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-8178
https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-8817
(Collectively, these 16 formal comment briefs are referred to as "the Comments").
Of course, detailed consideration of the canons of statutory construction also compel the conclusion that the Trio are invalid.
For example, the plain meaning of section 301 is that: "This section does not authorize withholding information from the public or limiting the availability of records to the public."
As another example, the clear legislative intent of the underlying environmental statutes is to protect both human health and the environment; the Trio, by contrast, seeks to squelch the truth and bury valid science.
Even if one were to make the absurd assumption that Congress enacted the Trio into law, it is clear that any reviewing court would necessarily conclude that it must invalidate the Trio to avoid the absurd (an unconstitutionally irrational) result of ignoring good science.
Response: The purpose of the rule is to increase transparency of the science EPA uses to support its significant regulatory decisions and ISI. Nothing in the final rule will lead to the suppression of scientific data and models or otherwise withhold information from the public or limit the availability of records to the public. As described in § 30.5, EPA will identify those studies that are given lesser consideration and provide a short description of why lesser consideration was given. The public can provide comments on that issue and can also raise additional studies that it believes the agency should have considered during the comment period on a proposed rule. Additionally, the Administrator's exemption in § 30.9 can be used to allow greater consideration for pivotal science where the underlying data and models are not publicly available. 
Comment [48-375]: Commenter (12727) writes that EPA's attempt to rely on the Housekeeping Act as a source of authority directly contradicts the statute's explicit allocation of these housekeeping authorities to "Executive departments" -- which EPA is not. The statute provides that "the head of an Executive department or military department may prescribe regulations for the government of his department, the conduct of its employees, the distribution and performance of its business, and the custody, use, and preservation of its records, papers, and property." By its terms, this statute authorizes an executive department "to regulate its own affairs" through "rules of agency organization procedure or practice."
EPA is not an "Executive department" within the meaning of section 301. As EPA acknowledges in the Supplemental Notice, section 101 of Title 5 explicitly lists the fifteen entities considered "Executive departments" under section 301, and EPA is not among them. The statute instead designates EPA as an "independent establishment," under section 104. Section 104 defines an "independent establishment" as "an establishment in the executive branch . . . which is not an Executive department, military department, Government corporation, or part thereof, or part of an independent establishment." Since EPA is an "independent establishment" rather than an "Executive department," the agency cannot rely on section 301 as the source of its authority for the 2018 Proposal.
Response: See response to Comment 31-135. 
Comment [48-376]: Commenter (12727) writes that EPA incorrectly asserts that EPA has gained the authorities of an "Executive department" through delegation and reorganization. In the Supplemental Notice, EPA claims that Reorganization Plan No. 3, which established EPA -- and contains no mention of the Housekeeping Act -- implicitly granted EPA this status when it transferred programs and authorities from the Departments of Interior and Health, Education, and Welfare to the newly formed EPA. Admitting that nothing in Reorganization Plan No. 3 explicitly grants this authority, EPA points to a 2008 OLC opinion, on the entirely unrelated topic of whether federal employees should be held responsible for lost or damaged federal property, which contends that section 2(a)(9) of the plan implicitly confers this authority alongside the explicit transfer of various functions.
Such a strained interpretation clearly contradicts the requirement for Congress to amend the statute, if it so wills, to recognize additional executive departments. Congress has demonstrated its capacity to grant newly created government entities the status of executive departments with housekeeping authorities when it so chooses. In 2002, Congress created the Department of Homeland Security (DHS) and, similar to EPA's Reorganization Plan No. 3, laid out the functions transferred from existing departments to the new department. In section 101 of the Homeland Security Act (HSA), Congress specifically provided that the DHS was being "established . . . as an executive department of the United States within the meaning of title 5." Congress has had decades to take equivalent action with regard to EPA, and Congress's choice to not update the "Executive department" list cannot be conveniently ignored. To this day, EPA remains an "independent establishment" under the Housekeeping Act. It defies any reasonable interpretation for EPA to be simultaneously designated as an "executive department" and an "independent establishment," when the latter is defined as an entity not listed among the former. To construe EPA as an "executive department" contradicts the plain language of the statute and congressional intent.
Response: The term "independent establishment" is not used in the Housekeeping Statute. Also see the response to Comment 31-135. 
Comment [48-377]: Commenter (12727) notes that the first case EPA cites for support, EPA v. General Electric Co., concerns the judicial reviewability of EPA's refusal to comply with a third-party subpoena under the APA. The court found the refusal reviewable; it did not find that EPA's regulations, which claimed section 301 among other authorities, applied to the subpoena as issued. The court referenced EPA's potential housekeeping authority only in dicta without any further discussion or analysis.
Response: See response to Comment 31-135. 
Comment [48-378]: Commenter (12727) notes that in the second case EPA references for support, Boron Oil Co. v. Downie, the court held that the lower courts lacked jurisdiction to compel an EPA employee to appear and testify in a state court action to which the government is not a party as EPA was protected by sovereign immunity. While finding the grant of sovereign immunity sufficient on its own merit, the court explained that the principle of federal supremacy supports the protection of sovereign immunity such that the state court could not compel testimony in violation of EPA's housekeeping regulations. But the court provided no explication of how EPA could be construed as an "Executive department" under the Housekeeping Act with relevant authority to issue such regulations. Moreover, the EPA regulations there in question claim an assortment of authorities, not only the Housekeeping Act.
Response: See response to Comment 31-135. 
Comment [48-379]: Commenter (12727) notes that the Supplemental Notice also broadly cites Chrysler Corp. v. Brown. But that case provides no additional basis to apply section 301 authority to an "independent establishment," as the case concerns the DOL -- a listed executive department under the Housekeeping Act. It also holds that the Housekeeping Act does not authorize issuance of substantive rules, as discussed infra.
Response: See response to Comment 31-135. The final rule is a rule of agency procedure and practice. The rule applies only to EPA and does not obligate or affect the rights of any outside parties. As the Supreme Court stated in Chrysler Corp., whether a rule affects individual rights and obligations is an "important touchstone" for distinguishing substantive rules from other types of rules. This final rule does not regulate the rights and obligations of any party outside of the EPA let alone have legal force and effect on them. Therefore, any incidental impacts on voluntary behavior outside of the EPA do not render it a substantive rule. Nothing in this rule would prevent scientists from submitting their data and studies to EPA. Researchers may voluntarily take steps to ensure that their studies are given greater consideration by making their data and models publicly available for independent validation.
Comment [48-380]: Commenter (12727) notes that in the primary case EPA cites for its alleged authority under the Housekeeping Act to issue the proposal, Chrysler Corp. v. Brown, the Supreme Court makes explicit that the Housekeeping Act cannot authorize substantive rules such as the current proposal. Chrysler Corp. concerned a government contractor who asked the court to enjoin release of documents regarding petitioner's employment practices. The documents had been requested under the Freedom of Information Act (FOIA), and a division of the DOL had determined they should be released, consistent with FOIA and disclosure regulations from the DOL's Office of Federal Contract Compliance Programs (OFCCP). The court concluded that section 301 did not authorize the OFCCP regulations, explaining that section 301 is a "housekeeping statute," authorizing only rules of agency organization, procedure, or practice as opposed to "substantive rules." It found "nothing in the legislative history of § 301 to indicate it is a substantive grant of legislative power to promulgate rules authorizing the release of trade secrets or confidential business information." The case offers no support for a more expansive reading of what constitutes non-substantive rules of agency organization, procedure, or practice for even an executive department -- which, as explained above, EPA is not.
Response: The final rule is a procedural rule of agency procedure and practice. It does not establish requirements for outside parties, nor does it affect the rights, interests, and obligations of outside parties. Even assuming there could be downstream practical effects on the voluntary behavior of outside parties and on outside parties' interactions with EPA, such impacts do not render this procedural rule substantive. See American Hosp. Ass'n v. Bowen, 834 F.2d 1037, 1051 (D.C. Cir. 1987) ("[A]gency rules that impose `derivative,' `incidental,' or `mechanical' burdens upon regulated individuals are considered procedural, rather than substantive.")
Comment [48-381]: Commenter (12727) writes that Congress took explicit action in 1958 to amend section 301 to "correct" agencies from abusing the Housekeeping Act by attempting to use its authority as a substantive basis to withhold information from the public. The Supreme Court noted in Chrysler Corp. "that Congress had looked carefully at the statute in 1958; that the Special Subcommittee on Government Information had `unanimously agreed that [§ 301] originally was adopted in 1789 to provide for the day-to-day office housekeeping in the Government departments,' and that attempts to construe it as something more was `misuse' which `twisted' the statute." The courts have repeatedly checked subsequent agency attempts to "twist" the statute to a range of substantive purposes and found those efforts illegal, including halting government's extralegal efforts to limit disclosure and inclusion of information in the scientific process. The Supplemental Notice now attempts to similarly and illegally twist the statute to withhold science from consideration in rulemakings that broadly affect the public interest.
Response: The purpose of the rule is to increase transparency of the science EPA uses to support its significant regulatory decisions and ISI. Nothing in the final rule will lead to the suppression of scientific data and models. 
Comment [48-382]: Commenter (12727) writes that the proposal is a substantive rule and hence not authorized by the Housekeeping Act.
The 2018 proposal and Supplemental Notice meet the court-determined criteria for a substantive regulation, refuting EPA's claim that it is proposing merely an "internal rule of agency procedure." A "substantive rule" is not defined in the APA, but in distinguishing between "substantive rules" and "interpretive rules, general statements of policy, or rules of agency organization, procedure, or practice," courts have described "substantive rules" as those "affecting individual rights and obligations," a quality which helps identify which rules are "binding" or "have the force of law." By contrast, courts have held that regulations issued pursuant to the authority under the Housekeeping Act "do not have the force and effect of law," and are "directory, not mandatory in nature." Courts have not been shy in finding executive departments' regulations to be substantive rules exceeding authorities under the Housekeeping Act.
In CropLife Am. v. EPA, the D.C. Circuit Court of Appeals considered whether an EPA press release, which stated that the agency would not consider third party-controlled human exposure studies for purposes of pesticide registration subject to case-by-case consideration of individual studies, was a substantive rule. The court held that the press release bound both EPA and registrants during pesticide registrations and so was a binding "substantive rule." In reaching its decision, the court considered two established case law formulations for determining whether an agency action constitutes a substantive regulation: the effects of the agency's action; and the agency's expressed intentions regarding the action. Consistent with Chrysler Corp., the two analyses overlap in recognizing that a substantive action "binds private parties or the agency itself with the `force of law.'" In Croplife, the court determined: "EPA's stated rule is binding on petitioners, who are now barred from relying on third-party human studies (even in cases where such studies formerly were approved), and is binding on the agency because EPA has made it clear that it simply `will not consider' human studies."
Like the action at issue in Croplife, both the original proposal and the Supplemental Notice would bind EPA to not consider or to discount a scientific study it could have previously given full consideration, all else being equal, if the study fails to meet the new requirement that underlying data and models be publicly available (or available through tiered access). The only possible exception would be if an otherwise disqualified study met the proposal's exemption criteria and the Administrator exercised his or her discretion to grant an exemption. Likewise, the proposed rule would bind the public, including organizations such as the Environmental Defense Fund (EDF), who can no longer receive the benefit of EPA's full consideration of valid but non-complying studies that they submit to the agency as part of an administrative record for an agency action, which EPA would previously have been required to consider as part of the rulemaking process. Furthermore, EPA's rule would require researchers interested in contributing to regulatory protections and ISI to alter their conduct or risk having their efforts deemed unusable. Ultimately, the rule would diminish the public's statutorily protected interests in regulations informed by the best available science, an outcome that results from substantive choices about the scientific evidence it will consider.
Response: The purpose of the rule is to increase transparency of the science EPA uses to support its significant regulatory decisions and ISI. Nothing in the final rule will lead to the suppression of scientific data and models. This final rule does not regulate the rights and obligations of any party outside of the EPA let alone have legal force and effect on them. Therefore, any incidental impacts on voluntary behavior outside of the EPA do not render it a substantive rule.
Courts have held that an agency may bind itself to comply with a procedural rule. The Accardi doctrine holds that "government agencies are bound to follow their own rules, even self-imposed procedural rules that limit otherwise discretionary decisions." United States ex rel. Accardi v. Shaughnessy, 347 U.S. 260, 267 - 68, 74 S.Ct. 499, 98 L.Ed. 681 (1954)). Accardi applies to "gratuitous" regulations that agencies did not need to promulgate because " `[a] court's duty to enforce an agency regulation [, while] most evident when compliance with the regulation in mandated by the Constitution or federal law,' embraces as well agency regulations that are not so required." United Space Alliance, LLC v. Solis, 824 F.Supp.2d 68, 82 (D.D.C. 2011) (citing Lopez v. F.A.A., 318 F.3d 242, 247 (D.C.Cir.2003); quoting United States v. Caceres, 440 U.S. 741, 749, 99 S.Ct. 1465, 59 L.Ed.2d 733 (1979)).
Comment [48-383]: Commenter (12727) writes that when evaluating additional factors courts use to distinguish substantive rules, the proposal identifies itself even more definitively as a substantive rule than the EPA action reviewed in Croplife. First, both the 2018 proposal and the Supplemental Notice were published according to notice-and-comment procedures in the Federal Register, unlike the EPA action evaluated in Croplife. Courts have considered whether the agency used full public notice-and-comment procedures, which an agency need not use when producing an "interpretive" rule or rule of agency procedure, as an indicator of a substantive rule. While courts consider an agency's own characterization of its action, they disregard claims that conflict with the record such as EPA's contention here that the 2018 proposal as supplemented "would not regulate the conduct or determine the rights of any entity outside the federal government." "The agency's characterization of its own action is not controlling if it self-servingly disclaims any intention to create a rule with the "force of law," but the record indicates otherwise." And instead of a paragraph in a mere press release, EPA proposes to publish in the Code of Federal Regulations (CFR) criteria for barring studies from both regulatory and non-regulatory use -- a hallmark of a substantive regulation. EPA's choice to put the rule through notice-and-comment rulemaking provides further evidence that EPA regards the rule as substantive.
Response: EPA voluntarily provided notice and comment on this procedural rule to notify the public of the internal steps the agency will take to increase transparency of the science used to support significant regulatory decisions and ISI. There are no provisions in the final rule that regulate outside conduct. Nor are there any provisions that bar the use of any studies. EPA has explained that this procedural rule has limitations on its scope in terms of conflicts with other substantive requirements and that, in situations where this rule applies, EPA will give greater consideration to those studies where the underlying data and models are available for independent validation. Many procedural rules are published in the CFR. Mere publication of those rules does not convert them into substantive rules.
Comment [48-408]: Commenter (12727) writes that argued in the inverse, the 2018 proposal and Supplemental Notice do not meet the requirements for a rule of agency organization, procedure, or practice, for purposes of the APA. Agency actions in this category are the opposite of substantive rules; they are those "that do not themselves alter the rights or interests of parties, although [they] may alter the manner in which the parties present themselves or their viewpoints to the agency." An agency action that "trenches on substantial private rights and interests" cannot be a rule of agency organization, procedure, or practice. A federal court recently vacated substantive regulations issued by the Department of Health and Human Services under their "mistaken[]" claim of authority under the Housekeeping Act, finding that "[t]he challenged rule is not . . . a mere housekeeping rule. The expansive definitions in the rule depart from the federal statutes, as explained above, changing the rights and responsibilities of health care providers."
By the same framework, the 2018 proposal and Supplemental Notice must be a substantive rule, exceeding the powers of the Housekeeping Act, because they affect private rights and interests. By restricting the scientific studies on which EPA may base final significant regulatory actions, EPA severely limits parties from relying on excluded studies in advocating for particular safeguards, or petitioning the agency to take a specific action, as the statute authorizes them to do. As noted in EDF's prior comments, because the rule would substantively impact agency conclusions and regulations, it impacts private rights and interests. The proposal does not allow private individuals to submit for consideration (or renders such submittal a nullity) studies that they would have been permitted to prior to the proposal, thus impacting the substantive standards that EPA is able to justify setting -- which has implications for the regulated community as well as for public health. EPA's proposed action "encodes a substantive value judgment [and] puts a stamp of approval or disapproval on a given type of behavior" by requiring regulatory actions to be supported only by certain scientific information deemed acceptable by the proposal.
Response: The final rule is a rule of agency procedure and practice. The rule applies only to EPA and does not obligate or affect the rights of any outside parties. Procedural rules cannot alter the rights or interests of parties but they "may alter the manner in which the parties present themselves or their viewpoints to the agency." James A. Hurson Assocs. v. Glickman, 229 F.3d 277, 280 (D.C. Cir. 2000). Because the agency will consider rather than categorically exclude studies where the underlying data and models are not available for independent validation, researchers need not alter the way they conduct their studies for them to be considered by EPA. However, if those researchers want their studies to receive greater consideration, they can voluntarily take steps to ensure that the data and models are publicly available to the greatest extent possible, given any statutory limitations on data availability. EPA will evaluate studies determined to be pivotal science, using the criteria in § 30.5, which will allow all high-quality studies to be factored into the agency's development of significant regulatory decisions and ISI.
Comment [48-433]: Commenter (12727) writes that EPA has even acknowledged that the "the bulk of the responsibility for instituting new methods for access to data and models" will "fall[] on outside parties" -- including both researchers and the CDC -- according to a memorandum prepared by committee staff to capture recent briefings on the rule before the House Committee on Science, Space, and Technology. For example, the memorandum records EPA's opinion that researchers, "would be responsible for managing the logistics of making the data and models publicly available in a manner that complies with the rule, in consultation with EPA staff," and the CDC would shoulder the burden of "hosting the data and models on its own servers, with CDC personnel working at the secure data enclave reviewing research proposals submitted by members of the public seeking to conduct their own analyses of study data and determining the level of access to grant on a case-by-case basis." These substantial impacts on parties outside of EPA further confirm that the proposal is not an internal or procedural rule, but a substantive rule which cannot be issued under the Housekeeping Act.
Response: This procedural rule does not impose obligations or requirements on outside parties. Neither the CDC nor researchers are required to take any actions due to this rule. EPA discussed in the preamble of the 2020 SNPRM that it is investigating secured data access options with the pilot study using a CDC secure data enclave to host EPA datasets in a restricted use environment. Any decision whether to use CDC's data enclave to host EPA data would be issued separately from this final rule. 
As previously stated, nothing in this rule would prevent scientists from submitting their data and studies to EPA. Because the agency will consider rather than categorically exclude studies where the underlying data and models are not available for independent validation, researchers need not alter the way they conduct their studies. However, if those researchers want their studies to receive greater consideration, they can voluntarily take steps to ensure that the data and models are publicly available to the greatest extent possible, given any statutory limitations on data availability. 
Comment [48-438]: Commenter (12727) writes that the Housekeeping Act provides no basis to violate other statutes, including landmark environmental laws and the IQA. As discussed above, the Housekeeping Act authorizes only procedural rules regarding "the custody, use, and preservation of [agency] records, papers, and property," as opposed to "substantive rules." Regulations issued under the Housekeeping Act "do not have the force and effect of law," and have been "held to be directory, not mandatory in nature." While an agency is bound to adhere to all the laws it is required to administer, it is particularly egregious that EPA claims that a procedural housekeeping rule, which lacks legal force, would somehow supersede other binding statutory authorities and allow EPA to rewrite bedrock environmental statutes. The 2018 proposal and Supplemental Notice would require EPA to weigh the extra-statutory factor of data availability and disregard other high-quality data which EPA is required to review under the scientific standards of multiple environmental statutes. The courts have barred previous unlawful efforts to amend binding statutory and regulatory requirements via the Housekeeping Act. Treasury Department rules derived under the Housekeeping Act were found explicitly unable to "override" other binding Treasury Department regulations promulgated pursuant to specific statutory delegation. Similar logic was at work in Chrysler Corp. Concluding that section 301 did not authorize regulations limiting the scope of the Trade Secrets Act, the Supreme Court found the "greatest significance" to be "the `housekeeping' nature of § 301 itself."
As discussed above, EPA has styled the 2018 proposal and Supplemental Notice as a substantive rule that would bind the agency from considering scientific studies without publicly available datasets during rulemakings. However, even if the proposal were found to be validly issued under the Housekeeping Act, it could not in any way authorize violating environmental statutes or binding regulations implementing these statutes. Like the Trade Secrets Act, the major environmental statutes are binding authorities that cannot be limited by the regulations issued under the Housekeeping Act. Nor can the Housekeeping Act authorize rules that would supersede implementing regulations for environmental statutes which have been issued by specific statutory delegation in the same manner as the Treasury Department regulations discussed above.
Response: The final rule does not bind the agency from considering any particular studies. Nor does the rule authorize violations of the statutes and regulations EPA administers. The rule expressly states in § 30.3 that "[i]n the event the procedures outlined in this rule conflict with statutes that EPA administers, or their implementing regulations, the statutes and regulations will control." 
Comment [54-248]: Commenter (11361) writes that this SNPRM demonstrates a disingenuous request for public input from the EPA.
The EPA is obligated through the APA to notify the public and invite public comment on proposed regulations. While fulfilling the procedural requirements of the APA through this SNPRM, the EPA downplays the public interest of this sweeping proposed regulation, repeatedly asserts that this proposed rule is an affair internal to the agency, and sets up a path for the agency to dismiss public input regarding the STRS rule.
In the first paragraph of the Executive Summary, the EPA appears to dissuade public comment by understating the interest of members of the general public in a proposed rule designed to significantly alter regulations protecting public health.
First, the EPA frames the proposed rule as internal to the agency, stating "This SNPRM does not regulate any entity outside the Federal Government. Rather, the proposed requirements would modify the EPA's internal procedures regarding the transparency of science underlying regulatory decisions." The EPA then concedes that "any entity interested in EPA's regulations may be interested in this proposal," providing an example of "entities that conduct research or another scientific activity that is likely to be relevant to EPA's regulatory activity."
The EPA claims to have authority to make extensive changes to how the agency uses science in its decision-making through the Federal Housekeeping Statute, and dwells on this point in the SNPRM for several paragraphs. Remarkably, the EPA suggests that this rule, which ignores established processes for evaluating scientific merit and would impact every regulation under the EPA's authority, is not a "substantive rule." The EPA asserts that the proposed rule "exclusively pertains to the internal practices of the EPA" and fails to mention that the EPA relies extensively on studies conducted by other entities, and indeed, is mandated to consider outside research under both the CAA and CWA. The EPA does not address how the application of the proposed rule could impact its ability to faithfully execute its statutory responsibilities under these acts; it simply makes reference to its intention to be consistent with its other statutes. As the EPA repeatedly asserts that this proposed rule deals with internal EPA processes, the EPA invites public comment about whether to pursue this proposed rule under only its housekeeping authority, or under its housekeeping authority along with its other statutory authorities, and conspicuously does not invite comment about whether or not invocation of the housekeeping authority is appropriate at all. Let it be clear: a rule of this breadth, with impacts extending into all significant agency decisions and actions, should by no means be considered a basic agency procedure and housekeeping.
Response: Rather than undermining public input, EPA voluntarily provided two opportunities for public comment and also held a public hearing on the 2018 proposal. EPA has used the criteria found in EPA's Framework for Human Health Risk Assessment to Inform Decision Making (Ref. 69) to evaluate extramural research for use in regulatory decisions. EPA has made that policy choice transparent by providing public notice before implementation. 
Comment [55-272]: Commenter (10974) writes that EPA does not have the authority to use the Federal Housekeeping Statute in 5 U.S.C. § 301 to promulgate this rule.
Lower courts have since addressed what differentiates a housekeeping policy from a substantive rule. A recent U.S. District Court decision analyzed what constitute "housekeeping" versus "substantive" rules, as the APA does not define what a "substantive" rule is. The court found that a "substantive" rule "announces new rights and imposes new duties  -  one that shapes the primary conduct of regulated entities." Other courts have found that substantive rules "jeopardize or substantially affect the rights and interests of private parties", and that substantive rules are "legislative-type" rules that are described as "affecting individual rights and obligations."
The crux of EPA's purported authority to use the housekeeping rule in this case relies, at its core, on the definition of a "substantive" rule, for the following two reasons.
First, the 2018 NPRM and the SNPRM would unmistakably, and significantly, affect individuals whose private health information may become public, as well as corporate CBI submitted to the EPA as included in research findings. Entities and individuals conducting research would also be affected  -  if they did not want to release underlying data, the EPA could (and likely would) give less credence to their studies while creating regulations. In these ways, the rule would meet the definition of a substantive rule.
Second, if finalized, this rule would have sweeping implications on how EPA and non-EPA research is conducted. It is inaccurate to imply, as EPA does, that this rule would not impact the entire field of environmental health research and state policymaking  -  it would. Due to the likelihood of this far-reaching impact, which will go well-beyond the confines of the agency itself and its internal housekeeping policy, EPA is not authorized to use this housekeeping authority in this context.
Based on our legal interpretation of 5 U.S.C. § 301 and the context under which it was created, the MPCA and MDH affirm that EPA does not have the authority to promulgate this substantive rule based solely, or in part, on the agency's housekeeping authority  -  nor any of the other underlying environmental rule authorities listed in the SNPRM, corrected or as otherwise listed in the 2018 NPRM.
Response: Nothing in the final rule mandates or compels the release of private health information or CBI. EPA has been clear in § 30.5 that where the Agency is making data or models publicly available, "it shall do so in a fashion that is consistent with law, protects privacy, confidentiality, confidential business information, and is sensitive to national and homeland security."
Because the agency will consider rather than categorically exclude studies where the underlying data and models are not available for independent validation, researchers need not alter the way they conduct their studies. However, if those researchers want their studies to receive greater consideration, they can voluntarily take steps to ensure that the data and models are publicly available to the greatest extent possible, given any statutory limitations on data availability.
Also see the response to Comment 31-135.
Comment [59-327]: Commenter (11243) writes that the failure of EPA to identify a colorable legal basis for this rulemaking is emblematic of the Trump administration's brazen disregard for the rule of law.
The original proposal laughably gestures at EPA's various authorizing statutes as legal authority. The ridicule this claim engendered appears to have spurred one of the most significant aspects of the supplemental proposal. Namely, the new claim that this rulemaking is authorized by the federal housekeeping statute. This argument has two critical flaws though. First, the federal housekeeping statute does not apply to the EPA, only executive departments. Second, even if the statute did apply to EPA, it would not supply the legal basis for something like this rulemaking.
Response: See the response to Comment 31-135.
Comment [59-328]: Commenter (11243) notes that EPA acknowledges that it is not an executive department but argues that it was nonetheless brought within the scope of the federal housekeeping statute through Reorganization Plan Number 3 of 1970, which created the agency. The essay appended to my oral presentation explains in greater detail why this argument should be rejected. For now, I will emphasize two points. One, Reorganization Plan Number 3 conspicuously makes no mention of the federal housekeeping statute. Instead EPA has left to infer the transfer of that authority to a vague catch-all provision. In essence, then the agency claims Congress implicitly intended for EPA to be considered a department, but just hasn't gotten around to officially declaring it.
Two, while Congress has updated the list of executive departments several times since 1970, it has never included the EPA. Most recently it did so with the DHS which, like EPA, was pieced together from several existing agencies.
Response: See the response to Comment 31-135.
Comment [59-330]: Commenter (11243) writes that one, the censored science rule is a far cry from the kind of modest and noncontroversial internal operating procedures that Congress envisioned with the federal housekeeping statute. To wit: the original censored science proposal is so controversial it attracted over six hundred thousand public comments. Two, even the Supreme Court case that EPA cites to support this argument that the rulemaking is covered by the federal housekeeping statute, Chrysler Corp. versus Brown , makes clear that the censored science rule exceeds the modest authority that the law provides. Among other things, the Court in Chrysler Corp. was troubled by how the rule at issue affected the relationship between the government and private sector entities.
Significantly, the operative function of the censored science rule is to affect the relationship between EPA and members of the public. Specifically, it would fundamentally alter how the public participates in the development of new rules by limiting the kinds of views that they can share on a scientific basis per those rules. Today you will hear many reasons for why the EPA should abandon the censored science rule. As I have explained, the lack of a legal basis for the rule provides one more reason. Thank you for your attention.
Response: There is no provision of the final rule that would affect the relationship between EPA and private sector entities. No party outside of EPA is obligated to take any steps to comply with this internal agency procedure. Further, the rule will not affect the public's ability to participate in the development of new rules. The agency provides notice of all studies that it considers in regulatory decisions; a practice that was codified at § 30.4. During the comment period on a proposed rule, the public can comment on EPA's evaluation of the science and can also submit studies to EPA that it believes the agency should consider.
Comment [71-404]: Commenter (11354) writes that there is a lack of authority. The proposal has changed the authority EPA seems to rely on. And it's solely on housekeeping authority statute for executive departments, in combination with corrective environmental statutes listed in the original proposal.
This really is a backward rulemaking; proposing an action and then asking for help to determine authority. It is cited authority that doesn't authorize the rule. And -- in an event of a conflict, the proposal says that other statute or regulations may apply.
Response: See the response to Comment 31-135 and APA response to Comment 25-94.
Comment [73-409]: Commenter (11576) writes that the Agency's new standards for evaluating scientific studies are not internal "housekeeping" requirements.
As the Supreme Court has noted, the federal housekeeping statute was originally "`enacted to help General Washington get his administration underway by spelling out the authority for executive officials to set up offices and file Government documents" -- documents "pertaining to the day-to-day business of Government[.]" In its current form, which has been codified in Section 301 of Title 5, the statute provides that:
The head of an Executive department or military department may prescribe regulations for the government of ... [the] department, the conduct of its employees, the distribution and performance of its business, and the custody, use, and preservation of its records, papers, and property. This section does not authorize withholding information from the public or limiting the availability of records to the public.
Both the terms of the housekeeping statute and its legislative history accordingly demonstrate that it is "simply a grant of authority to ... agenc[ies] to regulate ... [their] own affairs." In the words of the Supreme Court, Section 301 "authoriz[es] what the ... [APA] terms `rules of agency organization[,] procedure[,] or practice' as opposed to `substantive rules.'"
With its supplemental notice, the EPA has admitted that it is "not one of the 15 `Executive Departments'" Congress has identified in Title 5. The agency argues, however, that it has "been granted full section 301 or equivalent authority" under Reorganization Plan No. 3 of 1970 -- and that this authority is sufficient, maybe, to support the proposed rule. According to the EPA, the regulation "exclusively pertains to the internal practices" of the agency, as it merely "describes how EPA will handle studies when data and models underlying science that is pivotal to EPA's significant regulatory decisions or influential scientific information are or are not publicly available in a manner sufficient for independent validation and analysis." Because of this "internal" focus, the EPA contends, the proposed rule should be understood as an unremarkable exercise of the agency's housekeeping power.
The EPA's half-hearted attempt to cast the proposed restrictions on science as nothing more than internal "housekeeping" measures that lack substantive significance cannot be sustained. As a result, even if the agency has been granted "section 301 or equivalent authority[,]" that authority would not provide a foundation for the proposed rule.
Response: See response to Comment 31-135.
Comment [73-410]: Commenter (11576) writes that under the proposed regulation, the EPA would be required to consider a new (and sometimes dispositive) factor when deciding if it should rely on a particular scientific study: whether "the data and models underlying ... [the study] are publicly available[.]" As the EPA admits in its supplemental notice, this assessment of data availability would be separate from -- and subsequent to -- the agency's evaluation of a study's "quality[.]" It could accordingly result in the exclusion of "high-quality studies" from the agency's scientific analyses and rulemakings. In the words of the notice, if the proposed rule is finalized, the EPA:
would continue to use standard processes for identifying, evaluating, and reviewing available data, models, and studies. When the Agency has potentially identified multiple key studies or models of similar quality that could drive its subsequent decisions, the Agency will investigate the availability of the underlying data. If, for example, multiple high-quality studies exist but only two studies have data and models that are available for independent validation and reanalysis, EPA would only include those two studies as pivotal regulatory science and/or pivotal science in accordance with the 2018 proposed rulemaking. However, under the alternative approach in this supplemental proposal, EPA would consider using all available high-quality studies but give greater consideration to those two studies with data available for independent validation.
As the supplemental notice makes clear, the proposed rule would alter the EPA's substantive standards for evaluating scientific research -- undermining the agency's ability to protect public health and the environment in the process. The regulation accordingly can't be dismissed as a "housekeeping" measure that merely "govern[s] internal agency procedures." Indeed, the rule is at odds with the scientific standards that Congress has itself imposed on the agency. As the D.C. Circuit recently emphasized, the "EPA operates pursuant to multiple statutory mandates requiring that its decisions rest on various formulations of `the best available science'" -- among them, the CAA's requirement that "`[a]ir quality criteria ... accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare,'" and the TSCA's demand that the EPA's administrator "`make decisions ... based on the weight of the scientific evidence[.]'" In summarizing its own mission, the EPA has therefor acknowledged that it is obligated to "ensure that `national efforts to reduce environmental risks are based on the best available scientific information[.]'" The proposed rule is an unlawful effort to amend these standards by requiring the EPA to disregard or devalue high-quality studies based on an extra-statutory evaluation of data availability. As explained in comments on the original proposal, moreover, the regulation also promises to cause substantial harm to members of the public, who depend on the EPA to adopt safeguards based on the best available science.
In short, while the EPA has declared that it has "the authority to establish policies governing its reliance on science in the administration of its regulatory functions[,]" this authority actually lies in Congress. Because the proposed rule would require the EPA to make decisions based on a factor Congress omitted from the governing statutes, it is both arbitrary and unlawful.
Response: The purpose of the rule is to increase transparency of the science EPA uses to support its significant regulatory decisions and ISI. Nothing in the final rule will lead to the suppression of scientific data and models. EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In the final rule, EPA does not rely on, interpret, or apply provisions of a particular statute or statutes that it administers. EPA will undertake such efforts in forthcoming substantive actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. The rule expressly states in § 30.3 that "[i]n the event the procedures outlined in this rule conflict with statutes that EPA administers, or their implementing regulations, the statutes and regulations will control."
This final rule does not regulate the rights and obligations of any party outside of the EPA let alone have legal force and effect on them. Therefore, any incidental impacts on voluntary behavior outside of the EPA do not render it a substantive rule. See response to comment 25-94.
EPA has used the criteria found in EPA's Framework for Human Health Risk Assessment to Inform Decision Making (Ref. 69) to evaluate outside research for use in regulatory decisions. 
Comment [73-411]: Commenter (11576) writes that the EPA has itself confirmed the substantive nature of the proposed regulation by electing to pursue it through notice-and-comment rulemaking. As the Supreme Court has said, the only actions authorized under the federal housekeeping statute are "what the ... [APA] terms `rules of agency organization[,] procedure[,] or practice' as opposed to `substantive rules.'" And under the APA, "rules of agency organization, procedure, or practice" are exempt from the requirements of notice-and-comment rulemaking. If the proposed rule were actually an internal housekeeping measure, in other words, notice and comment wouldn't have been required. The EPA was correct in concluding otherwise. Its efforts to reframe the proposal now as one that "exclusively pertains to the internal practices" of the agency cannot be taken seriously.
During two recent briefings that were held at the request of the House Committee on Science, Space, and Technology, the EPA provided further confirmation that the proposed rule cannot be dismissed as an internal housekeeping measure. According to a memorandum prepared by the committee's staff, the EPA indicated that "the bulk of the responsibility for instituting new methods for access to data and models" under the proposed rule will "fall[] on outside parties" -- both researchers and the CDC. These responsibilities, moreover, promise to be burdensome. Researchers, for instance, "would be responsible for managing the logistics of making the data and models publicly available in a manner that complies with the rule, in consultation with EPA staff." And the CDC would be tasked with "hosting the data and models on its own servers, with CDC personnel working at the secure data enclave reviewing research proposals submitted by members of the public seeking to conduct their own analyses of study data and determining the level of access to grant on a case-by-case basis." In light of the significant burdens the rule would impose on other agencies and private parties, the proposal cannot be defended as a housekeeping measure that governs the internal operations of the EPA.
Response: See responses to Comments 31-135 and 48-433.
Comment [81-583]: Commenter (112715) writes that EPA lacks authority to promulgate the proposed rule, which conflicts with statutory requirements regarding EPA's consideration of scientific information.
As explained in the 2018 Comments, EPA lacks statutory authority to promulgate the proposed rule. Nothing in the supplemental proposal alters that conclusion. EPA's citation to the federal Housekeeping Statute, 5 U.S.C. § 301, either alone or in combination with other substantive statutes that EPA administers, does not in any way alleviate this problem. First, a rule that requires EPA to exclude or give less weight to scientific information based purely on the availability of underlying data for independent validation is inconsistent with EPA's legal duty under numerous statutes it implements to use the best available science. Second, EPA may not rely on a general grant of authority, i.e., housekeeping, to promulgate regulations or develop policies that are otherwise inconsistent with more specific statutory directives. Third, 5 U.S.C. § 301 does not apply to EPA. Fourth, even assuming EPA has some inherent housekeeping authority apart from 5 U.S.C. § 301, the proposed rule does not constitute a "housekeeping" measure given its massive substantive impact.
Response: EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In the final rule, EPA does not rely on, interpret, or apply provisions of any particular environmental statute or statutes that it administers. EPA will undertake such efforts in forthcoming substantive actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. The rule expressly states in § 30.3 that "[i]n the event the procedures outlined in this rule conflict with statutes that EPA administers, or their implementing regulations, the statutes and regulations will control."
Also, see the response to Comment 31-135. 
Comment [81-585]: Commenter (12715) writes that the Housekeeping Statute does not apply to EPA.
By its plain language, the federal Housekeeping Statute provides no authority to EPA. The statute states, in pertinent part, that "[t]he head of an Executive department or military department may prescribe regulations for the government of his department, the conduct of its employees, the distribution and performance of its business, and the custody, use, and preservation of its records, papers, and property." 5 U.S.C. § 301. Moreover, as EPA concedes, a separate statute, 5 U.S.C. § 101, provides an exclusive list of the executive departments that are covered by the Housekeeping Statute and EPA is not included in that list.
EPA instead argues that it "gained" the housekeeping authority of 5 U.S.C. § 301 through the Reorganization Plan No. 3 of 1970. That Plan created EPA by transferring certain functions from the Department of Health, Education and Welfare ("HEW") to the newly established agency. HEW was later divided into what are now the Departments of Education and Health and Human Services. What EPA fails to note, however, is that Congress then amended 5 U.S.C. § 101 to add the Departments of Education and Health and Human Services, granting those agencies the housekeeping authority of 5 U.S.C. § 301, but did not also add EPA. Congress subsequently amended 5 U.S.C. § 101 several times to add other federal entities to the list but, again, it never added EPA. It is thus clear that Congress knew how to give housekeeping authority to executive entities when it wanted to, and the absence of EPA from 5 U.S.C. § 101 reflects congressional intent to not confer the authority of 5 U.S.C. § 301 on EPA.
The inapplicability of the Housekeeping Statute here is further confirmed by a 2001 opinion by the U.S. Department of Justice (DOJ) OLC that rejected a similar attempt by the Office of Government Ethics (OGE) to arrogate to itself the housekeeping authority in 5 U.S.C. § 301. Opinion of the OLC of the DOJ, 25 Op. O.L.C. 13 (2001). As OLC observed, Congress used both "executive department" and "agency" in title 5 (see, e.g., sections 302, 305), "and we presume that that difference was intentional." Id. at 15. "The fact that Congress, in conferring particular powers, distinguished between the heads of executive departments in section 301 and the heads of agencies in section 302 counsels against assuming that Congress meant to confer the authority in section 301 on the heads of all executive agencies." Id. at 15-16. EPA thus may not use 5 U.S.C. § 301 as authority for the proposed rule.
Nor can EPA rely on the 2008 OLC opinion addressing EPA's authority to establish a policy for its employees' use of government personal property and agency-issued cell phones. Opinion of the OLC of the DOJ, 32 Op. O.L.C. 1 (2008). First, and far from supporting EPA's position, that opinion explicitly finds that "EPA is not an `Executive department' within the meaning of section 301," and that any housekeeping authority would come from EPA's organic statute. Id. at 82. Second, unlike the proposed rule here, the EPA policy addressed by the 2008 opinion had no effect outside of EPA and was not inconsistent with specific statutory directives regarding the subject matter of the policy. Thus, like the DOL regulations the Supreme Court found defective in Chrysler Corp. v. Brown, 441 U.S. 281, 310-12 (1979), the proposed rule finds no support in 5 USC § 301.
Response: In the preamble, EPA acknowledges that it is not listed as an "Executive Department" under 5 USC § 101. The 2008 OLC acknowledges the 2001 OLC opinion cited by the commenter. There, OLC finds authority for OGE's Touhy regulations in the Federal Records Act and OGE's organic statute, 5 U.S.C. app § 401. In its 2008 opinion, OLC described Reorganization Plan No. 3 as EPA's organic statute and found that the Plan provided EPA with housekeeping authority:
      The difficulty here, however, is that section 301 confers regulatory authority only on the "heads of Executive departments and military departments," and not the heads of other executive agencies, such as EPA. 5 U.S.C. § 301; see also 5 U.S.C. § 101; Authority of the Office of Government Ethics to Issue Touhy Regulations, 25 Op. O.L.C. 13, 15 (2001) (recognizing that section 301 authority is limited to the listed departments). In considering whether the EPA Administrator may exercise housekeeping authority equivalent to that under section 301, we must consider whether such authority has been conferred under EPA's organic statute. 
      The Reorganization Plan establishing the EPA vests the Administrator with authority equivalent in many respects to that enjoyed by the head of an executive department. Reorganization Plan No. 3 of 1970, § 1(b), 84 Stat. 2086, 2086 (July 9, 1970) (codified at 5 U.S.C. app. 189 (2006)).2 The Reorganization Plan, which names the Administrator the "head of the agency," id., transfers to the Administrator functions previously vested by law in the heads of other executive departments, including functions of the Secretary of the Interior and the Secretary of Health, Education, and Welfare. Id. § 2(a)(1) - (4). The Administrator's authority is not limited to those designated functions but also includes "[s]o much of the functions of the transferor officers and agencies" that are "incidental to or necessary for . . . the performance of," or "primarily related to," such functions. Id. § 2(a)(9). This ancillary authority includes "authority, provided by law, to prescribe regulations relating primarily to the transferred functions." Id. At the time of the Reorganization Plan, such ancillary authority included the housekeeping authority conferred by 5 U.S.C. § 301 on the heads of those departments to enable their subordinates to carry out efficiently the statutory functions transferred to the Administrator of EPA. See 5 U.S.C. § 301 (Supp. II 1966). To perform those transferred functions, the Reorganization Plan further provides that the Director of OMB shall transfer to EPA "personnel, property, records, and unexpended balances of appropriations . . . used, held, available, or to be made available in connection with the functions transferred to the Administrator or the Agency." Reorganization Plan No. 3 of 1970, § 4(a).
      Taken together, these provisions convey to the Administrator all of the house-keeping authority available to other department heads under section 301, including authority to adopt property management regulations. Congress has vested the Administrator with the authority to run EPA, to exercise its functions, and to issue regulations incidental to the performance of those functions. This grant includes the authority to assign responsibility to others within the agency and to issue regulations prescribing the standards by which those functions are to be performed. The effective and efficient management of the agency's personnel and property is plainly "incidental to" and "necessary for" the performance of the functions that the Administrator is charged with performing. Indeed, the Reorganization Plan specifically recognizes this authority by providing the Administrator not only with transferred functions but with the personnel and equipment necessary for the effective performance of those functions. 
32 Op. O.L.C. 79, 82-83 (2008).
Also see the response to Comment 31-135.
Comment [81-586]: Commenter (12715) writes that the proposed rule is not a matter of "housekeeping," but instead has broad, substantive effect.
Even assuming EPA has some inherent authority apart from 5 U.S.C. § 301 to make rules that apply exclusively to internal EPA practices, the proposed rule is not authorized by any housekeeping authority because it would unquestionably have broad impacts outside of EPA. Indeed, the very fact that EPA is engaging in notice-and-comment rulemaking is an implicit admission that the proposed rule is not a matter of housekeeping since housekeeping rules are exempt from APA requirements. "The APA exempts from [5 U.S.C. § 553's] procedural requirements: (1) interpretative rules; (2) general statements of policy; and (3) rules of agency organization, procedure, or practice." Mendoza v. Perez, 754 F.3d 1002, 1020-21 (D.C. Cir. 2014). Conversely, rules to which the notice and comment requirements do apply are considered "substantive rules." Id.
More importantly, the nature of the proposal is not, on its face, a simple matter of housekeeping. Restricting the information that EPA staff can consider in setting standards and criteria to protect public health and the environment or requiring staff to arbitrarily put a thumb on the scale when weighing information in setting such standards, will directly impact the decisions that EPA makes regarding standard setting. Those standards apply not to EPA but to the regulated community, and many have nationwide applicability. In addition, many states' environmental laws and regulations explicitly adopt EPA standards, or at the very least require an express justification for any deviation. See 2018 Comments at 18-19. Furthermore, some states lack resources to develop their own standards and federal standards therefore apply by default. Id. EPA's proposed rule would thus have far-ranging impacts outside the agency and would directly affect the health of the nation's residents and natural resources. Indeed, EPA explicitly acknowledges that the supplemental proposal will have an impact on third parties by, for example: (1) requesting comment on how to cause researchers outside of EPA to change their practices to increase access to data, 85 Fed. Reg. at 15,403; and (2) conceding that the development of ISI will have "a clear and substantial impact on important public policies or private sector decisions," id. at 15,398.
We are also aware that EPA confirmed to Staff of the House Committee on Science, Space, and Technology that "the bulk of the responsibility for instituting new methods for access to data and models falls on outside parties. The researchers would be responsible for managing the logistics of making the data and models publicly available in a manner that complies with the rule, in consultation with EPA staff." This concession flatly contradicts the SNPR's assertion that the proposed rule "pertains exclusively to internal practices at EPA." 85 Fed. Reg. at 15,398. Accordingly, EPA should set aside the pretense that the proposal is a matter of agency housekeeping and directly confront the conflict between the proposed rule and the substantive statutes that command EPA to use the best available science.
In sum, EPA has entirely failed to address the 2018 Comments regarding the lack of legal authority for the initial proposal. Whether EPA is arbitrarily barring consideration of scientific studies or information with nonpublic underlying data, or arbitrarily according such studies or information less weight, the proposed rule unquestionably has substantive impacts and conflicts with the statutory commands that EPA use the best available science; no citation to "housekeeping" authority can cure that fatal flaw.
Response: The final rule does not regulate any party outside of EPA but rather exclusively governs EPA's internal process for considering pivotal science. This rule does not require any researcher or other outside entity to provide data or models to EPA. Instead, it governs internal agency procedures for considering various studies using factors including data availability. Furthermore, the final rule gives greater consideration to studies where the data and models are available for independent validation rather than providing for a categorical exclusion, which could be considered by researchers who are developing science and deciding whether to make the underlying data and models available. 
Comment [82-717]: Commenter (12464) writes that the Housekeeping Statute does not grant EPA authority to promulgate the proposal.
The Proposal is not authorized by the Housekeeping Statute, 5 U.S.C. § 301, because (i) Congress did not intend for the Housekeeping Statute to delegate rulemaking authority to independent agencies such as EPA; (ii) Reorganization Plan No. 3 does not grant EPA housekeeping or equivalent authority; and (iii) even if EPA has such housekeeping authority, the Proposal is a substantive rule that may not be promulgated under the Housekeeping Statute or equivalent authority.
Response: See response to Comment 31-135.
Comment [82-718]: Commenter (12464) writes that Congress did not Grant Independent Agencies, such as EPA, Rulemaking Authority under the Housekeeping Statute
The plain text of the Housekeeping Statute demonstrates that it applies only to Executive departments and military departments, and not to independent agencies such as EPA. The statute provides, in relevant part, that "[t]he head of an Executive department or military department may prescribe regulations for the government of his department, the conduct of its employees, the distribution and performance of its business, and the custody, use, and preservation of its records, papers, and property." 5 U.S.C. § 301. While the EPA is clearly not a military department, the agency also does not qualify as an "Executive department" within the meaning of the statute. An earlier provision in the same chapter of the United States Code provides an exclusive list of "the Executive departments." 5 U.S.C. § 101. EPA is not on that list.
It is a firmly established rule of statutory construction that definitions of terms included within the framework of a law usually dictate the meaning of those terms as used in the statute. Importantly, the definition of "Executive departments" does not start with the words "include" or "such as," which would indicate that the list of departments is non-exhaustive. Rather, the definition provides that "the Executive departments are" and proceeds to list those departments. Under the ordinary use of the verb "are," i.e., to be equal in meaning, the definitional list of Executive departments is exclusive. Moreover, according to the canon of negative implication, EPA's omission from this long list of departments (which has been intermittently amended) indicates that EPA is not an "Executive department" for purposes of the Housekeeping Statute.
Response: See response to Comment 31-135.
Comment [82-719]: Commenter (12464) writes that EPA is an "independent establishment," defined as "an establishment in the executive branch (other than the United States Postal Service or the Postal Regulatory Commission) which is not an Executive department, military department, Government corporation, or part thereof." 5 U.S.C. § 104. For decades, the United States Government Manual included EPA in the list of "independent establishments and government corporations" rather than in the list of "departments." Since 2010, the manual has changed the terminology to "independent agencies and government corporations," but EPA remains part of the same list. Independent establishments do not have rulemaking authority under the Housekeeping Statute.
The legislative history of the Housekeeping Statute confirms that Congress intended to grant rulemaking authority only to those executive departments listed under 5 U.S.C. § 101. When Congress enacted the current version of the Housekeeping Statute in 1958, a staff memorandum submitted to the Special Subcommittee on Government Information of the House Committee on Government Operations noted:
The statute does not apply to independent agencies, although some of the regulatory agencies cited it as authority to withhold information in answers to subcommittee questions. After the subcommittee discussed the matter with regulatory agency officials, they admitted that title 5, United States Code, section 22, [now 5 U.S.C. § 301] applies only to the executive departments of the Federal Government.
Response: See the response to Comment 31-135.
Comment [82-720]: Commenter (12464) notes that the staff memorandum recognized that such independent agencies were not listed in the definition of an executive department under what is now 5 U.S.C. § 101. Indeed, not only did the DOJ support this interpretation of the statute, but the Subcommittee also decried independent agencies' disregard of the definition when claiming such authority in the past; such claims were an "unhappy history" that pointed to the fact that "the `housekeeping' statute [had] been twisted from the original authority of the department head." Moreover, during the Subcommittee hearing, independent executive agencies, some of which have been superseded by new agencies or departments, clearly acknowledged the inapplicability of the Housekeeping Statute to their operations.
The fact that Congress intended to exclude independent agencies, such as EPA, from the Housekeeping Statute is further evidenced by the fact that one proposed version of the bill used the word "agency" rather than "Executive department" in 5 U.S.C. § 301, but that version was explicitly rejected because the "term is often used in referring to the independent regulatory agencies, to which the section does not apply, [and] it could lead to confusion and uncertainty in the future application of the section."
Response: See response to Comment 31-135.
Comment [82-722]: Commenter (12464) notes that Congress has since amended the definition of an "Executive department" under 5 U.S.C. § 101 to add two of the former independent agencies that had recognized the inapplicability of the statute to their operations in 1958 -- the Housing and Home Finance Agency (now the Department of Housing and Urban Development) and Veterans Administration (now the Department of Veterans Affairs).
Congress, by contrast, has not added EPA to the list. As these examples demonstrate, when Congress wanted to grant housekeeping authority to a former independent agency, it knew how to do so explicitly by amending Section 101. If an agency could assert housekeeping authority absent explicit congressional authorization, it would render Section 101 effectively meaningless. In sum, given that EPA is an independent establishment and acknowledges that it is "not one of the 15 `Executive Departments' listed at 5 U.S.C. 101," it cannot avail itself of the Housekeeping Statute.
Response: See the response to Comment 31-135.
Comment [82-784]: Commenter (12464) writes that Reorganization Plan No. 3 does not grant EPA housekeeping authority or equivalent authority.
Reorganization Plan No. 3, which established EPA in 1970, does not grant the agency housekeeping or equivalent authority. The Supplemental Notice asserts that although the agency "is not one of the `Executive Departments' listed at 5 U.S.C. 101," EPA gained full housekeeping or equivalent authority because Section 2(a)(9) of the Reorganization Plan transferred so much of the functions of the transferor offices and agencies, including those of the Department of the Interior (DOI) and Department of Agriculture (USDA), "as is incidental to or necessary for the performance by or under the [EPA] Administrator of the functions transferred." 85 Fed. Reg. at 15,397. According to a 2008 OLC opinion, at "the time of the Reorganization Plan, such ancillary authority included the housekeeping authority conferred by 5 U.S.C. § 301 on the heads of those Departments to enable their subordinates to carry out efficiently the statutory functions transferred to the Administrator of the EPA." Thus, the Supplemental Notice claims that the housekeeping authorities were implicitly transferred to EPA as incidental powers.
This argument is unpersuasive. As discussed above, Congress has demonstrated that it knows how to update the list of "Executive departments" to include newly created departments. Moreover, the DHS, like EPA, was created through the transfer and consolidation of various existing programs and authorities spread across several federal agencies. Indeed, similar to Reorganization Plan No. 3, the HSA of 2002 lays out in minute detail those functions and authorities that were being transferred to the new department from the Departments of Agriculture, Energy, and Treasury. However, unlike the Reorganization Plan, the HSA specifically asserts that the DHS is an executive department within the meaning of the Housekeeping Statute:
"There is established a DHS, as an executive department of the United States within the meaning of title 5, United States Code." Thus, when Congress intends to grant housekeeping authority, it does so by amending Section 101 to include a newly created department. No such mandate exists in Reorganization Plan No. 3 for EPA.
Response: See the response to Comment 31-135.
Comment [82-785]: Commenter (12464) writes that this interpretation of Reorganization Plan No. 3 is confirmed by its legislative history. When describing EPA's functions, Russell Train, Chairman of the Council on Environmental Quality, noted that "a reorganization plan cannot create any new legal authorities or functions;" thus, when EPA would come into being its functions could only be those of its constituent parts that were clearly transferred. The DOI's and USDA's housekeeping authorities were not so transferred.
Response: See the response to Comment 31-135.
Comment [82-803]: Commenter (12464) writes that although the Supplemental Notice cites certain court decisions in support of EPA's claim of housekeeping authority, the decisions were not carefully reasoned and cannot overcome the plain language of the Housekeeping Statute. For example, in Davis Enterprises v. U.S. Environmental Protection Agency, the Third Circuit concluded -- without any analysis -- that "[t]he EPA's authority to govern its internal affairs is derived from 5 U.S.C. § 301." In another case, the U.S. Court of Appeals for the Second Circuit stated that "the Federal Housekeeping Statute, 5 U.S.C. 301, authorizes government agencies such as the EPA to adopt regulations regarding `the custody, use, and preservation of [agency] records, papers, and property.'" While other federal courts have blindly followed these holdings in enforcing EPA's Touhy regulations, both the Second and Third Circuits' holdings show that the courts believed that EPA's housekeeping authority derived from 5 U.S.C. § 301 alone because the EPA was a "government agency" within the meaning of the statute -- a mistake in application that Congress attempted to avoid by enumerating the executive departments in Section 101. Such holdings are the type of "drive-by" rulings that the Supreme Court has cautioned against and deserve little to no weight.
Response: See the response to Comment 31-135.
Comment [82-807]: Commenter (12464) writes that the Transparency Rule is a substantive rule that may not be promulgated under Section 301's rulemaking authority.
Even assuming that EPA has Housekeeping Statute authority, the Proposal is a substantive rule outside the scope of the authority granted under that law. The purpose of the Housekeeping Statute is to delegate to executive and military departments the authority to prescribe rules related to their internal organization and operation; this authority does not extend to "substantive" rulemakings affecting the rights of parties and the department's implementation of its statutory duties.
The Housekeeping Statute was originally enacted in 1789 to help "General Washington get his administration underway by spelling out the authority for executive officials to set up offices and file Government documents." In the legislative history of the 1958 amendments, Congress further made clear that the laws prescribed under the statute were limited to "day-to-day business" and recordkeeping. Thus the Housekeeping Statute was never intended to be used as an authority to promulgate substantive rules with external legal consequences. The rule only authorizes what the APA "terms `rules of agency organization procedure or practice.'"
Response: As the Supreme Court stated in Chrysler Corp., whether a rule affects individual rights and obligations is an "important touchstone" for distinguishing substantive rules from other types of rules. This final rule does not regulate the rights and obligations of any party outside of the EPA let alone have legal force and effect on them. Therefore, any incidental impacts on voluntary behavior outside of the EPA do not render it a substantive rule.
Comment [82-833]: Commenter (12464) writes that although the APA does not define "substantive" rules, such rules may be recognized by negative inference from the statute's definition of "interpretative rules" (commonly known as "interpretive rules"). Interpretive rules are "general statements of policy, or rules of agency organization, procedure, or practice." 5 U.S.C. § 553(b)(A). The U.S. Supreme Court has therefore described a "substantive rule" as a "legislative-type rule" or as one "affecting individual rights and obligations." In coming to this definition, the Court relied heavily on the Attorney General's Manual on the APA because neither the House nor Senate Report for the APA attempted to expound on the distinction between substantive and interpretive rules. In a footnote, the Attorney General's Manual describes "substantive" rules as those rules "other than organizational or procedural," "issued by an agency pursuant to statutory authority and which implement the statute," and having "the force and effect of law."
Consistent with this definition, courts have forbidden executive departments from using Section 301 to implement substantive regulations. For example, last year, in New York v. U.S. Department of Health and Human Services, a district court found that the Department of Health and Human Services' (HHS) "conscience rule," recognizing the right of HHS funding recipients to abstain from participating in medical procedures on account of religious or moral objections, was a substantive rule impermissibly promulgated under the Housekeeping Statute. In coming to this conclusion, the court explained that although the rule had "housekeeping features," it was "largely substantive -- and, indeed, in key respects transformative." Specifically, the court found that the rule was substantive as the "conscience rule" would "effectively supersede Title VII in the health care field" by bypassing the undue hardship exception, "newly restrict[] the ability of employers to inquire about employees' conscience objections," newly define the term "health care entity" to apply to pharmacists and medical laboratories, and construe "the Weldon Amendment and the [Affordable Care Act], for the first time, to apply to health care plan sponsors and third-party administrators." The court held that "contrary to HHS's depiction of [the "conscience rule"] as mere housekeeping, the Rule relocates the metes and bounds -- the who, what, when, where, and how -- of conscience protection under federal law."
Response: Commenter's reliance on New York v. U.S. Dep't of Health & Human Servs., 414 F. Supp. 3d 475 (S.D.N.Y. 2019) (New York) is misplaced. The district court held that HHS could not rely on housekeeping statutes to promulgate its conscience rule provisions finding that the rule was "heavily substantive" because "[i]t shapes the rights and obligations of those subject to the Conscience Provisions." New York at 527. 
      A rule that announces new rights and imposes new duties -- one that shapes the primary conduct of regulated entities -- is substantive. See N.Y.C. Emps. Ret. Sys. v. SEC, 45 F.3d 7, 12 (2d Cir. 1995) (citing White v. Shalala, 7 F.3d 296, 303 (2d Cir. 1993)); see also Chrysler Corp., 441 U.S. at 302, 99 S. Ct. 1705 (substantive rules "affect[ ] individual rights and obligations" (internal citations omitted)); Thomas v. New York, 802 F.2d 1443, 1447 (D.C. Cir. 1986) (substantive rules "jeopardize or substantially affect the rights and interests of private parties" (internal citations and quotation marks omitted)). In some contexts, the distinction between substantive and procedural rules can be elusive -- "one of degree" and not kind. Elec. Privacy Info. Ctr. v. U.S. Dep't of Homeland Sec., 653 F.3d 1, 5 (D.C. Cir. 2011). But not here. At the time the Rule was promulgated, the President stated that it conferred "new protections." And that the Rule recognizes new substantive rights and imposes new substantive duties on regulated entities in the health care sector is apparent from each of the Rule's (1) purpose, (2) definitions, and (3) assurance and certification requirements.
Id. at 522-23. 
EPA's final rule does not establish rights or impose obligations on any outside parties unlike HHS's conscience rule which established rights and imposed duties on regulated entities in the health care industry.
Also see the response to Comment 31-135.
Comment [82-836]: Commenter (12464) writes that the Proposal is similarly substantive because it would establish a rule declaring that studies whose data and models are not "publicly available in a manner sufficient for independent validation" are deemed immaterial (or given less weight). 85 Fed. Reg. at 15,405. Such a rule would not only impact the ability of private parties to challenge the scientific bases for EPA's regulatory actions going forward but would also plainly restrict the agency's use of science in its determination of ISI and regulatory decisions.
Additionally, the Proposal is a substantive rule because it will impose significant burdens on researchers. As discussed above, the burden of implementing the tiered-access proposal will largely fall on researchers and universities. In particular, researchers will be responsible for determining which data and models can be made publicly available and managing the logistics of making the data and models available in a manner that complies with the rule. As the Proposal would place the bulk of the responsibility and costs for managing public access to data and models on people outside of the agency, it clearly has a binding external impact, and is substantive in effect.
Response: The purpose of the rule is to increase transparency of the science EPA uses to support its significant regulatory decisions and ISI. Nothing in the final rule will lead to the suppression of scientific data and models. As described in § 30.5, EPA will identify those studies that are given lesser consideration and provide a short description of why lesser consideration was given. The public can provide comments on that decision and can also submit additional studies that it believes the agency should have considered. Nothing in this rule would prevent scientists from submitting their data and studies to EPA. Researchers may voluntarily take steps to ensure that their studies are given greater consideration by making their data and models publicly available for independent validation. Additionally, the Administrator's exemption in § 30.9 can be used to allow greater consideration for pivotal science where the underlying data and models are not publicly available. Also see responses to Comments 34-144, 34-155, 40-193 and 48-379.
Comment [84-788]: Commenter (11390) writes that the proposed rule falls outside EPA's rulemaking authority. 
Despite EPA's assertion that the amended Proposed Rule governs "internal agency procedures" and "does not regulate any entity outside the Federal government," the Proposed Rule is, in fact, a substantive regulation. It will profoundly impact how EPA interacts with regulated parties by changing how the agency performs its core functions.
"It is axiomatic that an administrative agency's power to promulgate legislative regulations is limited to the authority delegated by Congress." Bowen v. Georgetown Univ. Hosp., 488 U.S. 204, 208 (1988). EPA steps far beyond that authority in the SNPRM, proposing a regulation that would change the way the agency engages in virtually any decision-making or scientific process -- outside of agency adjudications and enforcement activities -- whether or not that process ultimately leads to the adoption of regulations. EPA has revised the Proposed Rule to cite the agency's housekeeping authority as its basis to regulate. But EPA has no statutory authority to issue the Proposed Rule, either through the Housekeeping Statute or via other environmental statutes EPA has previously cited.
EPA now asserts "housekeeping" authority to issue the Proposed Rule, maintaining that these sweeping changes to agency practice and policy are nothing more than "procedural" rules or "internal agency procedures" that "do[] not regulate any entity outside the Federal Government," akin to how EPA manages its filing system. This is a serious mischaracterization of the Proposed Rule's import and effect.
The so-called Housekeeping Statute, 5 U.S.C. § 301 -- which does not, on its face, even apply to EPA --  was "originally adopted in 1789 to provide for the day-to-day office housekeeping in the Government departments," and courts at every level have consistently found that it does not authorize substantive rules. See, e.g., Chrysler Corp. v. Brown, 441 U.S. 281, 310 (1979); U.S. ex rel. O'Keefe v. McDonnell Douglas Corp., 132 F. 3d 1252, 1255-56 (8th Cir. 1998); City and County of San Francisco v. Azar, 411 F. Supp. 3d 1001, 1023 (N.D. Cal. 2019). While the Housekeeping Statute might appropriately authorize "a regulation that governs...the filing of government documents pertaining to the day-to-day business of government," it does not confer an "unrestricted grant of authority" allowing the agency to make broad policy changes that drastically alter the way it discharges its fundamental duties. See Respect Inc. v. Committee on Status of Women, 815 F. Supp. 1112, 1123 (N.D. Ill. 1993). Indeed, Housekeeping Statute authority is generally limited to actions that are exempt from the very notice and comment rulemaking procedures EPA has undertaken for the Proposed Rule (and that EPA has tacitly acknowledged are required by doing so). See Chrysler Corp., 441 U.S. at 310 (the Housekeeping Statute "authoriz[es] what the Administrative Procedures Act terms `rules of agency organization procedure or practice' as opposed to `substantive rules.'"); 5 U.S.C. § 553(b)(3)(A) (exempting rules of agency procedure, organization, and practice from notice and comment rulemaking).
EPA's characterization of the Proposed Rule as merely "procedural" or "internal" is erroneous and misleading. In reality, the Proposed Rule would have the effect of amending substantive and well-settled standards for decision-making under a number of the environmental statutes EPA administers, including the CAA, the TSCA, the SDWA, and others. For example, CAA § 109 mandates that EPA set air quality standards based on "air quality criteria," which must "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare." 42 U.S.C. §§ 7409, 7408(a)(2). The TSCA requires EPA to use the "best available science" when evaluating the testing and regulation of chemicals. 15 U.S.C. § 2625(h). Decisions are to be made using the "weight of the scientific evidence," and EPA is required to consider all information related to a chemical substance, including hazard and exposure information, "that is reasonably available to the Administrator." 15 U.S.C. §§ 2625(i), (k). Similarly, the SDWA mandates use of the "best available public health information" when EPA determines whether to regulate a contaminant, and reliance on the "best available, peer-reviewed science and supporting studies" when making regulatory decisions. 42 U.S.C. §§ 300g-1(b)(1)(B)(ii), 300g-1(b)(3). While EPA claims that "in the event the procedures outlined in this proposed rulemaking conflict with the statute that EPA administers, or their implementing regulations, the statutes and regulations will control," the Proposed Rule so directly conflicts with multiple statutory requirements as to render this assertion meaningless. In sum, the Proposed Rule cannot be implemented without upending statutory standards for the use of science.
Such fundamental change to agency policy is the very type of action courts have consistently characterized as substantive. See, e.g., Mendoza v. Perez, 754 F.3d 1002, 1021 (D.C. Cir. 2014) (a substantive regulation "supplements a statute, adopts a new position inconsistent with existing regulations, or otherwise effects a substantive change in existing law or policy."); Rocky Mountain Helicopters, Inc. v. F.A.A., 971 F.2d 544, 546 (10th Cir. 1992) (a rule that "changes existing law, policy, or practice" is substantive). "Misuse" of the Housekeeping Statute as purported authorization for these rules has repeatedly been rejected as an "attempt to twist this simple administrative statute into an authorization for the promulgation of substantive rules." See U.S. ex rel. O'Keefe, 132 F. 3d at 1255; see also Exxon Shipping Co. v. U.S. Dep't of Interior, 34 F. 3d 774, 776-78 (9th Cir. 1994); In re Cincinnati Radiation Litigation, 874 F. Supp. 796, 821 (S.D. Ohio 1995) (Housekeeping Statute did not authorize a Department of Defense directive mandating that the Department's experimentation on human subjects could only be conducted where the subject provided full and voluntary consent).
EPA's Proposed Rule would singlehandedly alter the way EPA discharges its core statutory responsibilities under the key environmental statutes it is tasked with administering, at significant cost to public health and safety. Far from a mere procedural dictate or internal standard, the Proposed Rule dramatically changes the nature of EPA's activities -- both regulatory and non-regulatory -- in ways that are inconsistent with existing law and a major departure from past agency policy. The Housekeeping Statute does not authorize such game-changing rulemaking.
Response: EPA disagrees with the commenter that the final rule effects a "substantive change" to existing policy. As discussed in the preamble of the 2018 proposal and the 2020 supplemental proposal, the final rule compliments and implements, but does not change, existing agency policies and OMB guidance on information quality and data availability. The study evaluation factors listed in § 30.5 and taken from existing agency guidance, the Framework for Human Health Risk Assessment to Inform Decision Making (Ref. 69). These factors will be used to evaluate studies deemed pivotal to a regulatory action or ISI. Because the rule does not categorically exclude any studies from consideration, EPA can still rely on pivotal science where the underlying data and models are not publicly available for independent validation. Additionally, an Administrator's exemption granted pursuant to § 30.9 would allow greater consideration of pivotal science lacking data and model availability. 
In FCC v. Fox Television Stations, Inc., 556 U.S. 502, 515 (2009), the Supreme Court explained that a more detailed justification is required when the "new policy rests upon factual findings that contradict those which underlay its prior policy," or "its prior policy has engendered serious reliance interests." (internal citations omitted). Assuming, arguendo, that the final rule does effect a substantive change in agency policy, EPA has provided a "detailed justification" for the change in the 2018 and 2020 proposals. Because this is a rule of internal agency procedure, outside parties could not have developed reliance interests in the prior policy. 
Also see responses to Comments 31-135 and 73-410.
Comment [87-753]: Commenter (11400) writes that this rule is much broader and more substantive than a mere "housekeeping" measure. 
EPA is taking comments on its authority to promulgate this rule under its housekeeping authority. In our view, this is not "administrative housekeeping". This rule creates rules to exclude scientific studies and applies to complex and studies with large price tags. This is not an "internal" matter to EPA, but instead affects broad research parameters that impact the scientific and health communities, and the affected public.
Response: See responses to Comments 31-135, 32-144 and 48-379.
Comment [88-763]: Commenter (11401) writes that like criticisms of EPA's attempts to hold itself to high scientific standards, criticisms about the use of the housekeeping authority are misplaced. The criticism extends from the fact that the Federal Housekeeping Statute should be internally focused, but that's exactly what's laudable about this proposed rule. EPA is aiming to regulate itself by holding itself to high standards in ensuring the best science is used. Some argue that it would limit EPA's ability to regulate, but EPA should be limited to only regulate when backed by sound science. Regulated communities have to meet strict standards imposed by EPA. It is only right that EPA is held to high standards itself. However, statutory authority from substantive statutes should be used in conjunction with this housekeeping authority.
Response: Thank you for your comment in support of the rule. However, EPA is relying exclusively on its housekeeping authority to promulgate this rule. In the final rule, EPA does not rely on, interpret, or apply provisions of a particular statute or statutes that it administers. EPA will undertake such efforts in forthcoming substantive actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. The rule expressly states in § 30.3 that "[i]n the event the procedures outlined in this rule conflict with statutes that EPA administers, or their implementing regulations, the statutes and regulations will control."
Comment [89-802]: Commenter (11403) writes that the Federal Housekeeping authority cannot be applied to substantive rulemaking The Federal Housekeeping authority is also of no help to EPA. Apparently recognizing that the previously listed environmental statutes do not provide EPA with the authority it seeks, the agency has invoked an even more tenuous argument -- that it may impose these regulations pursuant to the Federal Housekeeping statute, 5 U.S.C. § 301. That provision provides, in full:
The head of an Executive department or military department may prescribe regulations for the government of his department, the conduct of its employees, the distribution and performance of its business, and the custody, use, and preservation of its records, papers, and property. This section does not authorize withholding information from the public or limiting the availability of records to the public.
Arguing that this provision gives EPA the power to establish "internal practices," EPA claims that because these regulations govern how EPA internally treats scientific studies, the Federal Housekeeping statute is an additional source of authority. Once again, the agency is far off the mark.
In short, EPA is simply mistaken in suggesting that these rules are simply a matter of "internal practices" with no substantive effect. As we and many other commenters have detailed, the rules have enormous implications for the manner in which EPA regulates on behalf of public health and the environment, and is certain to lead to dramatic real-world effects as EPA refuses to rely on legitimate scientific studies to make regulatory decisions.
Indeed, the Supreme Court made this precise distinction in Chrysler Corp. v. Brown, noting that the Federal Housekeeping statute is "simply a grant of authority to the agency to regulate its own affairs," and provides no power for an agency to take an action with substantive effects. Given the real world impacts of these rules, EPA has no power to promulgate them under the Federal Housekeeping statute.
The single precedent on which EPA relies, United States v. Manafort, does not suggest otherwise, for the internal Special Counsel regulations at issue there concerned such matters as assignment of cases; staffing and budget matters; and reporting and compliance matters. Given that the regulations actually had no substantive effect, the Court naturally found they were duly promulgated under the Federal Housekeeping statute. Here, by contrast, again, these rules will have demonstrable substantive effect in governing how EPA treats scientific studies.
Indeed, while claiming authority under the Federal Housekeeping statute, it is apparent that EPA itself recognizes it is enacting a substantive rule, because it has subjected the rules to notice and comment rulemaking. As the Court explained in Manafort, that process is reserved for substantive rules not subject to the Federal Housekeeping statute, and an agency proceeding under that statute simply takes action, without public involvement.
Moreover, while, as explained, the rules in Manafort are completely unlike those at issue here, there is a long line of precedents expressly rejecting agency attempts to rely on the Federal Housekeeping statute to promulgate substantive rules. As another court explained in similarly rejecting an agency's reliance on the Federal Housekeeping statute, EPA's argument here "is simply one more attempt to twist this simple administrative statute into an authorization for the promulgation of substantive rules."
As the Supreme Court has emphasized, "an administrative agency's power to promulgate legislative regulations is limited to the authority delegated by Congress." Because these regulations are not permissible under any substantive grant of authority to EPA, and may not proceed under the Federal Housekeeping statute, EPA must abandon this effort to undermine the use of scientific studies in its decision-making.
Response: See responses to Comments 31-135, 32-144 and 48-383.
Comment [90-825]: Commenter (11404) writes that EPA lacks "housekeeping authority" and such authority would not justify the Proposed Rulemaking in any event.
Finally, EPA is incorrect in its assertion that it can use federal "housekeeping authority" codified in Title 5, section 301 of the United States Code as a justification for the Strengthening Transparency in Regulatory Science Proposed Rulemaking. First, EPA lacks federal housekeeping authority conferred on heads of Executive Departments. EPA is not an "Executive Department" within the meaning of the statute. See 5 USC § 101. Executive Departments, as defined, are Cabinet-level Departments of the federal government. EPA is not among those departments. EPA's assertion that it "gained housekeeping authority through the Reorganization Plan No. 3 of 1970" (85 FR 15397) which established EPA ("Plan No. 3") is legally and factually incorrect. Plan No. 3 makes no mention of such authority. Moreover, even if Plan No. 3 had addressed this issue, President Nixon lacked authority to expand the list of Executive Departments authorized by statute to exercise housekeeping authority. EPA's reliance on the Second Circuit decision in EPA v. General Electric Company, 197 F3d 592 (1999) ("GE") is misplaced. The court in GE did not analyze, much less decide, that EPA possesses housekeeping authority. Second, even if EPA had federal housekeeping authority, the Proposed Rulemaking extends far beyond the reach of housekeeping authority. The Supreme Court has made clear that housekeeping authority is only to be used to promulgate rules of agency procedure, organization, or practice -- not substantive rules. See e.g., Chrysler Corp. v. Brown, 441 US 281 (1979) (describing the purpose of housekeeping authority as being "to provide for day-to-day housekeeping" and not to include "substantive rules." Id., at 283). EPA's Proposed Rulemaking extends far beyond the modest, noncontroversial rulemaking for which housekeeping authority provides justification. The Strengthening Transparency in Regulatory Science proposal would fundamentally change how EPA uses science for regulatory purposes -- both in its future rulemakings and retroactively. Thus, even if EPA possessed such authority, the Proposed Rulemaking cannot be justified by the limited authority intended to be conferred by housekeeping authority.
Response: See response to Comment 31-135.
Comment [92-850]: Commenter (11396) writes that in addition to the lack of a scientific justification, EPA's legal authorization for the Proposed Rule, as supplemented by the SNPRM, is also wholly lacking. The limited "housekeeping" authority under 5 U.S.C. 301 (assuming it applies, which it likely does not), authorizing some federal agencies to regulate their own internal affairs, may not serve as the legal foundation for a rule that could allow EPA to exclude the best available science from its policymaking. As explained above, excluding the best available science from EPA policymaking is patently at odds with the agency's duties under multiple federal statutes. EPA is proposing here to alter the substantive standards for evaluating scientific research. But the federal housekeeping statute in no way authorizes EPA to make such substantive and impactful changes. Nor may the agency rely on the authority of the federal statutes it administers, which provide no legal basis at all for this proposal.
Response: See responses to Comments 31-135, 32-144 and 73-410.
Comment [93-853]: Commenter (11395) writes that as EPA acknowledges in the SNPRM, the Supreme Court ruled in Chrysler Corp. v. Brown, [441 U.S. 281, 309 (1979)]. that the intended purpose of the Housekeeping Statute is "to govern internal departmental affairs" and, as such, authorizes "what the [APA] terms `rules of agency organization, procedure or practice' as opposed to substantive rules." Because the proposed Strengthening Transparency rule would have far reaching substantive impacts on a wide range of EPA programs and activities, it is clearly inappropriate to invoke the Housekeeping Statute as authority applicable to this proposed rule.
Response: See response to Comment 31-135.
Comment [94-879]: Commenter (11408) writes that in the 2018 proposal, the EPA glibly asserted that authority for the rulemaking could be found in "the statutes its administers, including provisions providing general authority to promulgate regulations necessary to carry out the Agency's functions under these statutes and provisions specifically addressing the Agency's conducting of and reliance on scientific activity to inform those functions." (The 2018 proposal goes on to cite specific provisions in various authorizing statutes that provide general rulemaking authority or that create scientific research programs to support implementation of the particular statute.) Unsurprisingly, this position met almost immediately with broad condemnation and was singled out as one of the most flawed aspects of the 2018 proposal.
The supplemental proposal now seeks to remedy this threshold weakness in the original proposal by offering a new alternative legal justification for the rulemaking. Specifically, the supplemental proposal employs a novel reading of an obscure law known as the Federal Housekeeping Statute to identify a legal basis for the rule. (The supplemental proposal also sought public comment on whether to rely on the Federal Housekeeping Statute as an independent basis for the rule or to use that statute's authority in conjunction with the specific provisions of its authorizing statutes that were cited in the 2018 proposal.)
This new attempt to find legal authority for the rule has two critical flaws. First, the Federal Housekeeping Statute doesn't apply to the EPA  -  only "Executive departments," a category that does not include the EPA. Second, even if the statute did apply to the EPA, the modest authorities it confers on covered federal agencies would not supply the legal basis for the kind of far-reaching requirements and limitations contemplated in the rulemaking. Because neither the 2018 proposal nor the supplemental proposal has identified a sound legal basis for this rulemaking, the rulemaking presents a clear case of an action that is "in "in excess of statutory . . . authority," rendering it highly vulnerable to rejection on judicial review.
Response: See response to Comment 31-135.
Comment [94-880]: Commenter (11408) writes that the operative provision of the Federal Housekeeping Statute reads: "The head of an Executive department or military department may prescribe regulations for the government of his department, the conduct of its employees, the distribution and performance of its business, and the custody, use, and preservation of its records, papers, and property." A related law provides the definitive list of the "Executive departments" covered by the Federal Housekeeping Statute. Since its creation, the EPA has remained conspicuously absent from this list.
The supplemental proposal acknowledges this obvious shortcoming but asserts that the 1970 legislation that created the EPA, Reorganization Plan No. 3, still served to bring the agency within the scope of Federal Housekeeping Statute. Reorganization Plan No. 3 formally established the agency while laying out all the programs and authorities that were being transferred to it from already existing agencies, including the DOI and an agency then known as the Department of Health, Education, and Welfare.
Significantly, however, Reorganization Plan No. 3 never directly mentions the Federal Housekeeping Statute. To overcome this barrier, the supplemental proposal adopts the legal theory that the EPA's Office of General Counsel (OGC) employed in a 2008 memorandum opinion concerning an unrelated regulation. That opinion contends that Section 2(a)(9) of the Plan implicitly confers this authority through a vague "catchall" provision. Specifically, this provision calls for the transfer of "So much of the functions of the transferor officers and agencies referred to in or affected by the foregoing provisions of this section as is incidental to or necessary for the performance by or under the Administrator of the functions transferred by those provisions or relates primarily to those functions."
In essence, then, the supplemental proposal argues that, when Congress formally approved Reorganization Plan No. 3, it has also implicitly intended to regard the EPA as an Executive department  -  or at least imbue it with some of the trappings of an Executive department  -  but in the nearly 50 year since has not gotten around to officially declaring it by updating the relevant statutory list. This argument is unpersuasive.
Since the enactment of the Federal Housekeeping Statute, Congress has amply demonstrated that it knows how to update the statutory "Executive departments" list, and it has done so on several occasions both before and after it approved Reorganization Plan No. 3. Indeed, Congress updated the list a little over a month after Reorganization Plan No. 3 to remove the Post Office Department from the list. That seemingly would have provided an ideal opportunity for Congress to take the step of explicitly adding the newly created EPA.
Congress has had several opportunities in the nearly 50 years since the EPA was created to update the statutory "Executive departments" list, and it has conspicuously failed to do so. Most notably, Congress could have done so in 1979 when it updated the list to reflect the reorganization of the old Department of Health, Education, and Welfare  -  from which many of the EPA's original functions and authorities had sprung  -  into the new Departments of Education and Health and Human Services, respectively. The fact that Congress has yet not updated the statutory "Executive departments" list to include the EPA despite ample opportunities to do so cannot be wished away by a memo from the agency's OGC.
Of greater relevance, Congress has shown that it knows how to update the statutory "Executive departments" list to include new departments that are created through the transfer and consolidation of various existing programs and authorities spread across several federal agencies, as was the case with the EPA. Congress took precisely this step when it created the DHS. Like Reorganization Plan No. 3, the HSA of 2002, adopted in response to the September 11th terrorist attacks, lays out those functions and authorities that were being transferred to the new DHS from where they had previously existed at agencies like the Departments of Agriculture, Energy, and Treasury. Significantly, in transferring these functions, the HSA also specifically provides in Section 101 that the DHS was being "established . . . as an executive department of the United States within the meaning of [the title of the U.S. code that contains the Federal Housekeeping Statute]."
Another problem with the supplemental proposal's argument is that the EPA has evolved as an agency in significant ways since its creation. This then raises intractable difficulties regarding the appropriate present-day application of the catchall provision contained in Reorganization Plan No. 3. Through various authorization and reauthorization acts, the present-day EPA is no longer the same agency that was established via Reorganization Plan No. 3, and it now carries out functions and authorities that were not contemplated at the time the Plan was drafted. Under the OGC's theory adopted by the supplemental proposal, it stands to reason that these new functions and authorities would not benefit from the implicit transfer of the Federal Housekeeping Statute authorities. That result would put current EPA officials (and potentially reviewing judges) in the awkward position of policing the lines between those agency functions and authorities covered by the Federal Housekeeping Statute and those that are not. Such a bizarre result weighs further against accepting the theory that the catchall provision in Reorganization Plan No. 3 served as an implicit transfer of the authorities conferred by the Federal Housekeeping Statute.
To be sure, the EPA has for decades issued regulations under claimed authority from the Federal Housekeeping Statute and the appropriateness of the legal basis for these regulations appears to have gone unchallenged. But a legal wrong cannot be made right through simple repetition. If it did, then agencies would face strong incentives to break the law as much as they could before getting caught. It is also worth noting that these previous applications involved genuinely noncontroversial internal procedures for the agency  -  the kind of procedures that the Federal Housekeeping Statute was designed to authorize, as noted below. As such, it would be unreasonable to construe the lack of a challenge to the legal basis for these actions as a broad recognition or acceptance of the appropriateness of their legal basis. Instead, it is more likely that their noncontroversial nature was such that they failed to attract the kind of intensive legal scrutiny that would have exposed their inadequate legal basis to begin with.
Response: See response to Comment 31-135.
Comment [94-881]: Commenter (11408) writes that even if the supplemental proposal was correct in its assertion that the EPA is covered by the Federal Housekeeping Statute, it does not necessarily follow that the authorities it confers would supply the legal basis for this rulemaking. On the contrary, it is clear that the modest authorities that the Federal Housekeeping Statute affords to covered agencies are insufficient to support this rule, since its primary effect is external in nature because it seeks to alter how the general public interacts with the EPA for the purpose of affecting its regulatory decision-making or informing its policy-relevant scientific determinations.
To support its contention that the Federal Housekeeping Statute provides it with authority to issue the rule, the supplemental proposal curiously cites the Supreme Court case Chrysler Corp. v. Brown. The citation of this case is curious because the Court's reasoning there cuts unmistakably against this argument. At issue in Chrysler Corp. was a DOL regulation that sought to compel the public disclosure of reports provided to the government by government contractors concerning their workforce composition, even though such disclosures would seemingly run afoul of the Trade Secrets Act. The Court struck down this regulation, finding that the modest reach of the Federal Housekeeping Statute was too limited to empower the DOL to displace the mandates imposed on it by Trade Secrets Act.
Crucially, in reaching its decision, the Court emphasized the essential "`housekeeping' nature" of the Federal Housekeeping Statute and characterized the law as "simply a grant of authority to the agency to regulate its own affairs." The Court had little trouble contrasting this understanding of the statute's modest reach with the DOL's regulation. Indeed, the material impacts of this regulation unavoidably extended well beyond what might be considered routine internal agency operating procedures given that its function was to alter the application of the Trade Secrets Act, a law that Congress enacted to govern privacy issues that arise from the federal government's interactions with private sector businesses. Accordingly, the Court concluded that Congress had not intended the Federal Housekeeping Statute to authorize something like the DOL's workforce composition disclosure regulation.
The logic employed by the Court in Chrysler Corp. is readily applicable to this rulemaking. As was the case there, this rulemaking cannot be reasonably understood as governing the EPA's internal operating procedures. That is because the provisions in this rulemaking are directly and inextricably tied to the implementation of the EPA's various authorizing statutes, as well as to its adherence to the APA. The rulemaking's impact on the APA is significant in this regard because the function of that law is to govern how the federal government interacts with members of the public in the development of new regulations. As such, this rulemaking is practically indistinguishable from the one at issue in Chrysler Corp., which, as noted above, sought to alter the DOL's adherence to the Trade Secrets Act, a law that similarly seeks to manage certain kinds of interactions between the government and the general public. In relevant part, the APA directs agencies to "give interested persons an opportunity to participate in the rule making through submission of written data, views, or arguments." It further directs them to develop a final rule based on the "consideration of the relevant matter presented."
The rulemaking would undeniably impact the EPA's compliance with these public participation requirements under the APA. In particular, its function is to fundamentally alter how members of the public are able to contribute to the development of the EPA's regulations and policy-relevant scientific determinations. Among other things, it would limit the kinds of "written, data, views, or arguments" that "interested persons" can submit to the EPA during the public comment process, while also drastically overhauling what the EPA can regard as "relevant matter presented" for its "consideration" in developing a final rule. As such, because this rulemaking goes well beyond routine internal housekeeping procedures, it plainly exceeds the authority that could be granted to the EPA by the Federal Housekeeping Statute.
That the rulemaking would have such a significant impact on how the public interacts with the EPA on matters important to the public can also be seen in the considerable controversy it has provoked since its inception. The 2018 proposal received nearly 600,000 public comments, nearly all of which were in opposition. At a recent House Science Committee hearing on the rulemaking, none of the committee members present and none of the non-agency invited witnesses offered support for the 2018 proposal. It is also noteworthy in this regard that Congress has rejected several bills in recent years that have sought to impose limitations on the EPA's consideration of science in developing new rules that are similar to those contained in this rulemaking. It would be hard to believe that the kind of modest and mundane authorities supplied by the Federal Housekeeping Statute could provide the legal basis for a rulemaking that has engendered such widespread opposition and controversy.
Response: See responses to Comments 31-135, 40-193 and 48-408.
Comment [97-899]: Commenter (11479) writes that the supplemental notice offers a different source of authority for the proposed rule, changing from a range of statutes to the Federal Housekeeping Act. The Act reads that "The head of an Executive department or military department may prescribe regulations for the government of his department, the conduct of its employees, the distribution and performance of its business, and the custody, use, and preservation of its records, papers, and property" (emphasis added). EPA is not an "executive department" of the federal government. Even if the Federal Housekeeping Act applied here, it is meant to cover procedural rules, which the APA defines as "rules of agency organization, procedure or practice," not substantive rules, like this proposed rule, that would prevent an agency from fulfilling its statutory duties.
Response: See response to Comment 31-135.
Comment [98-942]: Commenter (11500) writes that for the reasons discussed in the Supplemental Notice, they support EPA's proposal to rely on its internal Housekeeping authority as a basis for these proposed regulations. However, they also note EPA's statement that "This internal agency procedure is intended to be consistent with the statutes that EPA administers and EPA plans to implement this procedural rulemaking in accordance with all applicable statutory and regulatory requirements." As discussed in their Comments on the proposal, the CAA requires the maximum practicable scientific transparency in NAAQS proceedings. Accordingly, while EPA's Housekeeping authority is one appropriate basis for this proposal, the agency should make it clear that other statutes such as the CAA not only permit but require the actions EPA has proposed. This is necessary to ensure that future administrations cannot repeal or modify the rule simply by changing the agency's interpretation of the Housekeeping statute.
Response: Thank you for your comment in support of the final rule. EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this rule, EPA does not rely on, interpret, or apply provisions of a particular statute or statutes that it administers. EPA will undertake such efforts in forthcoming substantive actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule.
Comment [99-951]: Commenter (11480) writes that they have been a strong defender of EPA's authority to issue public health protections. Decisions made by EPA under a variety of statutes, from the CAA to the EPCRA to the TSCA, have impacted our members. EPA decisions to reduce hazardous pollutants have reduced workplace exposures and protected the health of workers and their families. And EPA decisions have advanced manufacturing technologies in the United States to ensure that the processes and products are less polluting and more globally competitive. In some cases, their members have opposed EPA decisions due to concerns over job security. Since EPA's creation they have evaluated regulatory decision-making by taking a hard look at the science used to justify regulatory proposals under a variety of statutes.
However, we question EPA's authority to issue this proposal. The cited authority  - the federal housekeeping statute -  was originally enacted for executive departments a century prior to the modern civil service. This supplemental proposal acknowledges that EPA is not an executive department, but it claims that the agency can use this authority anyway based on the Reorganization Plan No. 3 of 1970. The use of this supposed authority is dubious for this reason and because the implementation of this proposal would go beyond the internal organization or functioning of the agency by significantly impacting the implementation of statutes that the EPA is carrying out at the direction of the United States Congress.
Response: See the response to Comment 31-135.
Comment [113-994]: Commenter (11391) writes that EPA lacks legal authority to adopt the proposed rule.
In addition to being unsupported and unjustified, the Proposed Rule also lacks any statutory basis. Administrative agencies, such as EPA, have only the power conferred on them by Congress. Agencies may, therefore, only adopt regulations where authorized to do so by Congressional statute. In the 2020 SNPRM, EPA claims that the Proposed Rule is authorized under the Federal Housekeeping Statute but, as the Agency itself recognizes, that statute only authorizes the adoption of procedural rules governing agencies' internal affairs. The Proposed Rule goes beyond this, regulating the conduct of outside parties, and thus amounts to a substantive rule, which the courts have repeatedly held cannot be adopted under the Federal Housekeeping Statute. The courts have interpreted the Federal Housekeeping Statute as a "narrow" grant of authority to agencies to deal with internal "housekeeping" matters. In Chrysler Corp. v. Brown, the Supreme Court held that the Federal Housekeeping Statute "authoriz[es] what the APA terms rules of agency organization, procedure or practice as opposed to substantive rules." The court defined "substantive rules" as those that "affect[] individual rights and obligations." Building on that definition, in American Hospital Association v. Bowen, the U.S. Court of Appeals for the District of Columbia Circuit (D.C. Circuit) identified several key features of substantive rules. According to the court, substantive rules typically establish binding standards that create rights, impose obligations, or otherwise "significantly impact" private interests, for example, by "put[ting] a stamp of [agency] approval or disapproval on a given type of behavior."
Contrary to EPA's claim, the Proposed Rule is not merely an "internal rule of agency procedure," but rather a substantive rule, which binds parties outside the Agency. In this regard, we note that the Proposed Rule is similar to EPA's 2001 directive banning agency consideration of third-party human studies ("2001 Directive"), which was held to be a substantive rule in Croplife America v. EPA ("Croplife"). In that case, the D.C. Circuit reasoned that the 2001 Directive used "clear and unequivocal language," stating that EPA "will not consider . . . any human studies in its regulatory decision-making." Based on that language, the court concluded that the 2001 Directive bound both EPA and third parties, who were "concretely injured" because they could no longer rely on human studies that had previously been accepted by the Agency. Similarly, the Proposed Rule mandates that EPA "will only use," or under the alternative proposal "will . . . give greater consideration to," studies for which the underlying data are available. As such, the Proposed Rule will prevent or severely impede third parties from relying on other studies, previously accepted by EPA. Those parties are bound by the Proposed Rule, which "put[s] a stamp of . . . disapproval" on their reliance on certain studies, thus significantly affecting their interests. 
Given the above, EPA's characterization of the Proposed Rule as non-binding is plainly incorrect and does not undo its substantive effect. EPA points to dicta from U.S. v. Manafort to argue that its characterization of the Proposed Rule is determinative. However, as the D.C. Circuit held in Croplife, "an agency's characterization of its own action is not controlling if it self-servingly disclaims any intention to create a binding rule with the force of law, but the record indicates otherwise." Here, the record makes clear that the Proposed Rule "impose[s] requirements," which are expressed in mandatory terms. The Proposed Rule thus leaves EPA and third parties with no choice in how to deal with scientific studies.
By preventing the use of studies for which the underlying data are not available, the Proposed Rule reverses long-standing EPA policy, which further indicates that it is a substantive rule not authorized under the Federal Housekeeping Statute. Indeed, in Alcarez v. Block, the U.S. Court of Appeals for the Ninth Circuit held that a key feature of substantive rules is that they "create[] new" or "change existing law or policy." EPA implies that it has not previously had a coherent, existing policy regarding the use of scientific studies, but that is not the case. For at least the last two decades, EPA has consistently relied on studies for which the underlying data are not available, and repeatedly rejected claims that it cannot or should not do so. As far back as 1997, EPA concluded that it was "impractical and unnecessary" to exclude such studies from its review and that doing so "would make it extremely difficult, if not impossible, for EPA to regulate in complex technical areas at the forefront of science." EPA reiterated that view in 2016 and affirmed that it would continue to rely on all relevant studies (i.e., regardless of the availability of the underlying data). The Proposed Rule effects a sudden and, as discussed in Part I above, unjustified reversal of that twenty-year policy.
Response: See responses to Comments 31-135, 32-144, 48-379, 48-380, 48-382 and 48-408.
Comment [114-1008]: Commenter (11392) notes that the EPA is proposing the option of using its housekeeping authority independently as authority for taking this action or in conjunction with the environmental statutory provisions cited as authority in the 2018 proposed rulemaking (as clarified in this supplemental proposal). The Agency continues to consider whether it is appropriate to rely on its authority in the above-referenced environmental statutory provisions (potentially in conjunction with its housekeeping authority). The Agency will consider comments on this issue submitted in response to the 2018 proposal and in response to this SNPRM. 
EPA does not possess the legal authority to promulgate this rule. As Members of Congress, we can affirm that Congress has never authorized the Agency to use any "housekeeping authority" in this manner. Furthermore, Congress explicitly rejected the legislative precursors to this rule in the failed Secret Science Reform Acts of 2014 and 2015 and HONEST Act of 2017. In its desperate search for a statutory basis, EPA is attempting to redefine the rule as a procedural matter rather than a change in policy. The federal housekeeping statute was never meant to be used this way, and EPA's verbal sleight-of-hand to justify its use only demonstrates the tenuous nature of the Agency's position. 
First, the plain language of the Federal Housekeeping Statute at 5 U.S.C. 301 excludes EPA. The Agency is not one of the 15 Executive Departments listed at 5 U.S.C. 101, which are the Departments explicitly authorized to act pursuant to the Housekeeping Statute. The Agency asserts that it acquired housekeeping authority under the Reorganization Plan of 1970, as determined by the OLC in a 2008 memorandum opinion. That memorandum opinion was written in response to a narrow question, "whether the [EPA] may hold its employees liable for the negligent loss, damage, or destruction of government personal property or for the unauthorized personal use of agency-issued cell phones", which is in no rational conception related to the action proposed in the SNPRM. Whether the underlying assertion is correct or not, any housekeeping authority possessed by EPA must be narrow and limited in nature, consistent with the Agency's absence from 5 U.S.C. 101 and the language of the Housekeeping Statute itself. Congress established specific criteria for the housekeeping authority in 5 U.S.C. 301, namely that the authority extends to "the government of [the] department, the conduct of its employees, the distribution and performance of its business, and the custody, use, and preservation of its records, paper, and property." This is not catch-all language: it is meant to limit the authority to internal department affairs of a procedural nature, separate from the Agency policymaking process. In other words, the Federal Housekeeping Statute does not provide any authority to enact policies, but merely to manage internal affairs such as record-retention, employee behavior, and property management. 
Despite these limitations, EPA is determined to invoke the housekeeping authority for the "Strengthening Transparency" rule because it lacks any other authority. The SNPRM characterizes the rulemaking as "a rule of agency procedure to establish an agency wide approach to handling studies when the data and models underlying EPA's significant regulatory decisions and influential scientific information are publicly available and when those data and models are not publicly available." This characterization contains critical conceptual errors. Most importantly, the phrase "handling studies" is misleading. The rule does not concern how the Agency "handles" scientific studies procedurally; it concerns how the Agency considers and uses studies in its policymaking process. Additionally, the rule would not create a procedure for an agency wide approach to scientific studies; it would create a new policy to categorically change the way certain studies are considered in the regulatory process where no such distinction existed before. EPA seeks to portray a sweeping policy change that would impact every aspect of Agency regulatory activity as a modest procedural change, akin to a change in procedure for record retention or the management of Agency property. Congress never provided any such "housekeeping authority" for this purpose. 
EPA's own prior characterizations of the rule undermine its newfound depiction as a mere internal agency procedure. In a press release announcing the proposed rule on April 24, 2018, then-Administrator Scott Pruitt declared "the era of secret science at EPA is coming to an end" because the rule would allow Americans to "assess the legitimacy of the science underpinning EPA decisions that may impact their lives." EPA stated that the proposed rule was "designed to increase transparency in the preparation, identification, and use of science in policymaking." The end of an era, the legitimacy of EPA's science, the use of science in policymaking: these are hardly matters of internal agency procedure. They are fundamental questions about the Agency's policy for the consideration of science in its regulatory process, and they must be governed by the statutory authorities that Congress has enacted for that purpose. EPA cannot claim the authority to implement this policy under the Federal Housekeeping Statute by pretending that the rule is not a sweeping change in policy.
Response: See responses to Comments 31-135, 32-144, 48-379, 48-381, 48-382 and 73-410.
Comment [115-1018]: Commenter (11393) writes that EPA lacks authority for the proposed rule and SNPRM.
AARST challenges the authority EPA claims to rely on for the proposal. The text of the SNPRM proposes to rely solely on 5 U.S.C. § 301. 85 Fed. Reg. at 15404. The preamble says EPA is taking comment on whether to use 5 U.S.C. § 301, which the SNPRM calls EPA's housekeeping authority, independently or in conjunction with statutory provisions cited in the proposed rule and corrected in the SNPRM's preamble. 85 Fed. Reg. at 15398. The fact that EPA is still considering and asking what authority it can rely on for the proposed rule, twenty-two months after the initial proposed rule, shows that EPA does not have such authority. Housekeeping does not allow any federal agency to ask the public whether it has the authority to change the goals of a given statute and thereby, through a rushed rulemaking comment period, adopt the proposed authority. The SNPRM's statements on authority continues EPA's pattern of making rules first and searching for a justification later (which it still has not provided for this rulemaking). 
First, none of the initial or corrected environmental statutes provide EPA authority for the proposed rulemaking. In most cases, EPA simply cites its general authority for rulemaking under the statutes. Still that general authority alone cannot provide a basis for the rule, especially when, as here, the rule would conflict with the requirements of each of the statutes. Courts have made clear that "EPA cannot rely on its gap-filling authority to supplement [a statute's] provisions when Congress has not left the agency a gap to fill." NRDC v. EPA, 749 F.3d 1055, 1064 (D.C. Cir. 2014); see also New York v. U.S. EPA, 413 F.3d 3, 40-42 (D.C. Cir. 2005). EPA otherwise cites to statutes describing research programs established by the agency, with no relation to or authorization for the requirements the SNPRM seeks to create. AARST also disagrees with EPA's reliance on 5 U.S.C. § 301 because 1) it is questionable whether § 301 applies to EPA as EPA is not one of the fifteen "Executive Departments" listed at 5 U.S.C. § 101, and 2) the proposed rule and SNPRM are not merely internal rules of agency procedure -- they would significantly impact every aspect of EPA's scientific and regulatory work, the individuals who contribute to that work, and the Tribes and communities who rely on it. 
Response: See responses to Comments 31-135 and 73-410.
Comment [118-1021]: Commenter (11407) writes that EPA lacks authority for the proposed rule and SNPRM.
The SNPRM changes the authority EPA claims to rely on for the proposal, however either way, the proposal remains unlawful. The text of the SNPRM proposes to rely solely on 5 U.S.C. § 301. 85 Fed. Reg. at 15404. The preamble says EPA is taking comment on whether to use 5 U.S.C. § 301, which the SNPRM calls EPA's housekeeping authority, independently or in conjunction with statutory provisions cited in the proposed rule and corrected in the SNPRM's preamble. 85 Fed. Reg. at 15398. The fact that EPA is still considering and asking what authority it can rely on for the proposed rule, twenty-two months after the initial proposed rule, shows that EPA does not have such authority. The SNPRM's statements on authority continues EPA's pattern of making rules first and searching for a justification later (which it still has not provided for this rulemaking). As the NTAA explained in its 2018 comments: In order to protect human health and environment, EPA must have a solid foundation before it proposes rules and rulemaking as such portrays the agency's naiveté in seeking answers for questions about its own authority. 
First, none of the initial or corrected environmental statutes provide EPA authority for the proposed rulemaking. In most cases, EPA simply cites its general authority for rulemaking under the statutes. Still that general authority alone cannot provide a basis for the rule, especially when, as here, the rule would conflict with the requirements of each of the statutes. Courts have made clear that "EPA cannot rely on its gap-filling authority to supplement [a statute's] provisions when Congress has not left the agency a gap to fill." NRDC v. EPA, 749 F.3d 1055, 1064 (D.C. Cir. 2014); see also New York v. U.S. EPA, 413 F.3d 3, 40-42 (D.C. Cir. 2005). EPA otherwise cites to statutes describing research programs established by the agency, with no relation to or authorization for the requirements the SNPRM seeks to create. The Ute Mountain Ute Tribe also disagrees with EPA's reliance on 5 U.S.C. § 301 because 1) it is questionable whether § 301 applies to EPA as EPA is not one of the fifteen "Executive Departments" listed at 5 U.S.C. § 101, and 2) the proposed rule and SNPRM are not merely internal rules of agency procedure --  they would significantly impact every aspect of EPA's scientific and regulatory work, the individuals who contribute to that work, and the Tribes and communities who rely on it. 
Response: See responses to Comments 31-135 and 73-410.
Comment [119-1034]: Commenter (11484) notes that EPA seeks comment on whether to use its housekeeping authority, either independently, or in conjunction with other environmental statutes, as the basis of this rule. It would be inappropriate to use housekeeping authority in this rulemaking. While it is stated in the SNPRM that "EPA is authorized to promulgate this regulation under its housekeeping authority," legal precedent cited in the same document negates this argument. The SNPRM cites the U.S. Supreme Court case Chrysler Corp v. Brown to demonstrate that EPA's housekeeping authority applies only to "`rules of agency organization, procedure or practice' as opposed to substantive rules."
A plain language reading of the proposed rule makes clear that because it governs which scientific studies EPA must consider or ignore in issuing regulations, it will affect substantive rules. However, EPA argues in the SNPRM, incorrectly, that the proposed rule "exclusively pertains to the internal practices of the EPA." If EPA's role under the proposed rule were merely to acquire, consider, and file all relevant studies, the "internal practices" argument may be correct. However, the proposed rule would require the Agency to consider some studies but not others and would give the Administrator case-by-case discretion over such consideration, potentially completely altering the outcome of future rulemakings. Thus, the proposed rule has a direct impact on substantive rules and housekeeping authority does not apply. Unless the proposed rule is revised to exclude substantive rulemakings such as setting the NAAQS, the assertion of housekeeping authority fails on its face. 
In conclusion, Puget Sound Clean Air Agency (PSCAA) opposes the proposed rule in its entirety due to the fundamental flaw that it would limit EPA's access to the best available science by relying on arbitrary and nonscientific reasons to restrict studies from consideration, a flaw that the recent changes fail to correct. Further, the most recent version of the rule proposes to make inappropriate use of housekeeping authority, rendering it legally vulnerable. If enacted, this rule would result in federal regulations with a weaker evidence basis and erode the environmental and public health protections that our Agency is charged with carrying out.
I urge you to withdraw this flawed proposal that relies on dubious authority and contravenes EPA's statutory mandate to issue regulations based on the best available science. 
Response: See responses to Comments 31-135, 32-144, 34-155, 38-185, 48-381 and 73-410.
Comment [124-1053]: Commenter (11492) writes that the Federal Housekeeping Statute does not give authority for this Rule: According to legislative history, the Federal Housekeeping Statute was never intended by Congress to authorize an agency to use for substantive regulations. EPA cannot now use § 301 of the Federal Housekeeping Statute to give itself authority to create substantive regulations, because it expressly contradicts the purpose of the code it is invoking.
Response: See response to Comment 31-135.
Comment [125-1078]: Commenter (11495) writes that EPA is not an Executive Department subject to the Federal Housekeeping Statute.
The Housekeeping Statute permits "what the [APA] terms `rules of agency organization, procedure, or practice,' as opposed to substantive rules." Specifically, the statute grants "the head of an executive department" the authority to govern internal departmental affairs. The statute allows for the promulgation of regulations "for the government" of the department, "the conduct of its employees, the distribution and performance of its business, and the custody, use and preservation of its records, papers, and property." Congress amended the statute in 1958 to add a second sentence stating, "[t]his section does not authorize withholding information from the public or limiting the availability of records to the public."
The Housekeeping Statute, by its express terms, applies only to the heads of "executive department[s] or military department[s]." EPA is not one of the fifteen executive departments listed in 5 U.S.C. § 101, nor is it a military department. The EPA's own OLC has agreed that 5 U.S.C. § 301 does not apply to the EPA, stating in one OLC opinion that "the difficulty here, however, is that section 301 confers regulatory authority only to `heads of Executive departments and military departments' and not the heads of other executive agencies, such as EPA." Congress used the term "executive department" in this statute, while in other provisions it uses the term "agency." Such differences are typically presumed intentional by the Supreme Court, meaning that "executive department" and "agency" do not refer to the same things. Therefore, the executive departments listed in 5 U.S.C. § 101 are the only government entities to which the Housekeeping Statute directly applies.
There are other executive branch agencies that have been found to be excluded from the Housekeeping Statute. For example, the OGE may not issue regulations pursuant to 5 U.S.C. § 301 because OGE is not an executive department. OGE may only issue regulations concerning the production of agency records pursuant to other authority, and regulations concerning employee testimony on official matters can be issued pursuant to the implied authority conferred on it by its own organic statute, 5 U.S.C. § 401. Similarly, no OLC opinion has found that EPA may issue any regulation pursuant to the Housekeeping Statute. EPA even admits that the Housekeeping Statute does not apply directly to EPA, instead arguing that "EPA gained housekeeping authority through the Reorganization Plan No. 3 of 1970, 84 Stat. 2086 (July 9, 1970)."
Response: See response to Comment 31-135.
Comment [125-1081]: Commenter (11495) writes that the Reorganization Plan No. 3 of 1970 does not explicitly grant Housekeeping Authority to EPA.
It is not apparent anywhere in Reorganization Plan No. 3 of 1970 (the "Plan") that Congress or the President intended the Housekeeping Statute to apply to the newly created EPA. The Plan was prepared by President Richard Nixon and transmitted to the Congress on July 9, 1970 in accordance with Chapter 9 of Title 5 of the United States Code. One of the purposes of 5 U.S.C. § 901 is "to promote the better execution of the laws." The President's accompanying message to the Congress explained that the EPA was being created to rationally and systematically organize the government's environmentally related activities, specifically related to air, water, and land. The President noted that research on pollutants and their impact on the total environment was "critical" for the measurement of the success or failure of the country's pollution abatement efforts. One of the primary functions of EPA established under the Plan is "[t]he conduct of research on the adverse effects of pollution and on methods and equipment for controlling it, the gathering of information on pollution, and the use of this information in strengthening environmental protection programs and recommending policy changes."
Section 1 of the Plan establishes the EPA's existence. Section 2 of the Plan transfers to the EPA Administrator various functions of other agencies, such as the DOI's jurisdiction over the Federal Water Quality Administration and the Federal Water Pollution Control Act. Sections 2(a)(2) - (8) transfer other various discrete functions to EPA, while retaining some environmentally related functions in the Department of Health, Education, and Welfare, the DOI, and the USDA. Section 2(a)(9) transfers "[s]o much of the functions of the transferor officers and agencies referred to in or affected by the foregoing provisions of this section as is incidental to or necessary for the performance by or under the Administrator of the functions transferred by those provisions or relates primarily to those functions."
Section 2(a)(9) is not a catch-all provision that would allow the EPA Administrator broad housekeeping authority, as EPA now appears to assert. By its express terms, Section 2(a)(9) only grants the EPA Administrator authority "incidental to or necessary for" carrying out the functions detailed in the "foregoing provisions of this section" -- that is, Section 2(a)(1) through (8). This position is supported by the fact that Section 2(a)(9) specifies some functions that fall under the authority of Section 2 and some specific functions that should be "excluded," including the functions of the Bureau of Reclamation under section 3(b)(1) of the Water Pollution Control Act. Section 2(b) of the Plan goes on to specify some functions that will be transferred under this authority, including the DOI's Water Pollution Control Advisory Board, the boards provided for by the Federal Water Pollution Control Act, and the Department of Health, Education, and Welfare's Air Quality Advisory Board. The specificity of the language used throughout Section 2 of the Plan, and particularly in Section 2(a)(9), demonstrates that the Plan was not intended to transfer every imaginable function from DOI and other agencies to the EPA.
Nowhere in the Plan is the Housekeeping Statute mentioned, nor is the transfer of housekeeping authority to EPA mentioned. As already stated, the Housekeeping Statute applies only to the heads of military departments and the heads of the 15 "executive departments" listed in 5 U.S.C. § 101. The Plan does not explicitly add EPA to this list of executive departments. EPA offers no support for its broad claim of housekeeping authority other than noting that the Administrator is the head of the EPA, and that the Plan transferred functions and authorities of "various agencies and executive departments" to EPA.
However, the transfer of discrete functions, boards, and authorities to EPA does not equate to the transfer of sweeping housekeeping authority. Section 2(a)(9) only transfers those functions "incidental to or necessary for" the performance of the previously mentioned transferred functions. The authority to promulgate rules such as the Strengthening Transparency in Regulatory Science rule -- which drastically alters the agency's process for considering scientific studies in its decision-making process -- does not follow from the grant of "incidental to" or "necessary" functions authorized under Section 2(a)(9) of the Plan.
Response: See response to Comment 31-135.
Comment [125-1084]: Commenter (11495) writes that these substantive rules are not authorized under the Housekeeping Statute. In Chrysler Corp., this meant that the confidential information given to the government in the form of trade secrets could not be released under the authority of a § 301 regulation because to do so properly would require the agency to exercise substantive legislative power. Similarly, here, the use of the notice and comment process and the nature of the Supplemental Notice -- which limits the types of scientific information considered by EPA in all standard setting, and any preliminary scientific assessments, reports, and enforcements that use externally generated peer-reviewed science as a basis for reporting, action, or decision -- indicate that the Strengthening Transparency rule is "substantive," rather than merely "procedural" or "interpretive," and therefore cannot be authorized by the Housekeeping Statute.
The very cases and authorities that EPA cites in its Supplemental Notice, among others, are distinguishable from the Proposed Rule. For example, the EPA's OLC opinion, "Authority of EPA to Hold Employees Liable for Negligent Loss, Damage, or Destruction of Government Personal Property," concluded that the EPA Administrator could set rules for employee conduct on the job as well as outside of the workplace that might undermine the "efficient operation of the Department." This scenario regarding liability for breaching personnel and property rules is clearly a matter of "internal departmental affairs." The management of employees and personal property is clearly "incidental to" and "necessary for" the effective performance of the agency. The rule did not relate to the substantive work of the EPA, like the Supplemental Notice at issue here does.
Additionally, in Boron Oil Co. v. Downie, the Fourth Circuit held that an EPA regulation under 5 U.S.C. § 301 allowing employees to resist subpoenas to testify in state court was valid. US ex rel. Touhy v. Ragen, a case oft-cited by courts interpreting § 301, the Supreme Court held that the DOJ could resist a subpoena for documents pursuant to an intra-agency rule promulgated under the authority of 5 U.S.C. § 22, the predecessor to the current Housekeeping Statute. Clearly, the scenarios in Boron Oil and Touhy relating to an agency's discretion to resist subpoenas are authorized under the housekeeping statute because they directly relate to "preservation of agency records and papers." However, just because the DOJ may resist subpoenas with the help of § 301, does not make § 301 applicable to the EPA broadly or authorize the promulgation of the Supplemental Notice specifically.
More recently, in United States v. Manafort, the District Court for the District of Columbia (D.C.) held that DOJ special counsel regulations were not "substantive rules" that created individual rights but were merely statements of internal departmental policy. Citing Chrysler Corp., the court agreed with the DOJ that the regulations were not intended to create any rights and were "matters of agency management or personnel." DOJ did not seek notice and comment for the special counsel regulations, and DOJ even used language in those regulations stating that they did not create "any rights, substantive or procedural, enforceable at law . . . by any person or entity, in any matter, civil, criminal, or administrative." This type of language has been found by courts to effectively disclaim the creation of any enforceable rights.
Clearly, here, EPA has not followed the same steps as DOJ did in the promulgation of the special counsel regulations at issue in United States v. Manafort. EPA's Supplemental Notice is distinguishable from DOJ's special counsel regulations and other procedural rules because it is being promulgated under the notice and comment process, does not use language disclaiming the creation of any rights enforceable at law, and creates a new qualitative measure for the agency's consideration and use of science in the agency's fundamental functions. Accordingly, the Supplemental Notice is a substantive rule that falls outside the scope of the Housekeeping Statute.
Response: Throughout the rule's preamble, EPA clearly states that rule does not impact the rights or obligations of outside parties. Additionally, the regulatory provisions include mandates applicable only to EPA. See responses to Comments 31-135, 32-144, 48-382 and 48-383.
Comment [125-1092]: Commenter (11495) writes that just as EPA cannot base the Supplemental Notice on the general rulemaking authority granted by environmental statutes, it also cannot justify its action on the authority, if any, granted under 5 U.S.C. § 301. First, as detailed above, the Housekeeping Statute likely does not apply to EPA. Further, even assuming that it does apply, the authority granted by the Housekeeping Statute is not broad enough to override EPA's specific environmental statutory directives. Simply put, the EPA cannot use the guise of an internal procedural rule -- which this rule clearly is not -- to contravene explicit quality of information provisions in the substantive environmental statutes the agency is responsible for administering.
The Housekeeping Statute is "simply a grant of authority to the agency to regulate its own affairs." The statute does not provide a general, independent basis for deviating from a specific statutory mandate or limiting the scope of other provisions. Where the SDWA, for example, requires EPA to consider the "best available, peer-reviewed science," EPA cannot use this rule to effectively alter the statutory text to require the best, publicly available, peer-reviewed science. Thus, even if the Proposed Rule were limited to regulation of internal agency procedures, which it is not, EPA cannot use such housekeeping as a pretext for modifying the statutory provisions requiring consideration of the best available science.
EPA must base its decisions on such criteria as the latest scientific knowledge, the best available, peer-reviewed science, and/or generally accepted scientific principles or laboratory tests. Neither the environmental statutes nor the federal Housekeeping Statute suggests that EPA, in deciding how it will handle scientific information, can reject or attribute lesser weight to scientific evidence that otherwise meets those criteria solely because the underlying data are not publicly available. EPA's Proposed Rule both lacks sufficient legal authority and unlawfully contradicts the environmental statutes that the Agency is tasked with implementing.
Response: See responses to Comments 31-135, 32-144, 34-155 and 73-410. 
Comment [130-1135]: Commenter (11884) writes that in a nutshell, there is no reasonable way the Rule or any part of the Trio could be adopted pursuant to the EPA's so called "housekeeping" powers. Good or bad, the Trio marks a dramatic "sea change" for the world of environmental regulation. The term "housekeeping" connotes trivial changes that do not require a second thought. To quote administrative judge K. Bresler: "'housekeeping' refers to minor matters". K. Bresler, Legal Housekeeping Clear Writing (2014).
In Chrysler Corp. v. Brown, 441 U.S. 281, 310 (1979), the Supreme Court emphasized that nothing in the housekeeping statute permits an agency to "withhold information from the public or to limit the availability of records to the public." Id. (quoting the housekeeping statute's legislative history). With the Trio, EPA hopes to stifle good science. To use paraphrase the Court, EPA seeks to "twist" its housekeeping authority "into a claim of authority to withhold information". Id. n. 41 at 310; cf. id. (referencing "numerous" similar comments in the legislative history).
Response: See the responses to Comments 31-135, 32-144 and 48-381.
Comment [130-1136]: Commenter (11884) writes that even if the Trio were to be mischaracterized as a few "procedural" changes, they could not be considered as within the housekeeping statute because, when aggregated, they affect the public's substantive rights. The housekeeping law does not allow an agency to curtail "substantive rights". Id. at 310.
By the same token, myriad purely procedural changes could eviscerate substantive rights. Those procedural changes are invalid. For example, EPA could adopt a procedural rule that no party can make a submission unless it first deposits a million dollars in the EPA's bank account to cover any potential administrative costs. Or EPA could bar submission of documents by anyone who has a technical degree (e.g., a B.S. or M.D.) to avoid unduly influencing agency executives. Both of these purely procedural rules would have a substantive effect that go far beyond a permissible housekeeping rule.
Response: See responses to Comments 31-135, 40-193, and 48-379.
Comment [133-1166]: Commenter (11894) writes that all courts considering the issue, starting with the Supreme Court, have held that the Housekeeping Statute cannot be used as a basis for a legislative rule. Chrysler Corp. v. Brown, 441 U.S. at 310; see also, e.g. United States ex rel. O'Keefe v. McDonnell Douglas Corp., 132 F.3d 1252, 1255 (8th Cir. 1998) (applying Chrysler Corp. and finding that the Housekeeping Statute did not provide authority for substantive regulations); Exxon Shipping Co. v. United States Dept. of Interior, 34 F.3d 774, 776-78 (9th Cir. 1994) (Housekeeping Statute did not authorize regulations allowing that agency to withhold deposition testimony of federal employees).
EPA's proposals (both the original and supplemental proposals) have every indicia of a substantive, legislative rule: codification in the CFR, adoption through notice and comment proceedings, and mandatory language of command (e.g., "the agency will only use" in proposed section 30.5), See Long Island Care at Home, Ltd. v. Coke, 551 U.S. 158, 172 - 73 (2007) (adoption through notice and comment procedures); American Mining Congress v. Mine Safety and Health Admin., 995 F. 2d 1106, 1112 (D.C. Cir. 1993) (codification in Federal Register); General Electric Co. v. EPA, 290 F. 3d 377, 383 (D.C. Cir. 2002) (binding language).
Rules that bind an agency itself are legislative rules. In CropLife Am. v. EPA, 329 F. 3d 876 (D.C. Cir. 2003), the D.C. Circuit held that an EPA press release which stated that the agency would not consider third party-controlled human exposure studies for purposes of pesticide registration was a legislative rule. Like the supplemental proposal, the press release in CropLife purported to establish exceptions to its binding language to accommodate situations in which use of a study might be legally compelled. 329 F. 3d at 881-82. The court held that the press release bound both EPA and registrants during pesticide registrations and so was a binding "substantive rule;" it "binds private parties or the agency itself with the `force of law." 329 F. 3d at 883 (emphasis supplied).
The current "housekeeping" proposal, like the press release in Croplife, would set binding rules governing how EPA uses science in its decision processes. Its hollow assurances of later action to consider studies when legally compelled to do so does not alter its status as a legislative rule any more than the similar assurances in CropLife. Since the Housekeeping Statute provides no authority for issuing legislative rules, it cannot provide a basis for either the supplemental or initial proposal. Both would bind EPA, under the conditions set out by EPA, namely that underlying data and models be publicly available (with some proposed exemptions and a grant to the Administrator of discretion) to not consider a scientific study it could have previously considered. The proposed rules would also bind the public. Furthermore, EPA's rule would require researchers interested in contributing to regulatory protections and ISI to alter their conduct or risk having their efforts deemed unusable. Ultimately, the rule would diminish the public's statutorily protected interests in regulations informed by the best available science, an outcome that EPA itself has recently contended results from substantive choices about the scientific evidence it will consider.
The Housekeeping Statute's "antecedents go back to the beginning of the Republic, when statutes were enacted to give heads of early Government departments authority to govern internal department affairs." Chrysler Corp. v. Brown, 441 U.S. 281, 309 (1979). It authorizes what the APA "terms `rules of agency organization, procedure, or practice' as opposed to substantive rules," Id. at 310, and enables "department heads to make regulations governing day-to-day operation of the department," id. n 41. A typical housekeeping rule would govern purely internal agency procedures and practices such as storage and transport of records. The bottom line is that the Housekeeping Statute authorizes only rules of agency organization, procedure, or practice -- not "legislative" or "substantive" rules. That is not what EPA is doing with this regulation. Courts have found nothing in the legislative history to support a different reading. No case law offers support for an alternative reading.
Congress took explicit action in 1958 to amend Section 301 to "correct" agencies from abusing the Housekeeping Statute by attempting to use its authority as a substantive basis to withhold information from the public. As the Supreme Court noted in Chrysler Corp., Congress had looked carefully at the statute in 1958. The Special Subcommittee on Government Information had "unanimously agreed that [§ 301] originally was adopted in 1789 to provide for the day-to-day office housekeeping in the Government departments," and attempts to construe it as something more was "misuse," which "twisted" the statute. The courts have repeatedly checked subsequent agency attempts to "twist" the statute to a range of substantive purposes and found those efforts illegal. The supplemental proposal now attempts, similarly and illegally, to twist the statute to withhold science from consideration in rulemakings that broadly affect the public interest.
There is no merit to EPA's claim that this fits into the Housekeeping authority because it addresses how the agency "handles" information. Every EPA activity during the rulemaking process, as the agency receives, evaluates, and responds to information submitted to the record, and considers it in regulatory decisions, could be said to involve "handling" information in the sense EPA is using it. Under EPA's reasoning, those would all be "housekeeping" actions, and could be governed by housekeeping rules, rather than rules promulgated under the rulemaking provisions of the APA. EPA also argues that the proposal neither regulates nor affects "any entity outside the federal government." But EPA goes on to concede that the rule governs information that may have "a substantial impact" on important "private sector decisions" and a potential impact on the public or private sector impact of more than $500 million. 85 FR at 5398. And the regulation will, as also noted, impose substantive limits on the use of science in rulemaking under the eight environmental laws identified in the proposal.
The proposal, if finalized, would make major changes in how EPA conducts rulemakings that affect major segments of the economy - compelling EPA to ignore information regardless of its relevance and probity, based only on the public availability of the underlying data, and nullifying procedural rights under the APA, which ensures "interested persons an opportunity to participate in [a] rule making through submission of written data, or arguments" and directs that the agency consider "relevant matter presented." 5 USC § Section 553(c). To then claim that authority to do this is predicated on a law intended to make sure that federal agencies make their internal trains run on time would be laughable, if it were not tragic -- and part of the tragedy is that the proposed rule would effectively deprive commenters of the right to have their information given due consideration, and would deprive agency decision-makers of research and data -- of science -- essential to good decision-making.
The COVID-19 crisis, underway at this very moment as we write and file comments, has put into stark relief what happens when science is minimized, excluded, or ignored. But the rule is not about procedural rights. The decision to preclude full consideration of relevant and reliable information in developing regulations affects important "private sector decisions" and the level of environmental protection provided to the public alike. Thus, it is sad to read EPA's earnest assurances that the proposed rule "would not regulate the conduct or determine the rights of any entity outside the federal government," an assertion that is palpably wrong. One might as well say that a restriction on the evidence that a prisoner can use to prove his or her innocence does not affect his rights. The only sense in which the rule will have no effect outside the federal government is the narrow, hyper-literal sense in which firing a gun has no non-auditory effects because it is the fired bullets that cause such effects.
Response: See responses to Comments 31-135, 31-144, 34-155, 40-193, 48-379, 48-382, 48-383.
Comment [133-1167]: Commenter (11894) notes that EPA's supplemental proposal posits that the agency could use its housekeeping authority without "exercising substantive rulemaking authority delegated to it by a particular statute or statutes that it administers" to issue these rules. 85 FR at 15398. That begs the question of where the authority for the rule comes from because, as already explained, the Housekeeping Statute provides no legal basis for the proposed rule. Moreover, the supplemental proposal completely ignores the question of how a regulation not issued under the rulemaking authority of a statute can govern agency regulatory actions under the statute. What is plain and beyond refute is that the Housekeeping Statute, even as EPA rationalizes it, provides no authority for anything more than purely internal actions, certainly not for measures that have profound impacts beyond the agency and into society, much less for rules that authorize or mandate EPA to disregard relevant and reliable information in formulating rules or in responding to public comments on those rules.
Response: See responses to Comments 31-135, 32-144, 48-381 and 73-410.
Comment [133-2208]: Commenter (11894) writes that EPA now claims authority to base its data restriction proposal not just on the provisions of specific EPA laws that we addressed in our opening comments, but also on the general federal Housekeeping Statute, 5 U.S.C. § 301. This provision, enacted by the first Congress, provides that the head of an executive department or military department may prescribe regulations for the government of his department, the conduct of his employees, the distribution and performance of its business, and the use and preservation of its records, papers, and property. As the Supreme Court has recognized, this language only authorizes rules that govern internal agency operations and cannot be used to justify a legislative rule. In an attempt to bring the proposed science restrictions within that purpose, EPA's proposal states that it does not regulate any entity outside the federal government. Rather, they claim that the proposed requirements would modify the EPA's internal procedures regarding the transparency of science underlying regulatory decisions and that this rule would not regulate the conduct or determine the rights of any entity outside the federal government. Their position is that it exclusively pertains to the internal practices of the EPA. 85 Fed. Reg. 15398.
EPA specifically states that its proposal does not interpret or rely on the specific regulatory provisions of the various EPA statutes, despite the flood of comments it received demonstrating that these were indisputably the governing statutory authorities. But in a backhanded acknowledgment of a possible problem, the proposal states that this internal agency procedure is intended to be consistent with the statutes that EPA administers and EPA plans to implement this procedural rulemaking in accordance with all applicable statutory and regulatory requirements. Nonetheless, it says that in the event the procedures outlined in the proposed rulemaking conflict with the statutes that EPA administers, or their implementing regulations, the statutes and regulations will control. Id
Response: See the responses to Comments 31-135 and 73-410.
Comment [135-1217]: Commenter (11895) writes that EPA lacks authority for the proposed rule and SNPRM. 
The SNPRM changes the authority EPA claims to rely on for the proposal, however either way, the proposal remains unlawful. The text of the SNPRM proposes to rely solely on 5 U.S.C. § 301. The preamble says EPA is taking comments on whether to use 5 U.S.C. § 301, which the SNPRM calls EPA's housekeeping authority, independently or in conjunction with statutory provisions cited in the proposed rule and corrected in the SNPRM's preamble. None of the initial or corrected environmental statutes provide EPA authority for the proposed rulemaking. 
Courts have made clear that "EPA cannot rely on its gap-filling authority to supplement [a statute's] provisions when Congress has not left the agency a gap to fill." EPA otherwise cites to statutes describing research programs established by the Agency, with no relation to or authorization for the requirements the SNPRM seeks to create. The Tribe also disagrees with EPA's reliance on 5 U.S.C. § 301 because 1) it is questionable whether § 301 applies to EPA as EPA is not one of the 15 "Executive Departments" listed at 5 U.S.C. § 101, and 2) the proposed rule and SNPRM are not merely internal rules of agency procedure -- they would significantly impact every aspect of EPA's scientific and regulatory work, the individuals who contribute to that work, and the tribes and communities who rely on it. 
Response: See the responses to Comments 31-135 and 73-410.
Comment [136-1289]: Commenter (11907) writes that EPA is using a 19th-century statute giving department heads (Administrators) the right to manage personnel and internal record keeping so that it can contain the science it uses when drafting regulations. This statute (5 USC 301/Departmental Regulations), which was passed in 1874, is referred to by the EPA as giving it "housekeeping" authority to make internal decisions regarding what data, information, and studies it wants to use. The law states:
"The head of an Executive department or military department may prescribe regulations for the government of his department, the conduct of its employees, the distribution and performance of its business, and the custody, use, and preservation of its records, papers, and property. This section does not authorize withholding information from the public or limiting the availability of records to the public."
Furthermore, upon investigation into subsequent codes and statutes relating to this law, it appears to be a precursor to laws relating to national security. This law may give EPA broad discretionary powers that could be used to eliminate the use of confidential PII based epidemiology studies regarding the effects of pollutants of human and environmental health.
EPA could claim that the information is not "transparent" and therefore cannot be reproduced or only "quasi-reproduced" in a scientific setting. Simultaneously, the EPA could determine that CBI based studies are "exempt" from the transparency rule for reasons of corporate or even national security, thereby completely providing protection and "non-transparency" to industry-based science. This appears to be precisely the intent of this supplementary rule.
Pursuant to these concerns The Skokomish Tribe strongly objects to this supplementary rule to the Proposed Rule on Strengthening Transparency in Regulatory Science. We do not agree with the use of discretionary "housekeeping authority" under 5 USC 301 to make high level internal decisions regarding what constitutes science and to choose what data, information, or studies can, or cannot be used. The Tribe holds that policy and regulatory outcomes based on this rulemaking will have harmful human health effects within our tribal community and by extension will also have harmful effects on treaty protected habitat and ecosystems within our "usual and accustomed" (U&A) treaty protected hunting, fishing and gathering grounds. This rulemaking does not support "transparency" in science but quite the reverse. It supports secrecy in science. This is in direct opposition of EPA's mission, which is to "protect human and environmental health".
Response: See responses to Comments 31-135, 32-144, 38-185, 40-193 and 48-381.
Comment [138-1262]: Commenter (11922) writes that I.EPA has broad authority under the Federal Housekeeping Statute consistent with or in addition to the authorities in the environmental statutes it administers to adopt this internal agency procedural transparency rule.
EPA specifically seeks comment as to whether the Federal Housekeeping Statute (5 U.S.C. § 301) independently or in combination with substantive statutes it administers can serve as legal authority for this transparency proposal. As the preamble states, the Federal Housekeeping Statute is a general grant of authority to heads of executive departments to prescribe regulations governing their departments. NRECA agrees with EPA's conclusion that, although EPA is not an executive department, the statute nevertheless applies to the agency and, consistent with the case law cited, that the Housekeeping Statute alone can serve as ample authority to implement a procedural transparency rule. 85 Fed. Reg. at 15,397-98.
EPA has acknowledged that, in some instances, requirements in the substantive statutes and regulations it administers may supersede a transparency rule. 85 Fed. Reg. at 15,398. Nevertheless, certain provisions within EPA's regulatory mandate support the underlying goals of this Transparency Rule. Below are highlighted provisions within the substantive statutes EPA cites (including and beyond those specific provisions EPA cites in the Supplemental's preamble) that are useful references to bolster the Transparency Rule, both in terms of reliance on housekeeping authority and in substance.
CAA 
:: Section 103, 42 U.S.C. § 7403: This provision directs EPA to establish a national research and development program. As part of this program, the Administrator is authorized to "collect and make available . . . the results of and other information . . . pertaining to such research and other activities," to "develop effective and practical processes, methods, and prototype devices for the prevention or control of air pollution," and to "develop methods and techniques necessary to identify and assess the risks to human health from both routine and accidental exposures to individual air pollutants and combinations thereof," among other activities. 42 U.S.C. § 7403(b), (d). Implicit in this authority is that the Administrator should have access to all pertinent information, and the ability to evaluate how that information is collected and analyzed. These requirements correspond directly to the proposed Transparency Rule's requirement to "describe and document any assumptions and methods used" and to "describe variability and uncertainty" with regard to EPA's use of data and models underlying pivotal science or regulatory science. See 85 Fed. Reg. at 15,406. 
:: Section 301(a), 42 U.S.C. § 7601(a): This provision authorizes the EPA Administrator to promulgate regulations "necessary to carry out his functions under this chapter," including regulations "establishing general applicable procedures and policies for regional officers and employees" designed to "assure fairness and uniformity in the criteria, procedures, and policies applied by the various regions" and "provide a mechanism for identifying and standardizing inconsistent or varying criteria, procedures, and policies . . ." This authority reinforces the general housekeeping authority upon which EPA proposes to rely by confirming that Congress authorized EPA to promulgate regulations governing its delegated functions. 
:: Section 307(d)(3), 42 U.S.C. § 7607(d)(3): This provision requires that, when proposing a rule pursuant to that subsection, EPA must summarize "the factual data on which the proposed rule is based," as well as "the methodology used in obtaining the data and in analyzing the data. . . . All data, information, and documents referred to in this paragraph on which the proposed rule relies shall be included in the docket on the date of publication of the proposed rule." As with Section 103, these requirements support the Transparency Rule's goals. See 85 Fed. Reg. at 15,406.
CWA 
:: Section 101(e), 33 U.S.C. § 1251(e): Part of Congress's purpose in enacting the Clean Water Act was to encourage "[p]ublic participation in the development, revision, and enforcement" of any regulation or program established by EPA or a State under the CWA. The Transparency Rule's goal of increasing the ability of the public to participate fully is thus directly responsive to this declaration of policy.
:: Section 104, 33 U.S.C. § 1254: This provision requires EPA to establish national research programs for pollution "prevention, reduction, and elimination," and as part of that mandate to "collect and make available, through publications and other appropriate means, the results of and other information . . . pertaining to such research." Similar to EPA's authority under the CAA, this provision encourages transparency to allow EPA to carry out the functions Congress prescribed. 
:: Section 501, 33 U.S.C. § 1361: Similar to the housekeeping statute upon which EPA proposes to rely, this provision generally authorizes EPA to promulgate regulations to effectuate the goals and requirements of the Act, reinforcing that Congress has given EPA broad authority as to the research and data EPA uses in administering the CWA.
SDWA 
:: Section 1442, 42 U.S.C. § 300j-1: This section provides EPA authority to conduct research regarding water contamination, including "improved methods to identify and measure the health effects of contaminants in drinking water . . . and assessing the health-related hazards of drinking water." Similar to the CAA and CWA, EPA is also authorized to "collect and make available" information regarding this research.
:: Section 1445(a)(1), 42 U.S.C. § 300j-4(a)(1): This provision requires regulated persons and grantees to "establish and maintain such records . . . and provide such information as the Administrator may reasonably require by regulation to assist the Administrator in establishing regulations[,] . . . in evaluating the health risks of unregulated contaminants, or in advising the public of such risks." EPA thus has the ability to determine what information it requires to best implement the SDWA, consistent with the goals of the Transparency Rule. 
:: Section 1450(a)(1), 42 U.S.C. § 300j-9(a)(1): Similar to the housekeeping statute upon which EPA proposes to rely, this provision generally authorizes EPA to promulgate regulations to effectuate the goals and requirements of the SDWA, reinforcing that Congress has given EPA broad authority as to the research and data EPA uses in administering the SDWA. 
RCRA 
:: Section 2002(a)(1), 42 U.S.C. § 6912(a)(1): This provision generally authorizes EPA to promulgate regulations to administer RCRA, similar to the housekeeping statute EPA cites as authority for the Transparency Rule, and the other general authority provisions described above. 
:: Section 7004, 42 U.S.C. § 6974(b): This provision requires EPA to provide for and encourage public participation in the development of regulations and programs under RCRA, reinforcing the goal of the Transparency Rule to facilitate such participation. 
:: Section 8001, 42 U.S.C. § 6981: Similar to other statutes EPA administers, this provision requires EPA to conduct research. For RCRA, such research includes "adverse health and welfare effects of the release into the environment of material present in solid wastes," and this provision directs EPA to establish a management program to coordinate such research activities. EPA's promulgation of the Transparency Rule is consistent with EPA's authority to coordinate and manage the research it uses to administer RCRA. 
CERCLA 
:: Section 115, 42 U.S.C. § 9615: Through this provision Congress authorizes the President to delegate duties or powers under CERCLA and to promulgate necessary regulations, again similar to other general grants of authority and the housekeeping statute.
:: Section 311, 42 U.S.C. § 9660: The Secretary of Health and Human Services, consulting with the EPA Administrator, is directed to establish a research program that includes epidemiologic studies into the human health effects of hazardous substances and methods to assess human health risk. Implicit in this mandate is the ability of the Administrator to access all pertinent information, and the ability to evaluate how that information is collected and analyzed. As with the CAA, this provision of CERCLA corresponds to the proposed requirements of the Transparency Rule regarding research assumptions and methods. 
FIFRA 
:: Section 20, 7 U.S.C. § 136r: This general research section of FIFRA directs EPA to undertake research to administer the statute, and to ensure that the research is not duplicative of research other Federal agencies are undertaking. To this end, having an accurate picture of the content, methods, and validity of existing research is critical, and aligns with the purpose of the Transparency Rule. 
:: Section 25, 7 U.S.C. § 136w: Congress authorized EPA to promulgate regulations to implement FIFRA, like other statutes. One nuance in the FIFRA general authorization is that these regulations "shall take into account the difference in concept and usage between various classes of pesticides, including public health pesticides, and differences in environmental risk and the appropriate data for evaluating such risk between agricultural, nonagricultural, and public health pesticides." More specifically than other statutes, then, this mandate recognizes the importance of data in the ability to regulate with precision and accuracy, supporting EPA's goals in proposing the Transparency Rule. Additionally, this section requires EPA to provide for peer review of the studies conducted under FIFRA, which the Transparency Rule would facilitate. 
EPCRA 
:: Section 328, 42 U.S.C. § 11048: As a statute geared toward the provision of information, instead of the gathering of information and promulgation of specific standards, EPCRA contains less specific directives regarding research, but nevertheless contains in this section the same general grant of authority EPA has under the Federal Housekeeping Statute to promulgate regulations necessary to administer the program. 
TSCA 
:: Section 2, 15 U.S.C. § 2601: Congress declared as policy of the United States to develop "adequate information . . . with respect to the effect of chemical substances and mixtures on health and the environment and that the development of such information should be the responsibility of those who manufacture and those who process such chemical substances and mixtures." It makes eminent sense that EPA should be able to require the information manufacturers and processors develop to conform to standards that allow EPA to make use of that information in promulgating regulations, as EPA intends through the Transparency Rule. 
:: Section 4, 15 U.S.C. § 2603: Congress authorized EPA to require testing and development of information regarding substances the agency determines lack sufficient information. Requests for and agreements to provide such information must include "protocols and methodologies" to be submitted to EPA. TSCA therefore explicitly recognizes the importance of not just receiving the results of studies, but also information about how they are conducted, which the Transparency Rule is intended to enhance. See also id. § 2626 (authorizing EPA and HHS to conduct and contract with others to conduct projects to develop test methods to meet the requirements of section 2603). 
:: Section 10, 15 U.S.C. § 2609: Similar to other general research-and-development provisions, in this section TSCA directs EPA to conduct research necessary to administer TSCA. Additionally, this section directs EPA, in conjunction with other agencies, to develop a system to retrieve information useful to administer TSCA. Implicit in these requirements is the discretion to determine what information is necessary, consistent with the proposed Transparency Rule. 
:: Section 26(h), 15 U.S.C. § 2625(h): Congress has directed EPA, when promulgating TSCA regulations and requiring testing and other information gathering, to "use scientific information, technical procedures, measures, methods, protocols, methodologies, or models, employed in a manner consistent with the best available science." In doing so, EPA is also to consider "the extent to which the scientific information, technical procedures, measures, methods, protocols, methodologies, or models employed to generate the information are reasonable for and consistent with the intended use of the information;" "the degree of clarity and completeness" of documentation of data, assumptions, and methods; and "the extent of independent verification or peer review of the information . . . ." Congress thus clearly contemplated, and expected, rigorous standards for data gathering and validation. TSCA's requirements confirm the ability of EPA to harness its general rulemaking authority to ensure consistency of data gathering throughout EPA's regulatory mandate. 
These statutory summaries demonstrate that Congress has both authorized and directed EPA to facilitate and conduct research throughout the range of statutes the agency administers. Both implicitly and explicitly, and in general and specific terms, these statutes reinforce the need for accurate and complete information, supporting EPA's authority to promulgate the Transparency Rule, whether under the general Federal Housekeeping Statute or EPA's more specific statutory mandates, or both. 
Response: See responses to Comments 31-135 and 73-410.
Comment [144-1307]: Commenter (11915) notes that the proposal seeks comment on whether it is appropriate to rely on EPA's authority found in a variety of environmental statutes to promulgate this rulemaking, or to also cite the Federal Housekeeping Statute independently or in conjunction with these statutes. We recommend that EPA rely on the housekeeping statute in conjunction with overarching environmental authorities. The proposal is clearly intended to institute internal agency practices for the use of data and models, and is thus appropriate for and consistent with the housekeeping statute's authorization of agencies to "prescribe regulations for the government of his department, the distribution and performance of its business, and the...use...of its records, papers, and property." This is particularly fitting if the agency chooses the modified approach that does not exclude use of non-public data and models but rather allows use of all information while providing greater consideration to publicly available materials
Response: Thank you for your comment in support of the final rule. The final rule addresses how EPA will consider the relevant scientific studies when promulgating significant regulatory decisions or developing ISI. EPA will give greater consideration to studies where the underlying data and models are available for independent validation. See responses to Comments 31-135 and 73-410.
Comment [159-1405]: Commenter (12430) notes that in the NPRM, U.S EPA attempted to justify the proposed rule on the basis of several substantive statutes. Presumably out of recognition that these statutes do not support the proposal, the agency is now attempting to frame the proposal as a mere "housekeeping" measure. It invokes the Federal Housekeeping Statute, 5 U.S.C. section 301, which states in relevant part:
The head of an Executive department or military department may prescribe regulations for the government of his department, the conduct of its employees, the distribution and performance of its business, and the custody, use, and preservation of its records, papers, and property.
As a threshold matter, the Federal Housekeeping Statute does not apply to U.S. EPA for the reasons explained in the comment submitted on the SNPRM by California Attorney General Xavier Becerra and other attorneys general. Even if the Federal Housekeeping Statute applied to U.S. EPA, it would be cabined by substantive environmental statutes, which Congress enacted after the Federal Housekeeping Statute. Most significantly, by its terms, the Federal Housekeeping Statute only authorizes (other) agencies to adopt regulations for purely internal matters, such as record retention and employee conduct. It cannot redeem the SNPRM.
Response: See responses to Comments 31-135 and 73-410.
Comment [159-1407]: Commenter (12430) notes that U.S. EPA contends that the proposed regulation should qualify as a housekeeping measure because, allegedly, it "exclusively pertains to the internal practices of the EPA[,]" and would simply "establish an agency wide approach to handling studies[.]" To date, U.S. EPA has cited the Federal Housekeeping Statute as authority for regulations covering such topics as subpoenaed employees, subpoenas duces tecum, the handling of CBI, agency practices under the FOIA and the Privacy Act of 1974, and agency-sponsored research involving human subjects. Such regulations affect how U.S. EPA employees conduct internal procedural and administrative tasks. The SNPRM, by contrast, would govern the science that U.S. EPA would be permitted to consider when implementing a range of substantive environmental laws. The SNPRM would directly limit the public's ability to provide effective comment on future agency proposals, as it would force U.S. EPA to summarily reject public comments referencing studies or models relying on non-publicly available data. As detailed throughout the remainder of this comment, the SNPRM would profoundly affect future U.S. EPA regulatory decisions and "influential scientific information" -- which, per U.S. EPA's proposed definition, is information that would have a "clear and substantial impact on important public policies or private sector decisions." The amount of attention the proposal has attracted -- from the scientific community, elected officials, the media, and nonprofit organizations -- reflects the extent of its impacts. In response to the NPRM, for example, U.S. EPA received nearly 600,000 comments and more than 9,000 unique substantive comments. Many of these were comments from scientific organizations, scientists, and public health officials explaining the dire impacts the NPRM would have on public health and the environment. A purely internal housekeeping measure would not have drawn such widespread stakeholder interest.
Response: See responses to Comments 31-135, 32-144, 40-193, and 48-381.
Comment [159-1408]: Commenter (12430) writes that even if the Federal Housekeeping Statute somehow applied (which it does not), it could not authorize the proposal set forth in SNRPM. The Federal Housekeeping Statute does not compel U.S. EPA to adopt the SNPRM or authorize it to issue regulations counter to the substantive statutes it administers. These statutes consistently require U.S. EPA to use the best available science; there is no reasonable way that the general Federal Housekeeping Statute could be read as impliedly repealing the more specific and later-enacted substantive statutes.
Response: See responses to Comments 31-135 and 73-410.
Comment [162-1467]: Commenter (12435) writes that the 5 U.S.C. 301 "housekeeping authority" does not authorize EPA to internalize decisions with high consequence for policy, such as this one. The proposed rule has broad implications for and potential impacts on external stakeholders, including specific repercussions in Colorado. The Supreme Court has held that Section 301 is a "`housekeeping statute,' authorizing what the [APA] terms `rules of agency organization, procedure, or practice' as opposed to `substantive rules.'" Chrysler Corp. v. Brown, 441 U.S. 281, 310 (1979). In Chrysler, the Court analyzed whether 5 U.S.C. 301 authorized the DOL to pass regulations limiting the scope of the Trade Secrets Act and found that it did not. Similarly, the proposed rule, with its supplemental, constitutes the type of "substantive rule" excluded in this opinion. For example, the proposed rule may dictate what kind of evidence may be used to support CWA standards, which are the heart of the substance of this federal law. See 40 CFR Part 131. Similarly, this rule would set parameters around the "best available science" required to substantiate CAA standards. The EPA is not justified in attempting to use federal housekeeping statutes to internalize decisions of this type with broad consequences in order to shelter them from scrutiny and opposition.
Response: See response to Comments 31-135 and 73-410.
Comment [166-1511]: Commenter (12454) writes that the final rule would constitute regulations "for the government" of EPA, "performance of [EPA's] business", and "use . . . of [EPA's] records, papers, and property." NACoal recommends that EPA consider the Federal Housekeeping Statute as additional support for the final rule. While EPA is not an "executive department" within the meaning of the Federal Housekeeping Statute, the DOJ concluded in a formal opinion that the EPA Administrator has the same "housekeeping" authority under EPA's organic statute.
Response: Thank you for your comment in support of the final rule. See the response to Comment 31-135.
Comment [172-1549]: Commenter (12465) writes that the authority for the rule under the provisions of the housekeeping rule is a novel interpretation of the Federal Housekeeping Statute and as such the Agency would be well served by carefully considering stakeholder input on the underlying authority and the applicability under consideration. 
Response: Thank you for your comment. In drafting the final rule, EPA carefully considered all of comments received on 2018 and 2019 proposals. 
Comment [175-1573]: Commenter (12471) writes that in establishing this procedural framework to increase transparency, the Transparency Rule can rely solely on 5 U.S.C. § 301, the "Federal Housekeeping Statute" for authority. That statute states, "The head of an Executive department or military department may prescribe regulations for the government of his department . . .[concerning] the custody, use, and preservation of its records, papers, and property." Procedural regulations to "govern[]" EPA's framework to increase the transparency of ISI and the "use" of studies in EPA's possession would fall squarely within the Agency's authority under the Federal Housekeeping Statute. For additional support, the Agency might consider referencing authorities granted to the Agency under individual environmental statutes.
[...]
The Federal Housekeeping Statute authorizes neither the broad disclosure or withholding of information by government agencies. The Supreme Court has noted that given the Federal Housekeeping Statute's "long and relatively uncontroversial history, and the terms of the statute itself, it seems to be simply a grant of authority to the agency to regulate its own affairs." The Court further determined that "there is nothing in the legislative history of § 301 to indicate it is a substantive grant of legislative power."
Rather, the Federal Housekeeping Statute authorizes "what the [APA] terms `rules of agency organization, procedure or practice' as opposed to substantive rules." A procedural framework to increase the transparency of ISI would be such a non-substantive and internal rule of agency organization, procedure or practice authorized by the Federal Housekeeping Statute.
Response: Thank you for your comment in support of the final rule. See the responses to Comments 31-135 and 73-410.
Comment [177-1594]: Commenter (12596) notes that the Supplemental Transparency Proposal relies exclusively on the Federal Housekeeping Statute, citing the DOJ opinion for "housekeeping" authority. EPA retains the option to promulgate a Final Transparency Rule using substantive authority under environmental statutes, as it proposed in the original Transparency Proposal, while citing the Federal Housekeeping Statute for supplemental support. That said, we believe that the procedural transparency assessment framework discussed herein is well-tailored for the Supplemental Transparency Proposal's exclusive reliance on the Federal Housekeeping Statute.
[...]
A Final Transparency Rule that establishes a procedural transparency assessment framework would fall well within this standard. It would, consistent with the Federal Housekeeping Statute, simply prescribe rules for the performance of EPA assessment of studies within the Agency's custody. Such a procedural approach would not rely on a substantive grant of legislative power. The Administrator may be influenced by transparency assessments resulting from the Final Transparency Rule when taking separate substantive action under the CAA. Yet the procedural framework used to develop that assessment would still remain well within the Federal Housekeeping Statute's "surely ample legislative authority."
Response: EPA agrees with the commenter that it has adequate authority to promulgate this rule. See the response to Comment 31-135. EPA has decided to rely solely on its housekeeping authority and not the substantive environmental statutes cited in the 2018 and 2020 proposals.
Comment [184-2206]: Commenter (12707) writes that the Federal Housekeeping Statute alone or in conjunction with existing bedrock environmental laws does not authorize EPA to limit science considered in rulemaking processes. 
EPA's supplemental proposal is a second failed attempt at justifying an effort to limit or otherwise constrain its own ability to use the best available public health studies and science from outside EPA. The effort in the initial proposal to ground so-called "transparency" in bedrock environmental laws was exposed as untenable. EPA's addition of the Federal Housekeeping Statute to the mix and recitation of new sections of several environmental laws does not strengthen EPA's case. Whether or not EPA has 301 housekeeping authority is irrelevant. The statute authorizes EPA to control its own internal documents or materials; the statute and its authorities are not applicable to the long-standing public health studies EPA has relied upon to protect public health through the notice and comment rulemaking process.
EPA's justification that this action to constrain science or leave science up to the political whims of the administrator is about internal agency procedure doesn't hold. This proposal is not about internal paperwork, but about whether or not EPA can fulfill its core mission consistent with bedrock environmental laws.
Response: See responses to Comments 31-135, 32-144 and 73-410.
Comment [186-1700]: Commenter (12716) writes that EPA's assertion that it is authorized to promulgate this regulation under the Federal Housekeeping Statute is without merit. The statute provides that "[t]he head of an Executive department or military department may proscribe regulations for the government of his department, the conduct of its employees, the distribution and performance of its business, and the custody, use, and preservation of its records, papers, and property." As noted by EPA in its notice, the purpose of the Section 301 was to "grant early Executive departments the authority `to govern internal departmental affairs.'" Even if we were to accept EPA's argument that it was granted authority to use the Housekeeping Statute under the Reorganization Plan, it is not authorized to justify the promulgation of substantive regulations under the statute. A change in the way that the agency fulfills its obligations and promulgates regulations under the CAA and CWA that will impact public health and the environment is substantive in nature. This is not what was intended by the Housekeeping Statute.
Response: See response to Comments 31-135 and 73-410. 
Comment [186-1845]: Commenter (12716) writes that EPA's assertion that it is authorized to promulgate this regulation under the Federal Housekeeping Statute is without merit. The statute provides that "[t]he head of an Executive department or military department may proscribe regulations for the government of his department, the conduct of its employees, the distribution and performance of its business, and the custody, use, and preservation of its records, papers, and property." As noted by EPA in its notice, the purpose of the Section 301 was to "grant early Executive departments the authority `to govern internal departmental affairs.'" Even if we were to accept EPA's argument that it was granted authority to use the Housekeeping Statute under the Reorganization Plan, it is not authorized to justify the promulgation of substantive regulations under the statute. A change in the way that the agency fulfills its obligations and promulgates regulations under the CAA and CWA that will impact public health and the environment is substantive in nature. This is not what was intended by the Housekeeping Statute.
Response: See response to Comments 13-135 and 73-410. 
Comment [187-1710]: Commenter (12720) writes, finally, and equally problematic in the Agency's quest to find authority for the rulemaking, is the assertion that authority for the Proposal may be found under the Agency's so-called "housekeeping authority." EPA first floated this claim in its notice extending the comment period for the initial rulemaking, stating that "EPA is proposing this rule under authority of 5 U.S.C. § 301, in addition to the authorities listed in the April 30th document." See 83 Fed. Reg. 24,255, 24,256 (May 25, 2018). This statute, by its own terms, does not apply to EPA. 5 U.S.C. § 301. In citing this authority, EPA implicitly admits that the statutory authorities invoked in the original Proposal are lacking and reveals that the rulemaking lacks a basis in any statute.
The Agency again cites the federal housekeeping authority as a supposed statutory basis for the rulemaking in the Supplemental Proposal. 85 Fed. Reg. 15,396, 15,397 (March 18, 2020). EPA states that Section 301 of Title 5 provides "[t]he head of an Executive department or military department" authority to "prescribe regulations for the government of his department, the conduct of its employees, the distribution and performance of its business, and the custody, use, and preservation of its records, papers, and property." Id. The Agency then quotes the Supreme Court's description of section 301 as designed to "grant early Executive Departments the authority `to govern internal department affairs'" and "authorizes `what the [APA] terms `rules of agency organization, procedure or practice, as opposed to substantive rules.'" 85 Fed. Reg. at 15,397 (citing Chrysler Corp. v. Brown, 441 U.S. 281, 308- 12 (1979)).
The fatal problem with the Agency's citation to this authority is that, under plain statutory language, EPA is "not one of the 15 `Executive Departments' listed at 5 U.S.C. § 101, to which section 301 applies. Id. 5 U.S.C. § 101. As such, the Agency has invoked a statute as a source of authority that by its own terms does not apply to EPA. Congress has had ample opportunity since 1970 to add EPA to the list of "Executive Departments" defined in 5 U.S.C. section 101. Congress has not done so. This is dispositive and fatal to the rulemaking's reliance on the Federal Housekeeping statute. See, e.g., Chevron, U.S.A., Inc. v. Nat. Res. Def. Council, Inc., 467 U.S. 837, 842 - 43 (1984)("First, always, is the question whether Congress has directly spoken to the precise question at issue. If the intent of Congress is clear, that is the end of the matter; for the court, [] as well as the agency, must give effect to the unambiguously expressed intent of Congress."); Bowen v. Georgetown Univ. Hosp., 488 U.S. 204, 208 (1988) ("It is axiomatic that an administrative agency's power to promulgate legislative regulations is limited to the authority delegated by Congress.")
Response: See response to Comment 31-135.
Comment [187-1711]: Commenter (12720) notes that the Agency acknowledges that 5 U.S.C. § 301 does not apply to EPA in the Supplemental Proposal. 85 Fed. Reg. at 15,397. However, the Supplemental Proposal then claims that EPA "gained housekeeping authority through the Reorganization Plan No. 3 of 1970." Id. The agency suggests that this housekeeping authority is sufficient to authorize the instant rulemaking. Id. This suggestion is wrong.
The Supplemental Proposal starts its inapposite line of argument by noting that "[t]he Reorganization Plan created EPA, established the Administrator as `head of the agency' and transferred functions and authorities of various agencies and Executive departments to EPA." Id. at 15,397/3. The Supplemental Proposal does not argue, nor could it, that creation of EPA or establishment of the Administrator as its head authorizes the instant rulemaking. Id. Instead, the Supplemental Proposal appears to argue that the transfer of "functions and authorities of various agencies and Executive departments to EPA" somehow authorizes the current rulemaking. Id. This is wrong.
EPA first argues that "[s]ection 2(a)(1) - (8) of the Reorganization Plan transferred to EPA functions previously vested in several agencies and executive departments including the Departments of Interior and Agriculture." Id. Notably, however, EPA does not and cannot identify any subsection(s) in section 2 that even pretend to authorize the instant rulemaking and its myriad of restrictions and exemptions and other provisions. Id. EPA ducks the issue and simply references section 2(a)(1) - (8). This is understandable. There is no congressional delegation, no legislative history, no factual evidence and no suggestion of authorization for EPA to adopt the instant rulemaking via the transfer of the listed functions from the Secretary of the Interior, the Secretary of Health, Education and Welfare, the Council of Environmental Quality, the Atomic Energy Commission, the Federal Radiation Council or the Secretary of Agriculture to the EPA. See the section 2(a)(1) - (8), Reorganization Plan No. 3 of 1970, 84 Stat. 2086 (July 9, 1970) ("Reorganization Plan"). EPA's struggle to find authority for the instant rulemaking in 2(a)(1) - (8) of the Reorganization Plan fails badly.
Next, EPA argues that "Section 2(a)(9) also transferred so much of the functions of the transferor officers and agencies `as is incidental to or necessary for the performance by or under the Administrator of the functions transferred.'" 85 Fed. Reg. at 15,297/3. Section 2(a)(9) transfers to EPA:
So much of the functions of the transferor officers and agencies referred to in or affected by the foregoing provisions of this section as is incidental to or necessary for the performance by or under the Administrator of the functions transferred by those provisions or relates primarily to those functions. The transfers to the Administrator made by this section shall be deemed to include the transfer of (1) authority, provided by law, to prescribe regulations relating primarily to the transferred functions, ....
Reorganization Plan, section 2(a)(9) (emphasis added). EPA's inability to identify any specific transferred function that authorizes the instant rulemaking is fatal. 85 Fed. Reg. at 15,297/3. EPA's tautological argument about section 2(a)(9) fails for the same reason as the argument about section 2(a)(1) - (8): the transferred functions in no way provided authority for the instant rulemaking. EPA is unable to identify any authority for the rulemaking in the actual text of the Reorganization Plan or its legislative history, or in the actual text of any of the transferred functions, their referenced statutes, or their legislative history. EPA's own silence speaks volumes about this inability.
EPA's argument, here, about section 2(a)(9) echoes the original Proposal's reliance on the general authority to prescribe regulations in section 301 of the CAA. But as NRDC noted in its comments on the Proposal:
EPA cites 42 U.S.C. § 7601(a) of the CAA as one basis for the Proposal. But that section merely authorizes the Administrator to "prescribe such regulations as are necessary to carry out his functions under this chapter." The courts have made clear that "EPA cannot rely on its gap-filling authority to supplement the CAA's provisions when Congress has not left the agency a gap to fill." NRDC v. EPA, 749 F.3d 1055, 1064 (D.C. Cir. 2014); see also American Petroleum Institute v. EPA, 52 F.3d 1113, 1119 (D.C. Cir. 1995) ("the general grant of rulemaking power to EPA cannot trump specific portions of the CAA"); NRDC v. Reilly, 976 F.2d 36, 41 (D.C. Cir. 1992) (EPA cannot use its general rulemaking authority as justification for adding to a statutorily specified list); Sierra Club v. EPA, 719 F.2d 436, 453 (D.C. Cir. 1983) (same); Gonzales v. Oregon, 546 U.S. 243, 264 - 65 (2006) ("It would go . . . against the plain language of the text to treat a delegation for the `execution' of [the Attorney General's] functions as a further delegation to define other functions well beyond the statute's specific grants of authority."). Here, not only is there no statutory gap to fill, as explained further below, the Proposal is in direct conflict with other provisions of the Act. EPA cannot rely on 42 U.S.C. § 7601(a) to support this rule.
NRDC Comments on Proposal, at 36. Federal courts have held, repeatedly and resoundingly, that congressional grants of general authority to issue regulations to carry out functions of a statute provide no additional authority to "trump" portions of a law (American Petroleum Institute, 52 F.3d at 1119); to "define other functions well beyond the statute's specific grants of authority," (Gonzales, 546 U.S. at 264 - 65); or to "supplement" statutory provisions "when Congress has not left the agency a gap to fill." NRDC, 749 F.3d at 1064. Yet, this is precisely what EPA is doing, here, by purporting to "prescribe regulations relating primarily to the transferred functions" under the Reorganization Plan. See section 2(a)(9), Reorganization Plan. Reliance on section 2(a)(9) for this authority is unlawful for the same reasons that courts have found it unlawful for EPA and other agencies to rely on congressional grants of general authority to issue regulations to carry out functions of a statute. 
Response: See response to Comments 13-135 and 73-410. 
Comment [187-1712]: Commenter (12720) writes that finding no support in the Reorganization Plan itself, EPA must resort to citing a 2008 DOJ OLC memo "opin[ing]" that the "Reorganization Plan `convey[s] to the [EPA] Administrator all of the housekeeping authority available to other department heads under section 301.'"
First, the OLC memo itself reiterates that 5 U.S.C. 301 does not apply to EPA. OLC memo, p.81. Accordingly, EPA's reliance on 5 U.S.C. 301, or any interpretations made in the OLC memo, itself, as any authority for the instant rulemaking is completely inapposite, arbitrary, and capricious. See e.g., 85 Fed. Reg. at 15,404/3 (citing 5 U.S.C. 301 as statutory authority for proposed 40 C.F.R. part 30). This defect, by itself, is fatal to the lawfulness of the entire proposed rulemaking.
Second, the interpretations offered in the OLC memo supply no authority for the instant rulemaking, and obviously lack the force of law. See, e.g., Bowen v. Georgetown Univ. Hosp., 488 U.S. 204, 208 (1988) ("It is axiomatic that an administrative agency's power to promulgate legislative regulations is limited to the authority delegated by Congress.") Congress has delegated no authority to EPA to adopt the instant rulemaking. Neither the OLC memo, nor EPA's reliance on it, will receive any deference from a reviewing court.
Third, the OLC was not reviewing or analyzing any facts or activities remotely similar to the subject of the instant rulemaking when the OLC issued the memo on which EPA now relies. Rather, the OLC was addressing the inherently internal issue of "destruction of government personal property [and] the unauthorized personal use of agency-issued cell phones," which bore no relation to an agency's performance of its substantive, statutory and regulatory responsibilities. OLC Memo, 79.
Response: See response to Comment 31-135.
Comment [187-1713]: Commenter (12720) writes that even in the narrow bureaucratic situation present there, while the OLC concluded that "[t]he Reorganization Plan establishing the EPA vests the Administrator with authority equivalent in many respects to that enjoyed by the head of an executive department" OLC memo at 82, neither the Reorganization Plan nor 5 U.S.C. 301 authorizes the instant rulemaking. Nowhere in the Supplemental Proposal can EPA identify any passage in the OLC memo that purports to authorize a rulemaking that impacts EPA's performance of its substantive, statutory responsibilities as sweepingly as the instant rulemaking would. Instead, the OLC memo fully supports the opposite conclusion, that neither 5 U.S.C. 301 nor the EPA "Reorganization Plan" supports "substantive rules" like the instant rulemaking. Footnote 4 of the OLC memo states unambiguously that:
As a general matter, regulations that "directly affect the rights and obligations of private parties" or regulate the "citizenry at large" constitute "substantive rules" under the APA and usually must be promulgated in accordance with notice and comment procedures. Authority to Prescribe Regulations, 28 Op. O.L.C. at 107. In contrast, agency rules "govern[ing] only the conduct of government employees" are not substantive rules within the meaning of the APA and are specifically excluded from publication and notice and comment requirements. Id. The EPA policies at issue here pertain solely to the conduct of EPA employees and have no application to the "citizenry at large." Those policies are therefore not "substantive rules" that must be published under the APA.
OLC memo, pg. 84, n. 4.
Faced with this fatal restriction, EPA attempts to claim that both the Proposal and the Supplemental Proposal "would not regulate the conduct or determine the rights of any entity outside the federal government" as it "exclusively pertains to the internal practices of the EPA." 85 Fed. Reg. at 15,398. Ironically, EPA offers up this preposterous assertion in the sentence immediately following a sentence "describ[ing] how EPA will handle studies when data and models underlying science that is pivotal to EPA's significant regulatory decisions or influential scientific information are or are not publicly available in a manner sufficient for independent validation and analysis." Id. Even the Agency's own description of the Proposal and Supplemental Proposal make clear that the rulemaking is intrinsically substantive in nature --  "pivotal to EPA's significant regulatory decisions or influential scientific information." Id.
The EPA claim that this rulemaking addresses only internal practices at the Agency does not even pass the laugh test. As noted extensively in the now hundreds of pages of comments submitted by NRDC alone, the rulemaking would dramatically and "directly affect the rights and obligations of private parties" and regulate the "citizenry at large" by rewriting how science is considered, used, and ignored in the Agency's substantive rulemakings. Further, the fact that EPA originally relied (without any basis), 83 Fed. Reg. at 18,769, and still relies, 85 Fed. Reg. at 15,397, on the agency's eight substantive environmental statutes, proves this point. As noted above, these statutes themselves dictate EPA's obligation to use the highest-quality scientific information to inform the agency's work, in various different ways specific to each statute. Only once it was abundantly clear that the rulemaking lacks any authority in these substantive environmental statutes, did the Agency offer a wildly different, opposing theory that pretends this rulemaking governs "only the conduct of government employees."
Courts have made abundantly clear what constitutes a substantive rule versus one of entirely internal effect under the housekeeping statute (were the housekeeping statute applicable here, which it is not). Courts have found, based on both "§ 301's text and its "long and relatively uncontroversial" history, []that § 301 "is indeed a `housekeeping statute,' authorizing what the APA terms `rules of agency organization procedure or practice' as opposed to `substantive rules.' [] Subsequent courts have repeatedly rejected agency attempts [] to "twist this simple administrative statute into an authorization for the promulgation of substantive rules." [] New York v. United States Dep't of Health & Human Servs., 414 F. Supp. 3d 475, 521 (S.D.N.Y. 2019)(internal citations omitted).
Response: See response to Comments 31-135, 48-379, 48-383 and 73-410.
Comment [198-2217]: Commenter (13549) writes that the SNPRM changes the authority EPA claims to rely on for the proposal, however either way, the proposal remains unlawful. The text of the SNPRM proposes to rely solely on 5 U.S.C. § 301. 85 Fed. Reg. at 15404. The preamble says EPA is taking comment on whether to use 5 U.S.C. § 301, which the SNPRM calls EPA's housekeeping authority, independently or in conjunction with statutory provisions cited in the proposed rule and corrected in the SNPRM's preamble. 85 Fed. Reg. at 15398. The fact that EPA is still considering and asking what authority it can rely on for the proposed rule, twenty-two months after the initial proposed rule, shows that EPA does not have such authority. The SNPRM's statements on authority continues EPA's pattern of making rules first and searching for a justification later (which it still has not provided for this rulemaking). As the NTAA explained in its 2018 comments: In order to protect human health and environment, EPA must have a solid foundation before it proposes rules and rulemaking as such portrays the agency's naiveté in seeking answers for questions about its own authority.
First, none of the initial or corrected environmental statutes provide EPA authority for the proposed rulemaking. In most cases, EPA simply cites its general authority for rulemaking under the statutes. Still that general authority alone cannot provide a basis for the rule, especially when, as here, the rule would conflict with the requirements of each of the statutes. Courts have made clear that "EPA cannot rely on its gap-filling authority to supplement [a statute's] provisions when Congress has not left the agency a gap to fill." NRDC v. EPA, 749 F.3d 1055, 1064 (D.C. Cir. 2014); see also New York v. U.S. EPA, 413 F.3d 3, 40-42 (D.C. Cir. 2005). EPA otherwise cites to statutes describing research programs established by the agency, with no relation to or authorization for the requirements the SNPRM seeks to create. The Band also disagrees with EPA's reliance on 5 U.S.C. § 301 because 1) it is questionable whether § 301 applies to EPA as EPA is not one of the fifteen "Executive Departments" listed at 5 U.S.C. § 101, and 2) the proposed rule and SNPRM are not merely internal rules of agency procedure -- they would significantly impact every aspect of EPA's scientific and regulatory work, the individuals who contribute to that work, and the Tribes and communities who rely on it.
The Band appreciates the SNPRM's recognition that "in the event the procedures outlined in the proposed rulemaking conflict with the statutes that EPA administers, or their implementing regulations, the statutes and regulations will control." 85 Fed. Reg. at 15398. This statement alone should lead EPA to withdraw the proposed rule and SNPRM; the requirements conflict with every statute EPA administers and would cause EPA to unlawfully exclude valid scientific research from its work.
By way of example, in CAA Section 108(a)(2), Congress requires EPA to issue air quality criteria for air pollutants to "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air." 42 U.S.C. § 7408(a)(2). The SNPRM does not claim that studies that have not publicly released all their underlying data, such as those linking air pollution to mortality and disease, do not accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air. If EPA were to exclude such studies from consideration when issuing air quality criteria, it would be in violation of the CAA.
Response: See responses to Comment 31-135, 32-144, and 73-410. 
Comment [199-1935]: Commenter (12403) writes that the Federal Housekeeping Statute is not an appropriate legal authority for this action. The proposal is also inconsistent with provisions in the TSCA and would hamper implementation of CAA requirements.
Response: Implementation of the final rule should not impact compliance with CAA and TSCA requirements. As stated in § 30.3, if there is a conflict between this rule and any of the statutes that EPA administers, the rule would yield to those statutes. 
Comment [200-1965]: Commenter (12412) writes that the SNPRM changes the authority EPA claims to rely on for the proposal, however either way, the proposal remains unlawful. The text of the SNPRM proposes to rely solely on 5 U.S.C. § 301. 85 Fed. Reg. at 15404. The preamble says EPA is taking comment on whether to use 5 U.S.C. § 301, which the SNPRM calls EPA's housekeeping authority, independently or in conjunction with statutory provisions cited in the proposed rule and corrected in the SNPRM's preamble. 85 Fed. Reg. at 15398. The fact that EPA is still considering and asking what authority it can rely on for the proposed rule, twenty-two months after the initial proposed rule, shows that EPA does not have such authority. The SNPRM's statements on authority continues EPA's pattern of making rules first and searching for a justification later (which it still has not provided for this rulemaking). As the NTAA explained in its 2018 comments: In order to protect human health and environment, EPA must have a solid foundation before it proposes rules and rulemaking as such portrays the agency's naiveté in seeking answers for questions about its own authority.
First, none of the initial or corrected environmental statutes provide EPA authority for the proposed rulemaking. In most cases, EPA simply cites its general authority for rulemaking under the statutes. Still that general authority alone cannot provide a basis for the rule, especially when, as here, the rule would conflict with the requirements of each of the statutes. Courts have made clear that "EPA cannot rely on its gap-filling authority to supplement [a statute's] provisions when Congress has not left the agency a gap to fill." NRDC v. EPA, 749 F.3d 1055, 1064 (D.C. Cir. 2014); see also New York v. U.S. EPA, 413 F.3d 3, 40-42 (D.C. Cir. 2005). EPA otherwise cites to statutes describing research programs established by the agency, with no relation to or authorization for the requirements the SNPRM seeks to create. LTBB disagrees with EPA's reliance on 5 U.S.C. § 301 because 1) it is questionable whether § 301 applies to EPA as EPA is not one of the fifteen "Executive Departments" listed at 5 U.S.C. § 101, and 2) the proposed rule and SNPRM are not merely internal rules of agency procedure -- they would significantly impact every aspect of EPA's scientific and regulatory work, the individuals who contribute to that work, and the Tribes and communities who rely on it.
LTBB appreciates the SNPRM's recognition that "in the event the procedures outlined in the proposed rulemaking conflict with the statutes that EPA administers, or their implementing regulations, the statutes and regulations will control." 85 Fed. Reg. at 15398. This statement alone should lead EPA to withdraw the proposed rule and SNPRM; the requirements conflict with every statute EPA administers and would cause EPA to unlawfully exclude valid scientific research from its work.
By way of example, in CAA Section 108(a)(2), Congress requires EPA to issue air quality criteria for air pollutants to "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air." 42 U.S.C. § 7408(a)(2). The SNPRM does not claim that studies that have not publicly released all their underlying data, such as those linking air pollution to mortality and disease, do not accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutants in the ambient air. If EPA were to exclude such studies from consideration when issuing air quality criteria, it would be in violation of the CAA.
Response: See response to Comments 31-135, 32-144, 48-382, 73-410. 
Comment [201-2219]: Commenter (12474) writes that the SNPRM changes the authority EPA claims to rely on for the proposal, however either way, the proposal remains unlawful. The text of the SNPRM proposes to rely solely on 5 U.S.C. § 301. 85 Fed. Reg. at 15404. The preamble says EPA is taking comment on whether to use 5 U.S.C. § 301, which the SNPRM calls EPA's housekeeping authority, independently or in conjunction with statutory provisions cited in the proposed rule and corrected in the SNPRM's preamble. 85 Fed. Reg. at 15398. The fact that EPA is still considering and asking what authority it can rely on for the proposed rule, twenty-two months after the initial proposed rule, shows that EPA does not have such authority. The SNPRM's statements on authority continues EPA's pattern of making rules first and searching for a justification later (which it still has not provided for this rulemaking). As the NTAA explained in its 2018 comments: In order to protect human health and environment, EPA must have a solid foundation before it proposes rules and rulemaking as such portrays the agency's naiveté in seeking answers for questions about its own authority.
First, none of the initial or corrected environmental statutes provide EPA authority for the proposed rulemaking. In most cases, EPA simply cites its general authority for rulemaking under the statutes. Still that general authority alone cannot provide a basis for the rule, especially when, as here, the rule would conflict with the requirements of each of the statutes. Courts have made clear that "EPA cannot rely on its gap-filling authority to supplement [a statute's] provisions when Congress has not left the agency a gap to fill." NRDC v. EPA, 749 F.3d 1055, 1064 (D.C. Cir. 2014); see also New York v. U.S. EPA, 413 F.3d 3, 40-42 (D.C. Cir. 2005). EPA otherwise cites to statutes describing research programs established by the agency, with no relation to or authorization for the requirements the SNPRM seeks to create.
The NTAA also disagrees with EPA's reliance on 5 U.S.C. § 301 as:
1) It is questionable whether § 301 applies to EPA as EPA is not one of the fifteen "Executive Departments" listed at 5 U.S.C. § 101, and
2) The proposed rule and SNPRM are not merely internal rules of agency procedure --  they would significantly impact every aspect of EPA's scientific and regulatory work, the individuals who contribute to that work, and the Tribes and communities who rely on it.
The NTAA appreciates the SNPRM's recognition that "in the event the procedures outlined in the proposed rulemaking conflict with the statutes that EPA administers, or their implementing regulations, the statutes and regulations will control." 85 Fed. Reg. at 15398. This statement alone should lead EPA to withdraw the proposed rule and SNPRM; the requirements conflict with every statute EPA administers and would cause EPA to unlawfully exclude valid scientific research from its work.
By way of example, in CAA Section 108(a)(2), Congress requires EPA to issue air quality criteria for air pollutants to "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air." 42 U.S.C. § 7408(a)(2). The SNPRM does not claim that studies that have not publicly released all their underlying data, such as those linking air pollution to mortality and disease, do not accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air. If EPA were to exclude such studies from consideration when issuing air quality criteria, it would be in violation of the CAA.
Response: See responses to Comments 31-135, 32-144, 38-185 and 73-410. 
Comment [205-2035]: Commenter (13788) writes that as EPA admits in the SNPRM, the Agency "is not one of the 15 `Executive Departments' listed at 5 U.S.C. 101." Therefore, on its face the Federal Housekeeping Statute does not even apply to EPA. EPA claims authority based on Reorganization Plan No. 3 of 1970, which transferred functions previously vested in other agencies to EPA, as well as functions "incidental to or necessary for the performance by or under the Administrator of the functions transferred." However, EPA provides no reason to believe this rule is incidental or necessary for performance of the functions transferred. 
Response: The purpose of the rule is to increase transparency of the science EPA uses to support its significant regulatory decisions and ISI. Transparency of the underlying data, models, and process upon which the EPA makes its decisions is integral to demonstrating accountability and our dedication to public health and the environment. Also see the response to Comment 31-135.
Comment [207-2071]: Commenter (14397) notes that in the SNPRM, the EPA seeks comments on whether to use its housekeeping authority independently or in conjunction with "appropriate environmental statutory provisions." EPA proposes using the Federal Housekeeping Statute, found at 5 U.S.C. § 301 (see 85 Fed Reg at 15404), which states that "[t]he Head of an Executive department or military department may prescribe regulations for the government of his department, the conduct of its employees, the distribution and performance of its business, and the custody, use, and preservation of its records, paper, and property." It appears that the EPA itself does not believe that it has this authority, otherwise why would they still be seeking comments on this aspect of the proposal two years after it was first proposed. And EPA should well doubt the use of this provision in this instance - since the EPA was not one of the fifteen "Executive Departments" listed under 5 U.S.C. § 301 it is doubtful whether this provision is applicable. Additionally, the proposed rule and the SNPRM both go well beyond internal housekeeping provisions - they would dramatically impact every aspect of the EPA's regulatory responsibilities. Use of this provision would unnecessarily prevent the EPA from carrying out its mission under the CAA, as the EPA itself seems to recognize when it states "in the event the procedures outlined in the proposed rulemaking conflict with the statutes that EPA administers, or their implementing regulations, the statutes and regulations will control." (see 85 Fed Reg at 15398).
To illustrate the above argument, the CAA (Section 108(a)(2)) requires the EPA to set pollutant levels that "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air." 42 U.S.C. § 7408(a)(2). The EPA, in this SNPRM, does not attempt to argue that the studies under attack for not providing all underlying data don't accurately reflect the latest scientific knowledge. Therefore, if the EPA elects to exclude this data, it would be in clear violation of the CAA.
Response: See the response to Comments 31-135 and 73-410.
1.3 Specific Environmental Statues 
1.3.1 Clean Air Act (CAA) (NPRM and SNPRM Comments).
Comment: Sections 1.3.1.1 and 1.3.1.2 of this Chapter present comments received on the NPRM (Section 1.3.1.1) and SNPRM (Section 1.3.1.2) related to the EPA's legal authority for the rule under the CAA. The EPA provides the following general response to these comments:
Response: The EPA appreciates your comments. As discussed in Section I.C of the final rule preamble and in this RTC document in Section 1.2, EPA has decided it is most appropriate to rely on its authority to promulgate housekeeping regulations, rather than any of the substantive environmental statutes cited in the 2018 proposal or the 2020 supplemental proposal as legal authority for the rule. EPA intends to promulgate future science transparency rules under its substantive environmental statutes. Additionally, the final rule states in § 30.3 that "[i]n the event the procedures outlined in this rule conflict with statutes that EPA administers, or their implementing regulations, the statutes and regulations will control."
1.3.1.1 CAA (NPRM Comments)
Comment [37-36]: Commenter (0046) states that the proposal amends the air quality criteria, adopted under §108 (b) of the CAA, for particulate matter (PM) and lead. See proposed rule fn. 3 final sentence which states that the proposed rule would "preclude" "future regulatory actions" using the Lanphear study which is part of the criteria for the NAAQS for lead, and the Harvard Six Cities and American Cancer Society II study (Dockery and Pope) which are part of the criteria for the PM NAAQS. Air quality criteria cannot be amended without review by Clean Air Scientific Advisory Committee (CASAC). See CAA §109 (d)(2)(B). EPA consequently must submit its proposal to CASAC for its review, following all procedural requirements for public meeting and deliberations in doing so. CASAC must then submit its recommendations to the Administrator (see §109 (d)(2)(B) final clause), and the Administrator must consider these recommendations and provide a reasonable explanation for any actions that deviate significantly from those recommendations (CAA section 307 (d)(3)). EPA cannot proceed with this action until these requirements are satisfied.
Comment [657-4043] Commenter (0671) states that the impact of the proposed rule (and its obvious purpose) will be to hamstring the EPA by effectively preventing EPA from using human subject studies in rulemaking. No other federal agency has such a broad standard as the one envisioned here. The EPA knows that the IRB will not allow such studies' data to be made public, so to require public data from these studies will make them invisible to the rulemaking process. Therefore, by requiring public data the EPA will be violating the CAA's requirement to use the "best available science" by forcing it to ignore such science that happens to include PII.
The proposed regulation is a terrible idea that is contrary to the intent of the CAA and its amendments, imposes substantial administrative burdens on researchers and potentially the American people, and does not serves its stated purpose.
Comment [657-4198]: Commenter (0671) The EPA should not implement the proposed regulation. No part of the Agency should be responsible for implementing this proposed regulation because it runs counter to the text and intent of the CAA and amendments.
Comment [1535-4532]: Commenter (1550) states that, as a practicing scientist they recognize the difference between the need for supporting data, and requirements that are imposed solely to obstruct or censor evidence that an administrator finds inconvenient or which contradicts his preconceived positions.
Consider, for example, air pollution. The CAA is a bedrock environmental law that protects us from dangerous air pollutants. It would be endangered under this proposed rule because it relies on a longitudinal epidemiologic study of subjects whose participation involved a promise that the data would be used statistically without disclosing individual medical records. If the proposed rule is approved, we could lose the use of such statistical evidence and be unable to formulate valid science-based standards. 
The same principles apply to many other areas of environmental health research. Once again, there is a fundamental difference between disclosure necessary to validate data and disclosure rules created solely to obstruct or censor results one dislikes. Doing so would violate the principles and traditions of science and be counter to the mission of the EPA.
Comment [1544-4546]: Commenter (1559) notes that the CAA protects us from dangerous air pollutants. We must keep this law. This proposed rule compromises the CAA, and for that reason, this rule must not be approved. We need to maintain the CAA sweeping improvements to the air we breathe, and we need to strengthen it to include all air toxics that are endocrine disruptors, mutagens, and carcinogens. We need federal regulation of the halogens---chlorine, bromine, and fluorine. We need federal regulation of the petroleum products that go airborne such as petroleum-based fragrances, pesticides, insecticides, herbicides, and airborne plastic molecule--all of which come from petroleum bases which are endocrine disruptors, mutagens, and carcinogens. It is time to ban synthetic, petroleum-based fragrances as 30% of the population has some discomfort from them, and this particularly applies to petroleum-based fragrances in dryer sheets that make it very difficult to breathe in local neighbors. We need to ban all synthetic fragrances and require substitution with 100% pure essential plant or fish oils.
Comment [1642-4554]: Commenter (1657) states that the CAA only requires EPA to use the "best available science", it doesn't not require that the EPA use only publicly available data, nor does it specify that the science on which EPA rules are based must be from the United States. Indeed, after May 25, 2018 any self-imposed requirement that the EPA use only public data on human subjects will prevent EPA from using the best available science when that science comes European Union (EU) member countries.
While the EPA has no control over statutory requirements in other jurisdictions the EPA does have a statutory obligation to obey U.S. law, and it will likely not be able to obey the CAA under the proposed rule after May 25, 2018.
Comment [1959-109]: Commenter (1973) expresses that they believe that the proposed rule would deprive policy makers of the real-world epidemiologic evidence, based on real exposures of real people, that has been and will continue to be vital for future revisions of the NAAQS, drinking water standard, pesticide standards, and other health based standards based on a false understanding of transparency and replication in science. This is direct contradiction to requirements of the CAA, for example, to consider the best available science, and would create an approach to the use of scientific evidence on the adverse health effects of air pollution by EPA that would be at odds with the World Health Organization (WHO), the Royal Society for Medicine, and other national review bodies. It would also put EPA in direct opposition to the standards for considering evidence of health effects of smoking, high cholesterol, etc. by the CDC, of the studies considered for issuing guidelines generated by the NIH, and of the Food and Drug Administration (FDA) reliance on non-publicly available studies to approve drugs for use by people.
Comment [2322-160]: Commenter (2337) notes that the EPA's core mission, reflected in numerous environmental statutes, is to protect public health and the environment. These same statutes require that EPA, in fulfilling this mission, support and rely on the best available science. While the specific terms of each statute vary, their direction is similar: EPA is charged with conducting and supporting research into the impacts of pollutants on human health and the environment, using available scientifically accepted data to determine when pollutants pose a danger to human health or the environment, and establishing standards with a sufficient margin of safety to protect public health. Indeed, this requirement that EPA use the latest or best available science is repeated in numerous environmental statutes. The CAA, for example, directs the Administrator to reflect "the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare ..." when updating air quality criteria. Notably, public availability or independent verification are not disqualifying requirements.
Comment [2480-4920]: Commenter (2495) states that the EPA must use all the available scientific information in order to meet the goals Congress identified in the CAA. This is the standard met by the ISA for Ozone and Related Photochemical Oxidants, to cite one example. The agency should also follow the advice of CASAC. 
Comment [4816-4226]: Commenter (4827) states that, under the CAA, the EPA is required to establish and regularly update federal standards for hazardous air pollutants (HAP), including asthma-causing PM and ozone. These standards, the NAAQS, form the backbone of our nation's air quality protections. Although the NAAQS did not prevent my sister's asthma, they have and continue to bring about substantial improvement in our nation's air quality since their first formulation.
But EPA's proposed rule would have excluded peer-reviewed studies that form the scientific basis of the NAAQS. For example, peer-reviewed studies would be excluded because the underlying data and models cannot be disclosed even in partial form. In fact, the Standards would likely not have been issued had the proposed rule been in place when they were first enacted in the 1970s, because EPA would have tossed out the underlying studies, tying its hands from taking action on an imminent public health concern. Without a doubt, many more Southern Californians would have their lives altered, or even cut short, by dangerous levels of air pollution.
If adopted, the proposed rule will deprive EPA policymakers from real-world evidence and studies that are vital to EPA's review of the NAAQs into the future. Further, the proposal directly contravenes the comprehensive federal and state regulatory programs Congress envisioned when drafting the CAA of 1970. It reduces our public health legislation to mere declarations, as EPA would be severely delayed, if not rendered entirely unable, to establish future Standards using the "best available science." 
Comment [4827-2070]: Commenter (4838) asserts that the proposed rule violates several environmental statutes because it hinders EPA's ability to rely on best available science, or the most up-to-date information, as they require. The CAA, CWA, SDWA, TSCA, and EPCRA all require certain decisions or regulatory criteria be based on the most up-to-date science. These criteria are described as "best available science," "latest scientific knowledge," and "best available public health information." The proposed rule would illegally limit EPA's ability to rely on best available science in violation of these statutes.
Comment [4829-2023]: Commenter (4840) states that many of laws authorizing the activities of the EPA require the agency to use the best science in protecting the public's health and environment. For example, the CAA mandates that air quality criteria "accurately reflect" the "latest scientific knowledge." In the past, the EPA has considered all available studies in issuing these criteria without consideration of the availability of the underlying data. Promulgation of this proposed rule would be an arbitrary and capricious violation of the provisions of these laws.
Comment [4867-2034]: Commenter (4878) notes that the CAA authorizes EPA to establish NAAQS to protect public health and to regulate emissions of HAPs. EPA works with local governments to reduce air pollution and uses scientific studies that could be impacted by the proposed rule to revise its national air quality standards and NAAQS on a regular basis.
Very little is currently known about air pollution and its impacts on the brain. Recent studies have linked particulate exposures to Parkinson's disease including a large study done in Denmark. This study used several thousand people with and without a current diagnosis of Parkinson's disease. Using extremely specific (within 5-50 meters of the front door) geo-coding to estimate participant's exposure to contaminants, the study estimated that ambient air pollution from traffic increased risk of developing Parkinson's disease by 9 percent. Researchers found an increased risk of Parkinson's disease after exposure to PM in studies from Taiwan and South Korea, as well. 
In addition to concerns stated in the previous section about the usefulness of data if enough information is redacted to protect privacy, these studies raise additional challenges because they were performed internationally. In the Danish study, participants are protected by EU law. Going forward, an EU study's compliance with the proposed rule will need to be reconciled with the new General Data Protection Regulation, which is seen as more restrictive than the United States' Health Insurance Portability and Accountability Act (HIPAA) of 1996.The privacy directive raises many issues for science not discussed here, but it is clear that many studies that involve people located in the EU will have a difficult time both complying with the new directive and providing enough information to EPA to be considered.
Studies coming from other countries are vital to health determinations in the Unites States because people in other countries are exposed to chemicals at different rates than in the U.S. The ability to compare and contrast exposure to health outcome can be enlightening; average PM concentrations in South Korea and China are several times higher than in the United States, making relatively subtle effects stand out more easily. Studies done in other countries can also help researchers tease out whether an effect is dose or length of exposure dependent. The inability to review and use international research in determinations will virtually guarantee EPA is missing major findings and important data. 
Comment [4870-2300]: Commenter (4881) states that, in dismissing the last-resort claim of data inaccessibility in the case of American Trucking Associations, Inc. v. EPA, the Court pointed out that the CAA imposes no obligation on the EPA to obtain and publicize the raw data underlying published studies on which the Agency relies. The Court distinguished EPA's reliance on a study's results from its reliance on the raw data underlying such results, noting that raw data often are unavailable due to proprietary interests of a study's scientific investigators or confidentiality agreements with study participants. The Court endorsed EPA's own claim submitted in that case:
"If EPA and other governmental agencies could not rely on published studies without conducting an independent analysis of the enormous volume of raw data underlying them, then much plainly relevant scientific information would become unavailable to EPA for use in setting standards to protect public health and the environment․ [S]uch data are often the property of scientific investigators and are often not readily available because of ․ proprietary interests ․ or because of [confidentiality] arrangements [with study participants]."
Comment [4873-2109]: Commenter (4884) expresses that the CAA requires that NAAQS be set at levels protecting the public health without regard to cost. I believe our other federal environmental laws have the same legal requirement. Footnotes numbered one through five at pg. 18769 of the subject Federal Register frustrate the CAA legal mandate to set NAAQS at levels sufficient to protect public health. In sum, these citations call for the best available science as long as it is publicly available. This truncated scientific information now deemed "transparent" will further be vetted by advisory boards; some likely representing other vested interests. What policy recommendations these conflicted councils may make are then to be ruled upon and perhaps over-ruled by the EPA Administrator. The Administrator can discount the value of clean air to breathe without fear of contradiction.
It is the avowed intent of this proposal to make these changes only for future EPA regulations adopted under the provisions of 40 CFR Part 30. Such limitations are meaningless under the provisions of the CAA. This law calls for the NAAQS to be reestablished every five years at the latest.
Comment [5154-393]: Commenter (5165) states that the National Association of Clean Air Agencies (NACAA) appreciates this opportunity to comment on the U.S. EPA's proposed rule, "Strengthening Transparency in Regulatory Science," 83 Fed. Reg. 18,768 (Apr. 30, 2018). NACAA is the national, non-partisan, non-profit association of 156 local and state air pollution control agencies in 41 states, D.C. and four territories. The air quality professionals in our member agencies have vast experience dedicated to improving air quality in the U.S. These comments are based upon that experience. The views expressed in these comments do not represent the positions of every state and local air pollution control agency in the country.
The commenter agrees with EPA that "the best available science must serve as the foundation of EPA's regulatory actions." Indeed, reliance on best available science is a fundamental requirement of the CAA and other environmental statutes that EPA administers. For example, the CAA requires EPA to establish NAAQS at levels "requisite to protect the public health" with "an adequate margin of safety." In meeting this obligation, EPA is required to develop air quality criteria that "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air, in varying quantities." Science-based decision making is at the very core of our shared mission to protect public health and the environment from the harmful effects of air pollution
Comment [5865-327]: Commenter (6111) states that the EPA's historic position is consistent with the Statutes. For example, one of EPA's core duties under the CAA is to set and periodically review the NAAQS for six common air pollutants. In carrying out this responsibility, Congress commanded EPA to use "the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects [of air pollution] on public health or welfare." 
Furthermore, because EPA is required under the Statutes to assess the public health benefits of its regulations, it must take into account all relevant science and cannot arbitrarily exclude certain studies demonstrating those benefits. Under the CAA, EPA must set the NAAQS at a level "requisite to protect the public health." 
Many of the fundamental public health studies on which EPA has based key rules and standards under the Statutes are studies for which the raw data were not or could not have been released. Attachment 1 to this letter contains a partial list of studies [note: included under a separate excerpt] that likely contain confidential data; these are all studies on which EPA has relied and cited as the basis for its actions under some of the Statutes. Until now, release of the underlying raw data was not an EPA criterion for determining the "best available" reports, studies, analyses, or models. Indeed, none of the Statutes invoked by EPA as support for the proposed rule limits EPA in this fashion; none of the Statutes requires EPA to make raw data publicly available.
EPA's proposed new approach, which conflicts with the agency's obligations and curtails its authority, is irrational at best and detrimental to public health and safety at worst.
Comment [5971-4586]: Commenter (5981) notes that the CAA allows the EPA via Risk Management to improve the safety protocols followed by first responders and workers at facilities which use hazardous chemicals. 
The EPA historically investigates such emergency incidences involving death or injury to study causes and create Risk Management. Because these incidences and subsequent studies cannot be reproduced ethically, first responders and workers at hazardous facilities would be at greater risk. This would also negatively affect the public living near these facilities. 
Comment [6099-564]: Commenter (6109) notes that the EPA cites two provisions of the CAA as legal support for its proposal: § 103 - which details research and development programs and lists authorized activities, and § 301(a) - which gives EPA authority to "prescribe such regulations as are necessary to carry out its functions under this chapter." In citing § 103, EPA fails to acknowledge that the list of authorized activities does not include arbitrarily rejecting best available science, and in referencing § 301(a) EPA does not cite judicial authority or agency precedent for using this provision to limit the evidence that can be used in rulemaking.
Comment [6099-565]: Commenter (6109] states that, interestingly, EPA does not cite § 307(d) of the CAA as a basis of legal authority for this proposed rule. Subsection (d) ("Rulemaking") refers specifically to data disclosure (§ 307(d)(3)(A)). This may be because, as EPA is aware, courts have already rejected claims that § 307 prohibits the use of nonpublic data in decision making. In American Trucking Associations, Inc. v. EPA, 283 F.3d 355, 372 (D.C. Cir. 2002), the court said, "we agree with EPA that requiring agencies to obtain and publicize the data underlying all studies on which they rely "would be impractical and unnecessary." In doing so, the D.C. Circuit Court directly cited EPA's own argument: 'If EPA and other governmental agencies could not rely on published studies without conducting an independent analysis of the enormous volume of raw data underlying them, then much plainly relevant scientific information would become unavailable to EPA for use in setting standards to protect public health and the environment... [S]uch data are often the property of scientific investigators and are often not readily available because of ... proprietary interests ... or because of [confidentiality] arrangements [with study participants].'
EPA is seeking to rescind its previous position on the use of nonpublic data without providing an explanation of how, in the absence of these data, EPA will continue to adequately develop or implement regulations that are protective of human health.
EPA also fails to acknowledge that the CAA contains provisions that contradict this proposed rule. Section 109 of the CAA requires EPA to base air quality standards on "air quality criteria," which, per (§108(a)(2)) must "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health and welfare." Section 108 does not include a provision for ignoring existing data. In addition, § 109 requires that NAAQS must allow "an adequate margin of safety" and "protect public health." If EPA chooses to ignore relevant science, how will the agency know if it is setting standards with an adequate margin of safety?
Comment [6104-612]: Commenter (6114) states that the EPA itself seems to question whether it has the authority to promulgate this rule. The FR notice asks commenters to provide information on what areas of the CAA they feel may demonstrate this type of authority. The EPA should have a solid foundation before it proposes rules and not have to seek answers for questions about its own authority.
Comment [6112-1491]: Commenter (6137) notes that the EPA cites two provisions of the CAA as authority for the Proposed Rule. As further detailed below, neither provision provides such authority. 
First, EPA cites CAA § 103, 42 U.S.C. § 7403, as authority for the Proposed Rule. 83 Fed. Reg. at 18,769. However, this section provides no authority for such a rule. Instead, the section requires EPA to create a research and development program for prevention and control of air pollution and to conduct research on health effects, among other issues. Specifically:
 Section 7403(a) requires EPA to "establish a national research and development program for the prevention and control of air pollution," which includes funding or conducting studies, establishing technical advisory committees, and related activities.
 Section 7403(b) provides a list of specific "[a]uthorized activities" that EPA may take in establishing the "research and development program" under subsection (a), including, for example, collecting and making available information pertaining to the program, cooperating with other agencies, and making grants and contracts for research. Id. § 7403(b)(6).
 Section 7403(c) requires EPA to "conduct a program of research, testing, and development of methods for sampling, measurement, monitoring, analysis, and modeling of air pollutants."
 Section 7403(d) requires EPA to "conduct a research program on the short-term and long-term effects of air pollutants, including wood smoke, on human health."
 Section 7403(e) requires EPA to conduct ecosystem research.
 Section 7403(f) requires EPA to oversee an "experimental and analytical research effort, with the experimental research to be carried out at the Liquefied Gaseous Fuels Spill Test Facility."
 Section 7403(g) requires that, in carrying out purpose of subsection (a), EPA shall "conduct a basic engineering research and technology program to develop, evaluate, and demonstrate nonregulatory strategies and technologies for air pollution prevention."
 Section 7403(h) authorizes certain research by the National Institute of Environmental Health Sciences (NIEHS).
 Section 7403(i) discusses coordination of research with "other Federal ecological and air pollution research efforts."
 Section 7403(j) discusses acid rain research.
 Section 7403(k) discusses air pollution conferences.
Notably absent from this long list of explicit requirements and responsibilities is rulemaking authority, much less authority to exclude scientific studies from consideration by EPA in any "regulatory decisions" for any reason, including whether or not the underlying data is, or can be made publicly available. See 83 Fed. Reg. at 18,773-74 (proposed 40 C.F.R. §§ 30.2, 30.3, 30.5) (indicating application of Proposed Rule only to use of studies and data in "significant regulatory decisions"). Indeed, the authorized activities included in Section 7403(b) are quite specific, including actions such as collecting and disseminating information, making grants and contracts, and even "construct[ing] facilities, provid[ing] equipment, and employ[ing] staff as necessary to carry out this chapter." 
Congress knew how to authorize EPA rulemaking activities elsewhere in the CAA. That § 7403 does not include such authority, much less authority to restrict science in particular, shows a clear intent not to grant such authority. Meghrig v. KFC W., Inc., 516 U.S. 479, 485 (1996). Rather, the purpose of § 7403 is plainly to promote research and to advance and increase the use and consideration of data, not to restrict it.
Equally problematic, EPA's proposed action serves none of the goals and meets none of the requirements of § 7403. The Proposed Rule is not a "research and development program" and does not include the requisite components of such a program necessary for EPA to act pursuant to its authority under this provision. EPA is not proposing any grants or research fellowships, see § 7403(b), or any air pollutant monitoring, analysis, modeling, and inventory research, see § 7403(c), or any "basic engineering research and technology program to develop, evaluate, and demonstrate nonregulatory strategies and technologies for air pollution prevention," see § 7403(g). Nor is EPA proposing to conduct any epidemiological studies on air pollution or to develop any methods or techniques for human health risk assessment, see § 7403(d). 
In addition, even if EPA could otherwise act pursuant to this provision, EPA may not develop health risk assessment methods and techniques applicable to air pollutants without including the following requisite statutory elements: 
 The creation of an Interagency Task Force, id. § 7403(d)(2)(A);
 An evaluation of each of the listed HAP under § 7412(b)(1) "based on reasonably anticipated toxicity to humans and exposure factors" listed therein, and which "shall be reviewed by the Interagency Task Force," id. § 7403(d)(2)(B);
 Preparation of environmental health assessments for each of the HAP, with specific deadlines, that "shall be prepared in accordance with guidelines developed by the Administrator in consultation with the Interagency Task Force and the SAB," including a specific list of scientific elements that includes "available toxicological and epidemiological information," "a determination of gaps in available information," and "where appropriate, an identification of additional activities . . . needed to identify the types or levels of exposure which may present significant risk of adverse health effects in humans." Id. § 7403(d)(2)(C). 
EPA's Proposal does not include any, much less each, of these required components for an exercise of § 7403 authority. Thus, even if EPA otherwise had authority to act pursuant to this provision, the Proposed Rule is inconsistent with and contravenes the very provision which EPA itself cites.
For each and all of these reasons, § 7403 does not give EPA authority to regulate science or to exclude from EPA regulatory decisions the consideration of a subset of scientific studies.
Second, EPA cites section 301(a) of the CAA, 42 U.S.C. § 7601(a), as authority for the Proposed Rule. 83 Fed. Reg. at 18,769. This section of the CAA authorizes only "such regulations as are necessary to carry out [the Administrator's] functions under [the CAA]." 42 U.S.C. § 7601(a)(1). As discussed below, this provision does not authorize the Proposed Rule.
It is beyond cavil that general rulemaking provisions do not "`provide [EPA] Carte blanche authority to promulgate any rules, on any matter relating to the CAA, in any manner that the [EPA] wishes.'" North Carolina v. EPA 531 F.3d 896, 922 (D.C. Cir. 2008), on reh'g in part, 550 F.3d 1176 (D.C. Cir. 2008) (citing Citizens to Save Spencer County v. EPA, 600 F.2d 844, 873 (D.C. Cir. 1979)). Rather, such regulations must be "necessary" to carry out another statutory duty. See, e.g., id. ("EPA cannot claim retiring excess Title IV allowances is `necessary' for EPA to ensure SIPs [State Implementation Plans] comply with section 110(a)(2)(D)(i)(I)."); 42 U.S.C. § 7601(a). And "`EPA cannot rely on its gap-filling authority to supplement the CAA's provisions when Congress has not left the agency a gap to fill' -- i.e., `when there is statutory language on point.'" WildEarth Guardians v. EPA, 830 F.3d 529, 539 (D.C. Cir. 2016) (citing Nat. Res. Def. Council v. EPA, 749 F.3d 1055, 1063 - 64 (D.C. Cir. 2014)); see also Nat. Res. Def. Council v. EPA, 749 F.3d 1055, 1063 - 64 (D.C. Cir. 2014) ("[W]e have consistently held that EPA's authority to issue ancillary regulations is not open-ended, particularly when there is statutory language on point." (citing Am. Petroleum Inst. v. EPA, 52 F.3d 1113, 1119 (D.C. Cir. 1995))).
EPA has not argued, nor could it, that the Proposed Rule is "necessary" for fulfilling the agency's rulemaking duties under the CAA. Rather, as discussed infra, Section IV.A of the commenters' comment letter, EPA's proposed exclusion of scientific data that is not publicly available is antithetical to the purposes of the CAA. Moreover, EPA itself has repeatedly determined, and the D.C. Circuit has affirmed, that disclosure of the data underlying studies on which the agency relies is not necessary to fulfill the Agency's transparency and public comment obligations under the CAA, 42 U.S.C. § 7607(d).
For example, when EPA set the 1997 PM NAAQS, "[s]everal commenters questioned EPA's ability to rely on studies demonstrating an association between PM and excess mortality without obtaining and disclosing the raw `data' underlying these studies for public review and comment." 62 Fed. Reg. 38,652, 38,689 (July 18, 1997); see also EPA, Responses to Comments on the 1996 Proposed Rule on the NAAQS for PM (July 1997), https://www3.epa.gov/ttn/naaqs/standards/pm/data/rtc_pm.pdf. EPA responded that "[i]t would be impractical and unnecessary for EPA to review underlying data for every study upon which it relies as support for every proposed rule or standard." 62 Fed. at 38,689. EPA made clear that disclosing such data was not its general practice, in part because EPA was not relying on the underlying data but rather on the study results themselves. Id. EPA recognized that "[i]f EPA and other governmental agencies could not rely on published studies without conducting an independent analysis of the enormous volume of raw data underlying them, then much plainly relevant scientific information would become unavailable to EPA for use in setting standards to protect public health and the environment." Id. EPA explained: 
[S]uch data are often the property of scientific investigators and are often not readily available because of the proprietary interests of the investigators or because of arrangements made to maintain confidentiality regarding personal health status and lifestyle information of individuals included in such data. Without provisions of confidentiality, the possibility of conducting such studies could be severely compromised. Id. 
And when the 1997 PM NAAQS was challenged, the D.C. Circuit affirmed EPA's consideration of relevant scientific epidemiological evidence without disclosure of all of the raw data. See Am. Trucking Ass'ns. v. EPA, 283 F.3d 355, 372 (D.C. Cir. 2002) (The court "agree[d] with EPA that requiring agencies to obtain and publicize the data underlying all studies on which they rely `would be impractical and unnecessary.'" (quoting 62 Fed. Reg. at 38,689) (emphasis added)).
In Coalition of Battery Recyclers Association v. EPA, 604 F.3d 613, 623 (D.C. Cir. 2010), petitioners again challenged EPA's failure to disclose underlying data for a study on the health effects of lead exposure, on which it relied to issue the 2008 Lead NAAQS. Specifically, the petitioners contended "the Lanphear study [on which EPA relied] contained such errors that EPA acted arbitrarily and capriciously in relying on results from the study without first obtaining and making public the underlying data for the study." Battery Recyclers, 604 F.3d at 622-23. EPA reiterated in its briefing that it would be "impractical and unnecessary" to disclose such data. See EPA Respondent Brief (Doc. No. 1230237) at 47 (Feb. 16, 2010) (citation and quotation marks omitted; emphasis added), Battery Recyclers, 604 F.3d at 623. The D.C. Circuit again agreed with and upheld EPA's determination that disclosure of underlying data is not necessary to consider the results of a health study to be relevant to a clean air rulemaking pursuant to 42 U.S.C. § 7607(d). The court applied its prior holding in American Trucking, that "`[t]he CAA imposes no such obligation' and that `requiring agencies to obtain and publicize the data underlying all studies on which they rely would be impractical and unnecessary.'" Battery Recyclers, 604 F.3d at 623 (quoting American Trucking, 283 F.3d at 372). Though petitioners "attempt[ed] to distinguish their request on the ground that in American Trucking the court was addressing requests for data underlying several studies, while they request only that EPA obtain and make public the data underlying the Lanphear study," id. at 623, the court found that argument unpersuasive, again "noting that raw data often is unavailable due to proprietary interests of a study's scientific investigators or confidentiality agreements with study participants." Id. at 623 (citing American Trucking, 283 F.3d at 372).
EPA also cannot argue that excluding studies from the Agency's consideration when the underlying data cannot be publicly disclosed is "necessary" for ensuring the Agency relies on the best available science. As an initial matter, nowhere in the Proposed Rule does EPA find that particular studies, much less all studies that rely in part on the collection of confidential raw data (such as people's names and health records), are bad science, or even less reliable science. Nowhere does EPA show how health studies that rely in part on confidential personal information can never be relevant in any way to CAA rulemakings. Nor could it, as EPA has found such studies relevant and has relied on such studies for decades, and they are commonly accepted and valued as important scientific information of health effects within the scientific community. See Section IV.A [of comment letter]. EPA cites no examples of situations where unsound, unlawful, or arbitrary decision-making resulted from an agency's reliance on studies that do not fit its newfound notions of "transparency" and "integrity." Rather, courts have repeatedly upheld actions that have relied on such studies, as cited above.
Moreover, EPA has not demonstrated that it is consistent with scientific principles to categorically exclude peer-reviewed scientific information from all consideration in a rulemaking, as EPA proposes to do. If EPA has any doubts or concerns regarding the merits of a particular study, it must address those doubts or concerns for that particular study in the context of a given rulemaking, where agency staff, internal scientific experts, scientific advisory committees, or commenters contend that study is relevant. EPA has provided no scientific justification for ignoring an entire class of health science simply because the underlying data has not been disclosed. Whether underlying data on which a study relies is made public or not simply has no bearing on whether a scientific study is good science, is accurate, is reliable, and is relevant to a scientific question (such as the health effects of air pollution). And to the extent EPA requires additional verification of a study, there are myriad ways it can do so without disclosing confidential data (for example, requesting an independent scientific body to conduct a confidential review). See Section IV.A, infra, of the commenters' comment letter. 
Furthermore, there is no statutory gap with respect to what studies EPA should consider (nor does EPA attempt to identify any). As further discussed below, sections of the CAA that govern air standards and rulemakings specify the applicable standards and generally require consideration of all available science. EPA has been adopting rules under most of these provisions for decades without finding any need for restrictions of the sort EPA proposes here. The Agency provides no explanation, and none exists, for suddenly finding "gaps" in these provisions.
Finally, EPA fails to acknowledge that it has had a longstanding policy of considering health studies without requiring disclosure of all underlying raw data. Indeed, it tries to minimize its prior position in a footnote, stating that: "Historically, EPA has not consistently observed the policies underlying this proposal, and courts have at times upheld EPA's use [of] non-public data in support of its regulatory actions." 83 Fed. Reg. at 18,769, n.3. It ignores the fact that EPA itself has consistently considered and used health studies dependent on non-public information for clean air rulemakings. Further, the Agency's longstanding policy has been that "EPA does not generally undertake evaluations of raw, unanalyzed scientific data as part of its public health standard setting process." 62 Fed. Reg. at 38,689. Only in "extreme cases  -  for example where there are credible allegations of fraud, abuse or misconduct  -  would a review of raw data be warranted." Id. That EPA now finds this data so important that it must be publicly disclosed before the Agency will even consider a study represents a monumental shift in course.
EPA is not working on a blank slate. Therefore, it must do more than just explain the change. Rather, EPA must provide "a more detailed justification," FCC v. Fox Television Stations, Inc., 556 U.S. 502, 515 (2009); see also Motor Vehicles Mfrs. Ass'n v. State Farm Mut. Auto. Ins. Co., 463 U.S. 29, 41-42 (1983), because its new policy directly contradicts the Agency's prior findings that such studies are relevant to clean air rulemakings and provide evidence of health effects that the Agency can and must consider. EPA has not provided any reasoned explanation for its departure, much less an explanation with the requisite detail to justify its about-face.
Comment [6112-1582]: Commenter (6137) states that Agency "regulations, in order to be valid, must be consistent with the statute under which they are promulgated." Decker v. Nw. Envtl. Def. Ctr., 568 U.S. 597, 609 (2013) (quoting United States v. Larionoff, 431 U.S. 864, 873 (1977)). As discussed below, the Proposed Rule violates a number of provisions in the statutes upon which EPA relies as statutory authority. For this reason, too, the Proposed Rule is invalid.
The CAA's specific rulemaking provisions do not allow EPA to create the restrictions on the consideration or use of health science that EPA proposes. Rather, these provisions govern each type of CAA rulemaking, and to the extent science can and must be considered under these provisions, EPA may not lawfully restrict the use of such science. The Proposed Rule contravenes a number of CAA provisions and is thus unlawful.
First, sections 108 and 109 of the CAA do not allow EPA to restrict science as proposed and demonstrate that the Proposed Rule cannot lawfully be applied to any NAAQS rulemakings. These provisions specify that EPA's air quality criteria (on which the NAAQS are based) must "accurately reflect the latest scientific knowledge." 42 U.S.C. § 7408(a)(2); id. § 7409(b) (requiring those "criteria" be used to set NAAQS). This language unambiguously requires EPA to consider "all identifiable effects on public health," not just some. Id. § 7408(a)(2). The criteria "shall include information" on defined factors, "to the extent practicable." Id. This provision leaves no room for EPA to ignore or exclude studies because underlying data is not disclosed. 
EPA cannot possibly ensure its air quality criteria "accurately reflect the latest scientific knowledge" if it refuses to even read certain studies based on an arbitrary public disclosure test. EPA's past practice illustrates this: for decades, the Agency's practice has been to review all available scientific studies, including those relying on non-public data. See, e.g., Battery Recyclers, 604 F.3d at 616; see also EPA, ISA for PM (Final Report, Dec 2009) (2009), https://cfpub.epa.gov/ncea/risk/recordisplay.cfm?deid=216546; EPA, Air Quality Criteria for PM, Vols. II-III (1996), https://cfpub.epa.gov/ncea/risk/recordisplay.cfm?deid=2832. The legislative history of § 108 confirms that Congress's intent was for EPA to "establish such national goals on the basis of the best information available," not to sit idly by until industry representatives no longer dispute questions regarding health effects. S. Rep. No. 91-1196 (Sept. 17, 1970), CAA 70 Leg. Hist. 19, 110 (emphasis added). 
In addition, the Proposed Rule would direct EPA to violate the statutory procedures that must be followed when the Agency sets NAAQS. This includes EPA's appointment of an independent scientific review committee, including certain defined members, see 42 U.S.C. § 7409(d)(2)(A), and consideration of the recommendations of that committee when setting the NAAQS, id. § 7409(d)(2)(B) -(C); see also, e.g., Mississippi v. EPA, 744 F.3d 1334, 1346 (D.C. Cir. 2013) (explaining NAAQS development process). That committee shall advise EPA regarding whether there are "areas in which additional knowledge is required." 42 U.S.C. § 7409(d)(2)(C). In promulgating NAAQS in the past, EPA has recognized that the CAA requires it to consider scientific advice and recommendations from such experts, including those that rely on health studies where underlying data is not disclosed. Directing the Agency to ignore scientific studies presented by CASAC, just because the underlying data is not public, contravenes these statutory requirements.
Second, section 7409 of the Act requires EPA to adopt NAAQS based on the criteria, at levels requisite to protect public health with an adequate margin of safety. "[T]he Act requires [a] . . . preventative and precautionary" approach to setting NAAQS, whereby EPA must protect public health from "not just known adverse effects, but those of scientific uncertainty or that research has not yet uncovered." Am. Lung Ass'n v. EPA, 134 F.3d 388, 389 (D.C. Cir. 1998) (citation and quotation marks omitted). Congress "specifically directed" EPA "to protect against . . . effects whose medical significance is a matter of disagreement." Lead Indus. Ass'n v. EPA, 647 F.2d 1130, 1154 (D.C. Cir. 1980); accord Nat. Res. Def. Council, Inc. v. EPA, 824 F.2d 1146, 1152 (D.C. Cir. 1987) (en banc) (discussing legislative history). EPA's proposal would flout these precedents by refusing to consider scientific studies  -  even those published in peer reviewed journals by reputable scientists  -  based on an arbitrary data transparency policy. 
To the extent the CAA allows EPA to weigh particular studies based on its expert judgment, this does not authorize EPA to categorically exclude an entire class of studies from being considered when performing a rulemaking to fulfill the Agency's statutory directive to protect public health and welfare. 42 U.S.C. § 7409(b)(1) (requiring primary NAAQS to be standards "the attainment and maintenance of which in the judgment of the Administrator, based on such criteria and allowing an adequate margin of safety, are requisite to protect the public health"). EPA cannot rationally engage in its task of determining an appropriate level for the NAAQS if it entirely excludes relevant health studies from its consideration simply because underlying data has not been publicly released. Under the Proposed Rule, EPA would refuse to consider studies indicating that adverse health effects occur at a specific pollutant level -- even where multiple studies reach the same results -- where the studies fail to meet the agency's arbitrary disclosure tests. This would contradict the statutory requirement to assure public health protection by ignoring some of the most important health science relevant to that question and is the epitome of irrational agency action. 
Third, the Proposed Rule violates section 7412 of the CAA, which includes myriad provisions that require EPA to evaluate health risks and effects of HAP and to set emission standards to reduce these risks and effects, based on certain science-based legal tests applicable to particular § 7412 rulemakings. See, e.g., 42 U.S.C. § 7412(a), (b)(1)-(4), (f)(1)-(2). In no place does the statute limit EPA's consideration of scientific studies on health effects or risks to those studies where underlying data is publicly disclosed, nor does it authorize EPA to so limit its consideration of such scientific information. Instead, § 7412 includes language repeatedly indicating the requirement, embodying Congressional intent, for EPA to consider all relevant scientific information regarding health risks and effects, actual or potential, of HAP. 
For example, § 7412(f) requires EPA to investigate and report, among other things, on "the actual health effects with respect to persons living in the vicinity of sources," and "any available epidemiological or other health studies" regarding the effects of HAP, as part of the residual risk requirements. Id. § 7412(f)(1)(C) (emphasis added); id. § 7412(f)(1) (also providing other requirements for EPA's investigation and report to Congress). EPA submitted that report to Congress in 1999. Section 7412(f) further provides that, in the absence of Congressional action on recommendations provided in EPA's Residual Risk Report to Congress, EPA "shall . . . promulgate standards for [each air toxics] category or subcategory if promulgation of such standards is required in order to provide an ample margin of safety to protect public health in accordance with this section (as in effect before November 15, 1990)." Id. § 7412(f)(2)(A). This provision also directs that:
Emission standards promulgated under this subsection shall provide an ample margin of safety to protect public health in accordance with this section (as in effect before November 15, 1990), unless the Administrator determines that a more stringent standard is necessary to prevent, taking into consideration costs, energy, safety, and other relevant factors, an adverse environmental effect. If standards promulgated pursuant to subsection (d) and applicable to a category or subcategory of sources emitting a pollutant (or pollutants) classified as a known, probable or possible human carcinogen do not reduce lifetime excess cancer risks to the individual most exposed to emissions from a source in the category or subcategory to less than one in one million, the Administrator shall promulgate standards under this subsection for such source category.
Id. § 7412(f)(2)(A). It would not be possible for EPA to fulfill its statutory directives under § 7412(f) to ensure that air toxics emissions standards "provide an ample margin of safety to protect public health," and to assess and remove any unacceptable health risks, unless EPA considered all relevant scientific studies in assessing such health risks. Id. § 7412(f)(2). 
Further, § 7412(f)(2) also explicitly refers to EPA's Benzene Rule, which interpreted the prior version of this provision and which itself relied on certain studies and guidelines for which underlying data was not disclosed. Id. § 7412(f)(2)(B) (citing 54 Fed. Reg. 38,044 (Sept. 14, 1989)). As EPA determined, and Congress, the D.C. Circuit, and EPA have affirmed through citation and reliance on that rule, § 7412(f) standards must be "based on the most current scientific knowledge," and on risk assessment guidelines and methods developed by EPA scientists and expert independent scientists. 54 Fed. Reg. at 38,062-63. EPA has repeatedly recognized this reliance on an expansive array of scientific support that includes information that relies on epidemiological and other health studies for which the underlying data is not published in later § 7412(f) rulemakings as well.
Indeed, EPA itself has interpreted its legal responsibility pursuant to this provision as "incorporating into our assessments the best available science with respect to dose-response information." To achieve that, EPA has followed scientific recommendations by the Office of Air Quality Planning and Standards, and has prioritized certain sources of such dose-response information according, in part, to "level of peer review received." These guidelines direct EPA to consult dose-response assessments such as a reference concentration (RfC, for inhalation), reference dose (RfD, for ingestion), and a unit risk estimate (URE, for cancer risk) and/or slope factor (SF, for cancer risk). EPA's scientific method is to consult and rely on IRIS (an EPA database containing peer-reviewed scientific health assessment information) as a top priority source of such information, due in part to the high level of peer review. As EPA's guidelines explain: "IRIS is a critical resource for risk assessors because the database contains toxicity information that reflects a consensus among EPA program offices." EPA also prioritizes dose response information from the U.S. ATSDR, and the California EPA Office of Environmental Health Hazard Assessment. Each of these recognizes the value of relevant scientific information without regard to whether full underlying data can be or has been publicly disclosed.
Section 7412(a)(11) likewise illustrates the constraints the CAA imposes on limiting consideration of science when establishing cancer risk. This provision defines "carcinogenic effect" as having "the meaning provided by the Administrator under Guidelines for Carcinogenic Risk Assessment as of the date of enactment." 42 U.S.C. § 7412(a)(11). These Guidelines for Carcinogenic Risk Assessment ("Cancer Guidelines"), in turn, direct that EPA shall rely on "established scientific peer review processes," and state that "[t]he cancer guidelines incorporate basic principles and science policies based on evaluation of the currently available information." The Cancer Guidelines also provide that EPA's Supplemental Guidance for Assessing Susceptibility from Early-Life Exposure to Carcinogens ("Supplemental Guidance") should be considered along with the Guidelines. Both the Cancer Guidelines and Supplemental Guidance cite as relevant and in some instances important some of the very types of scientific studies that EPA's Proposed Rule would categorically exclude: epidemiological studies which rely on private or confidential medical information, or assessments such as IRIS, California EPA's (Cal. EPA) assessments, and other health reference concentration information that rely on such studies. The Cancer Guidelines do not preclude the consideration of any "one kind of data" as relevant, but instead "cover the assessment of available data," explaining that "[i]t is very important that all analyses consider the basic standards of quality, including objectivity, utility, and integrity."
Thus, EPA's Proposal to ignore studies because underlying data is not disclosed is antithetical to the Guidelines upon which EPA relies when determining carcinogenic effects in § 7412(a)(11). The statute's text and incorporation of EPA's science guidelines on cancer risk are unambiguous and leave no gap to fill regarding what "carcinogenic" means. Thus, in regulating carcinogens EPA may not apply the Proposed Rule's exclusion of any relevant health science regarding carcinogens and carcinogenic risk from air pollutants in rulemakings. 
Consistent with the reliance on all relevant health science to determine carcinogenic effects, for cancer and other health risks under § 7412(f), EPA has an existing policy of what it describes as using the "the best available science with respect to dose-response information. The recommendations are based on the following sources, in order of priority": (1) EPA IRIS values which have all gone through independent, external peer-review; (2) ATSDR values, which follow an approach similar to EPA's IRIS program; and (3) Cal. EPA values for which "[t]he process for developing these assessments is similar to that used by EPA to develop IRIS values and incorporates significant external scientific peer review." Likewise, for non-cancer health risks from air pollution, EPA's guidelines do not exclude science that is relevant, even if underlying data is not disclosed. Notably, the vast majority of health reference values that EPA uses in § 7412(f) come from EPA's IRIS program, which includes a scientific literature review of all available relevant studies, without excluding any due to a lack of disclosure of underlying data.
Fourth, the Proposed Rule is unlawful because EPA is considering costs and is relying on implementation costs as a basis for the Rule, in direct violation of § 7409, § 7412(f)(2), and other provisions of the CAA where cost is not a relevant or permissible factor in determining health and environmental impacts. See, e.g., 83 Fed. Reg. at 18,768 (indicating that Proposed Rule would apply to "regulations for which the public is likely to bear the cost of compliance"); see also id. at 18,774 (proposed § 30.8) (requiring agency to implement the Proposed Rule "in a manner that minimizes costs"). Section 7412(f)(2) prohibits consideration of economic costs in assessing and determining whether the health risks that a major air toxics source causes are "unacceptable," as it requires a determination of what is required to provide an "ample margin of safety to protect the public health." Nat. Res. Def. Council, Inc., 824 F.2d at 1164-65 (quotation marks omitted); Benzene Rule, 54 Fed. Reg. at 38,048-49 (citing Vinyl Chloride decision as prohibiting consideration of costs when determining a "safe" or "acceptable" emission level). Similarly, as EPA explained in the Cancer Guidelines: "Risk assessments may be used to support decisions, but in order to maintain their integrity as decision-making tools, they are not influenced by consideration of the social or economic consequences of regulatory action." Cancer Guidelines, supra n.59, at 1-5 to 1-6. It is therefore both unlawful and arbitrary to use cost as a justification to ignore and exclude health science from residual risk air toxics assessments, and thus as part of the determination of whether risk is acceptable or unacceptable, pursuant to § 7412(f)(2).
Fifth, section 7412(n) directs EPA to "perform a study of the hazards to public health reasonably anticipated to occur as a result of emissions by electric utility steam generating units of pollutants listed under subsection (b) after imposition of the requirements of this chapter," and to list such sources "after considering the results of [this] study." 42 U.S.C. § 7412(n)(1). This provision includes no limitation on the data EPA can or must consider for this question, based on EPA's own interpretation. Thus, previously, in fulfilling its duty pursuant to this provision, EPA considered a wide array of scientific studies as relevant, regardless whether underlying data was disclosed. Excluding consideration of relevant scientific material addressing such hazards solely because underlying data is not available would arbitrarily lead to an incomplete assessment of relevant information and flout the statute's preventative and health-protective intent. EPA may not apply the Proposed Rule under this provision, as EPA has already recognized  -  for example, in acting pursuant to § 7412(n)(1) to reach the determination that it is "appropriate and necessary" to regulate power plants due to their health hazards  -  but rather must consider the types of studies the Proposed Rule would ignore. EPA cannot depart from its decision to consider such studies relevant without meeting the State Farm and Fox tests, see supra at 23, which it unquestionably has not done here. 
In addition, the listing and delisting provisions for HAP and source categories, and the requirements for the urban air toxics program, require EPA to assess particular and potential health effects and risks from HAP. See, e.g., 42 U.S.C. § 7412(b)(2)-(3), (c)(9), (k). EPA's Proposal to ignore relevant scientific information due to the lack of public disclosure violates these requirements. 
Sixth, CAA § 7429 requires EPA to evaluate health risks and does not allow the exclusion of relevant scientific information. For example: 
 § 7429(a)(3)  -  standards must include new unit siting requirements that, on a site-specific basis, minimize potential risks to public health or the environment;
 § 7429(e) requires permits to include site-specific provisions "if the Administrator or the State determines that emissions in the absence of such limitations or measures may reasonably be anticipated to endanger public health or the environment";
 § 7429 (h)(3) requires residual risk review under § 112(f), and § 129(b)(1) requires the inclusion of any residual risk standards in the guidelines for existing units.
Seventh, the Proposed Rule is antithetical to the very purpose of the CAA, which is "to protect and enhance the quality of the Nation's air resources so as to promote the public health and welfare and the productive capacity of its population." 42 U.S.C. § 7401(b)(1). A "primary goal" is "pollution prevention." Id. § 7401(c). Each of the above provisions must be read in a way that advances that goal; the Proposed Rule would do the contrary and is thus inconsistent with the statute and unlawful. EPA's Proposal runs directly counter to these goals and objectives by arbitrarily excluding consideration of science that discloses health impacts of air pollution. The Proposal is not neutral. It only excludes health-based science (dose-response studies, epidemiological studies) where underlying data is not disclosed, generally because it cannot or should not be disclosed to protect individual participants' privacy and confidentiality. EPA cannot exclude whole categories of scientific data untethered from a specific context or study, but rather must assess each health study on a case-by-case basis to determine whether or not it should be considered in a particular rulemaking, under EPA's long-standing scientific guidelines and policies and its regular approach in CAA rulemakings. Instead, this Proposed Rule excludes health studies from consideration as a class, up front, before EPA is even in the rulemaking stage under its authority. And it does so not for scientific reasons, but simply due to industry preferences  -  most notably because industry cannot pick them apart by replicating decades of air pollution health effects, or by contacting individuals who shared private medical information to replicate the collection of data.
While the CAA provides for protection of "public health and welfare," the Proposed Rule favors excluding science even if it is the most relevant and important evidence regarding how to protect public health. While the CAA aims for "pollution prevention" to protect public health, the Proposed Rule would prevent consideration of science relevant to these very goals. Therefore, EPA's Proposed Rule is unlawful and arbitrary. Thus, EPA cannot lawfully satisfy § 7429 for similar reasons as described above, unless it evaluates relevant information on risks.
Eighth, a number of important CAA provisions require EPA to act based on a finding that air pollution is reasonably anticipated to endanger health or the environment. See, e.g., 42 U.S.C. §§ 7415, 7422, 7521. The D.C. Circuit has ruled that such language "requires a precautionary, forward-looking scientific judgment about the risks of a particular air pollutant, consistent with the CAA's `precautionary and preventative orientation.'" Coal. for Responsible Regulation v. EPA, 684 F.3d 102, 122 (D.C. Cir. 2012) (citation omitted). Such a precautionary approach does not require scientific certainty. "If a statute is `precautionary in nature' and `designed to protect public health,' and the relevant evidence is `difficult to come by, uncertain, or conflicting because it is on the frontiers of scientific knowledge,' EPA need not provide `rigorous step-by-step proof of cause and effect' to support an endangerment finding." Id. at 121 (citation omitted). Thus, the court expressly rejected the notion that EPA can or should disregard uncertain or "difficult to come by" evidence under "endangerment" statutes. Indeed, the court rejected the notion that EPA could not rely on studies that synthesized the research of others:
It makes no difference that much of the scientific evidence in large part consisted of "syntheses" of individual studies and research. Even individual studies and research papers often synthesize past work in an area and then build upon it. This is how science works. EPA is not required to re-prove the existence of the atom every time it approaches a scientific question.
Id. at 120. Thus, in making endangerment determinations, there is no lawful or rational basis for EPA to automatically exclude reliance on any studies that synthesize and evaluate research by others. 
The Courts have also rejected EPA's attempts to avoid endangerment determinations based on considerations other than the specific endangerment criteria. In Massachusetts v. EPA, the Court held that the endangerment language in section 7521(a)(1) required EPA to assess whether motor vehicle emissions cause or contribute to air pollution which may reasonably be anticipated to endanger public health and welfare due to climate change, and to so determine exclusive of any other policy considerations. Massachusetts v. EPA, 549 U.S. 497, 532-34 (2007). Likewise, here, EPA cannot avoid its duty to make endangerment findings by arbitrarily rejecting scientific studies to serve vague and disingenuous policy interests such as allegedly fostering greater public trust and greater transparency in agency decisions. 
The Proposed Rule also conflicts with specific language in § 7415 requiring an endangerment finding notification, 
      [w]henever the Administrator, upon receipt of reports, surveys or studies from any duly constituted international agency has reason to believe that any air pollutant or pollutants emitted in the United States cause or contribute to air pollution which may reasonably be anticipated to endanger public health or welfare in a foreign country or whenever the Secretary of State requests him to do so with respect to such pollution which the Secretary of State alleges is of such a nature, the Administrator shall give formal notification thereof to the Governor of the State in which such emissions originate. 
42 U.S.C. § 7415(a). This provision does not allow EPA to ignore any such "reports, surveys or studies," if they show "reason to believe" that an air pollutant endangers public health. Id.
Similarly, § 7422 directs that EPA "shall review all available relevant information," to determine whether to make an endangerment finding for certain radioactive pollutants (including source material, special nuclear material, and byproduct material), cadmium, arsenic and polycyclic organic matter. 42 U.S.C. § 7422(a) (emphasis added). This language expressly forecloses EPA's refusal to consider available studies based on an arbitrary transparency screen. See also 42 U.S.C.§ 7521(a)(1)-(a)(3)(B) (providing for EPA to promulgate revised standards for heavy duty trucks "[o]n the basis of information available to the Administrator concerning the effects of air pollutants emitted from heavy-duty vehicles or engines and from other sources of mobile source related pollutants on the public health and welfare, and taking costs into account"). 
Finally, the CAA's rulemaking provision for air standards and limitations does not allow EPA to ignore relevant scientific information, including information provided by Commenters, and likewise may not direct a court to ignore this data. Section 7607 of the CAA  -  which provides for judicial review of air rulemakings  -  prescribes more detailed rulemaking procedures than those provided by the APA for a designated list of air emission standards and rules. 42 U.S.C. § 7607(d)(1). These procedures protect the public's right to notice and comment, in part, by requiring EPA to place into the docket and to consider and respond to all such comments. Id. § 7607(d)(4)(B)(i) ("Promptly upon receipt by the agency, all written comments and documentary information on the proposed rule received from any person for inclusion in the docket during the comment period shall be placed in the docket."); id. § 7607(d)(5) ("In promulgating a rule to which this subsection applies (i) the Administrator shall allow any person to submit written comments, data, or documentary information; (ii) the Administrator shall give interested persons an opportunity for the oral presentation of data, views, or arguments, in addition to an opportunity to make written submissions . . ."); id. § 7607(d)(6)(B) ("The promulgated rule shall also be accompanied by a response to each of the significant comments, criticisms, and new data submitted in written or oral presentations during the comment period."). 
EPA's Proposed Rule contravenes these requirements and is therefore unlawful because it allows EPA to decide, before even receiving comments, that it will not consider or respond through regulatory action to any comments that submit scientific material for which underlying information is not disclosed. Under § 7607(d), EPA may not lawfully decide to ignore an entire class of science; if submitted by commenters as relevant, EPA must consider and respond in the context of the statutory test that applies to its rulemaking. Failure to do so is unlawful and arbitrary. Id. § 7607(d)(9).
Relatedly, for certain rules where recommendations are provided from scientific experts, § 7607 requires additional material to be placed into the docket, without EPA discretion. Id. § 7607(d)(3) (requiring a "statement" that "shall also set forth or summarize and provide a reference to any pertinent findings, recommendations, and comments by the Scientific Review Committee established under section 7409(d) of this title and the NAS, and, if the proposal differs in any important respect from any of these recommendations, an explanation of the reasons for such differences."). EPA's Proposed Rule unlawfully violates this provision because it would direct EPA to refuse to consider or discuss such information if based on studies for which underlying data were not disclosed. 
More generally, EPA may not attempt to restrict, before a rulemaking has even begun, the type of information it will consider in that rulemaking. Doing so impinges on the federal courts' authority to determine what scientific evidence is relevant to application of CAA requirements in rulemakings. Pursuant to § 7607, the relevant court of appeals, and most frequently the D.C. Circuit, has jurisdiction to consider a petition for review of an EPA air rule. 42 U.S.C. § 7607(b). This grant of jurisdiction includes a grant allowing the court to decide what record material is relevant. Id.; see also id. § 7607(c). Notably, the court rules provide that the record on review of an agency order or regulation must include, inter alia, "the pleadings, evidence, and other parts of the proceedings before the agency." D.C. Cir. R. 16 ("If necessary, the court may direct that a supplemental record be prepared and filed."); see also Fed. R. App. P. 16. Similarly, the Federal Rules of Evidence require courts, not EPA or any other federal agency, to determine what evidence is "relevant" and "admissible." See, e.g., Fed. R. Evid. 401402 (allowing courts, and Congress by statute, but not federal agencies, to prescribe rules of evidence and determine admissibility of evidence and expert testimony). Scientific information submitted by commenters undoubtedly qualifies as "evidence" that the court must consider, even if EPA refuses to do so. EPA may not lawfully prevent submission of such evidence, or attempt to exclude it from a rulemaking record. Id.
Where EPA previously attempted to restrict or change the statutory test and authority granted to courts to evaluate CAA cases, the D.C. Circuit rejected that as unlawful and outside of the bounds of EPA's authority. Nat. Res. Def. Council v. EPA, 749 F.3d 1055, 1062 (D.C. Cir. 2014) (vacating affirmative defense to civil penalties because it changed the standard set by statute for court's discretion in enforcement cases, and thus violated § 7604 and § 7413). For this reason, too, the Proposed Rule is invalid.
Comment [6112-1626]: Commenter (6137) contends that the EPA's Proposal is also deficient because it fails to follow particular procedural requirements under the CAA that are legally required for EPA to issue certain types of rules. Specifically, the CAA requires that the CASAC complete a review of air quality criteria and the NAAQS every five years. 42 U.S.C. § 7409(d)(2)(B). Because the Proposed Rule aims to "preclude [the Agency] from using [nonpublic] data in future regulatory actions," it would impact the CASAC's review of air quality criteria. 83 Fed. Reg. at 18,769, n.3. Accordingly, EPA was required to follow the path prescribed by the CAA  -  including submission of the proposed rule to CASAC for review, allowing CASAC to provide recommendations to the Administrator, and then requiring the Administrator to include a statement regarding the recommendations made by CASAC in the Proposed Rule. 42 U.S.C. §§ 7409(d)(2)(B), 7607(d)(3); see also 42 U.S.C. § 7607(b), (d). Yet, there can be no dispute that none of these steps were taken before issuance of the Proposed Rule.
Comment [6112-2470]: Commenter (6137) asserts that it would be irrational and impracticable to require EPA to seek out the underlying data for, and demand separate peer review of, all data and models the Agency uses for purposes of characterizing the quantitative relationship between dose or exposure and magnitude of a predicted health or environmental impact. For instance, in the last ozone NAAQS review, EPA reviewed more than 4,000 studies and references for the ISA, and cited more than 2,200 in the final ISA. See EPA, Health and Environmental Research Online, ISA Ozone (2013) (last updated July 2, 2018), https://hero.epa.gov/hero/index.cfm/project/page/project_id/1628. It is neither practicable nor necessary for EPA to demand production of underlying data from thousands or even hundreds of studies, and to require new peer reviews of each. EPA cannot construe the statute in a way that would render it impossible to complete the review and revision of the NAAQS that Congress mandated. 
EPA further tries to justify the Proposed Rule as a way to ensure the Agency is not arbitrary and capricious in its conclusions. But as previously noted, the D.C. Circuit has twice rejected the notion that EPA must obtain and disclose data underlying the studies it relies on in NAAQS development.
Comment [6112-3473]: Commenter (6137) states that, to the extent EPA suggests the Rule may be applied retroactively, there is simply no basis for doing so. It is well-established that an agency cannot apply a rule retroactively absent clear congressional intention for such application. E.g., Sierra Club v. Whitman, 285 F.3d 63, 68 (D.C. Cir. 2002) (referring to "unusual ability to implement rules retroactively"); Bowen v. Georgetown Univ. Hospital, 488 U.S. 204, 208 (1988) ("Retroactivity is not favored in the law. Thus, congressional enactments and administrative rules will not be construed to have retroactive effect unless their language requires this result. By the same principle, a statutory grant of legislative rulemaking authority will not, as a general matter, be understood to encompass the power to promulgate retroactive rules unless that power is conveyed by Congress in express terms." (citations omitted)). EPA has not identified any such congressional intention in any of the statutes at issue, and thus retroactive application of the Rule would be unlawful. 
Nor would the statutes upon which EPA relies support such an application. For example, as explained above, the CAA does not allow -- and, as the D.C. Circuit has held, certainly does not require, e.g., American Trucking, 283 F.3d at 372 -- applying the so-called transparency provisions of the Proposed Rule at all in rules subject to the procedural requirements of CAA § 307(d). Such rules include NAAQS. Further, far from suggesting that EPA could lawfully reopen long-settled NAAQS to apply a new and novel standard of review, the CAA requires EPA to review and revise air quality criteria and NAAQS at least every five years and to "promulgate such new standards as may be appropriate." 42 U.S.C. § 7409(d)(1) (emphasis added). This carefully chosen language confirms that Congress did not intend for EPA to apply rules like the Proposed Rule to undo existing NAAQS, but instead intended for regular reviews of scientific information to result in new NAAQS.
Comment [6112-4453]: Commenter (6137) expresses that, in addition, in performing certain CAA and other rulemakings, EPA may not consider the economic implications of considering or excluding certain science at all. See Section IV.A of the commenters' comment letter. Yet EPA's failure to provide any lawful or rational justification for its costs/benefits statement for the Proposed Rule suggests that economic cost may have impermissibly played a role in its analysis. Had EPA focused on health rather than cost, EPA could not possibly find that the benefits of ignoring health science outweigh the costs for public health rulemakings. 
Comment [6116-2627]: Commenter (6141) states that, most importantly, the CAA and other major federal environmental statutes provide EPA with broad authority to establish procedures, requirements, and criteria for adopting rules, regulations, and policies for how the Agency adopts new regulations, rules, and guidance. For example, section 301(a) of the CAA provides the EPA Administrator with broad authority to establish regulations "as are necessary to carry out ... the functions" under the CAA, while section 307(d) provides extensive authority on the manner in which EPA may conducting notice and comment rulemakings, including provisions for ensuring a robust rulemaking record and adequate opportunities for public participation.
Comment [6118-1681]: Commenter (6143) states that the EPA has issued its proposed rule on Strengthening Transparency in Regulatory Science pursuant to its authority under several environmental statutes that that it implements, including the CAA; CWA; SDWA; RCRA; CERCLA; EPCRA; FIFRA; and TSCA. This proposed action is fully consistent with the authorities and objectives laid out in those statutes. EPA must ensure that any final action it takes on this proposal continues to respect the balance reflected in those statutes between sound, transparent science, and legitimate privacy interests.
Comment [6119-1360]: Commenter (6144) states that the CAA has requirements to update pollution standards that provide for an "adequate margin of safety" to protect public health. This determination can only be reliably made using the best available science. This proposal would, however, prevent EPA from using the best available information to set science-based pollution standards that would provide for an adequate margin of safety because the proposal excludes studies where the raw data and other underlying information cannot be made public for good reason. EPA should not cite the CAA as an appropriate source of statutory authority. Further, the CAA encourages a very close partnership with the NAS and other independent scientific institutions. When EPA is considering a proposal to reinterpret how it uses science in regulatory decisions, it should first and foremost consider comments from and engage with NAS and similar bodies, before moving forward in such a rushed manner. If EPA is arguing that it derives its authority from statutes like the CAA, it should only do so after consultation with independent scientific bodies and significant stakeholder engagement.
Comment [6128-2225]: Commenter (6153) notes that the CAA requires that EPA set NAAQS to protect the public health, and to do so with an adequate margin of safety. NAAQS must be based on air quality criteria that "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air." EPA must comprehensively consider the state of scientific knowledge, and set standards to "protect individuals who are particularly sensitive to the effects of pollution." These susceptible groups include children; the elderly; and those with heart disease, lung disease, or asthma. Recently, Patricia Koman and colleagues have suggested that pregnant women should be considered an at-risk population due to physiological changes, noting, "Women with preexisting conditions, women experiencing poverty, and groups that suffer systematic discrimination may be particularly susceptible to cardiac effects of air pollutants during pregnancy."
Comment [6149-2755]: Commenter (6174) contends that there is no basis in existing bedrock environmental laws that authorizes EPA to limit science considered in rulemaking processes. EPA cites several key laws as a justification for this current proposal to limit or constrain the science it can consider. Nowhere in the cited statutes is there any basis for demanding access to raw data, nor does this relate sensibly to any definition of best available science. Rather it undermines the use of best available science as called for in numerous environmental statutes including the CAA. Further, there is no basis in the statutes for politically appointed administrators to choose which science will be considered and which may not be.
The CAA, for example, has requirements to update pollution standards that provide for an adequate margin of safety for public health. This determination can only be reliably made using the best available science. However, this proposal would prevent EPA from using the best available information to set science-based pollution standards that would provide for an adequate margin of safety. EPA should not cite the CAA as an appropriate source of statutory authority.
In sum, there is no statutory authority for EPA to rely on to censor or constrain science. Indeed, quite the opposite. Further, such an effort would only serve to undermine EPA's essential role.
Comment [6152-637]: Commenter (6122) states that the proposed rule is not based on any valid statutory authority and is arbitrary and capricious.
EPA cites multiple statutory provisions in an attempt to support the proposed rule that it believes authorize this type of rulemaking, "including provisions providing general authority to promulgate regulations necessary to carry out the Agency's functions under these statutes and provisions specifically addressing the Agency's conducting of and reliance on scientific activity to inform those functions . . ." None of the specific statutory provisions provided by EPA provide specific authority for EPA to fundamentally rework the foundational processes it uses to assess and include/reject science. Indeed, if such authority existed, one would have expected EPA to have discovered this authority many decades earlier. Instead, EPA is attempting to use general grants of authority to create a completely novel approach to assessing science. However, as the Supreme Court has stated, "Congress...does not alter the fundamental details of a regulatory scheme in vague terms or ancillary provisions -- it does not, one might say, hide elephants in mouseholes." And if this is true, neither can the EPA alter the fundamental details of all of its regulatory schemes based on such vague provisions of law. We review each of the provisions listed by EPA, none of which discuss transparency, dose response, pivotal regulatory science, benefit-cost calculation, or changing "agency culture," the stated goals of this rulemaking.
Section 103 of the CAA directs the EPA to "conduct, and promote the coordination and acceleration of, research, investigations, experiments, demonstrations, surveys and studies" on air pollution. Curtailing the use of science based on arbitrary notions of transparency does the opposite of what Section 103 requires.
Section 301(a) of the CAA gives the Administrator authority to "prescribe such regulations as are necessary to carry out his functions" relating to the CAA. This catch-all provision certainly provides the authority to prescribe regulations, but EPA has not provided a single rationale explaining how the proposed rule is "necessary" to the carrying out functions under the CAA.
Comment [6155-632]: Commenter (6125) notes that Congress first established the legal standards that NAAQS must meet in the 1970 CAA. EPA must establish primary and secondary NAAQS for air pollutants, "emission of which ... cause or contribute to air pollution which may reasonably be anticipated to endanger public health or welfare". CAA section 108 (a)(1)(A). For each such pollutant, the law requires NAAQS not just to "protect the public health" but to do so "with an adequate ​margin of safety." CAA § 109(b)(1) (emphasis added). In setting these standards, EPA cannot consider the costs of meeting them, see Whitman v. American Trucking, 531 U.S. 457, 464-71 (2001) (Scalia, J.) but must instead comprehensively consider the state of scientific knowledge. To make sure this happens, Congress required NAAQS to be based on "air quality criteria," CAA § 109 (b), which must "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air." CAA §108(a)(2) (emphasis added). The Supreme Court agrees that NAAQS must be based on "published air quality criteria that reflect the latest scientific knowledge." Whitman, supra, at 457. Accord, State of Mississippi v. EPA, 744 F. 3d. 1344, 1346 (D.C. Cir. 2013). 
Air quality criteria must include information on "variable factors...which of themselves or in combination with other factors may alter the effects on public health or welfare of such air pollutant"; the types of air pollutants which ... may interact with such pollutant to prove an adverse effect"; and "any known or anticipated adverse effects on welfare." CAA section 108 (a)(2) (A)-(C). To make sure that EPA based its decisions on this latest useful science, in 1997 Congress also required EPA to appoint a scientific review committee  -  the CASAC -- to review the scientific information to be embodied in the air quality criteria and used to determine NAAQS. CAA § 109(d).
These statutory provisions originated in the 1970 Senate version of the CAA. The Senate Report on them stated that: "Margins of safety are essential to any health-related environmental standards if a reasonable degree of protection is to be provided against hazards which research has not yet identified" Library of Congress. Environmental Policy Division, (197480). A legislative history of the Clean Air Amendments of 1970, together with a section-by-section index. Washington: U.S. Govt. Print. Off. at p. 410 (emphasis added). The courts agree. See, e.g. American Trucking Association v. EPA (ATA III), 283 F.3d. at 369 (D.C. Cir. 2002). 
EPA could not obey these statutory commands unless it considered (1) all knowledge that met the standards of quality observed by the scientific community to establish health effects of pollution; (2) knowledge that, though it might not meet these standards, was nevertheless "useful in indicating" these effects or pointing to "hazards which research has not yet identified;" and (3) all the knowledge regarding the factors which may result in these pollutants' synergistic effects on public health and welfare.
 Since the NAAQS provisions were enacted in 1970, EPA has conducted many NAAQS rulemakings. The agency does not establish per se, a priori rules regarding study inclusion or exclusion, but rather evaluates each of the individual studies  --  and there are thousands typically evaluated for each NAAQS review  --  on its merits based on reasoned criteria. While details of the development and review of the criteria and standards have evolved, in practice, over the last four decades, EPA has endeavored to include all relevant peer reviewed scientific studies in the process, even searching out relevant newer literature to be sure that what is considered in a regulatory context is comprehensive. During this period, EPA has included tens of thousands of peer reviewed studies of health effects, exposure, and atmospheric interactions, and monitoring in reviews of criteria and standards. 
A requirement that any study must be excluded from consideration unless the raw data and full methodologies are made available for each of them is both impractical and inconsistent with the legislative mandate and EPA's practice over the last 40 years. Reasoned criteria that EPA has typically applied include:
 Are the study populations, subjects, or animal models adequately selected, and are they sufficiently well-defined to allow for meaningful comparisons between study or exposure groups?
 Are the statistical analyses appropriate, properly performed, and properly interpreted?
 Are likely covariates adequately controlled or taken into account in the study design and statistical analysis?
 Are the air quality data, exposure, or dose metrics of adequate quality and sufficiently representative of information regarding ambient conditions?
 Are the health, ecological or welfare effect measurements meaningful, valid, and reliable?
 Do the analytical methods provide adequate sensitivity and precision to support conclusions?
Thus a science regulation that applies to the NAAQS is unlawful unless EPA can show that the new standard can be established and implemented consistent with the applicable statutory requirements, including those requiring the use of "scientific knowledge" and specifying how to use it. This proposal has neither acknowledged that such requirements exist nor explained how it can, under the regulation, produce scientific information and rulemaking records meeting them, and there is nothing in the record to demonstrate that it can. Thus, contrary to the implicit suggestion in the proposal that EPA is working with a blank slate when it puts a unique spin on science under the CAA, it is entering a field where Congress has provided expansive directives. Moreover, the agency and the congressionally mandated CASAC have long followed an inclusive approach to selecting and evaluating the relevant scientific literature, consistent with these directives. 
Automatic disclosure of data underlying scientific studies, regardless of the obstacles, has never been a precondition to EPA consideration of those studies in setting NAAQS. In two cases cited in the proposal, plaintiffs challenged rules on the basis that there was no public access to underlying data. In each case, EPA declined to provide the data, and in each case, the court affirmed that denial, and endorsed EPA's reasoning in doing so.
Comment [6155-665]: Commenter (6125) states that even if the per se rule could satisfy the substantive requirements of the NAAQS  -- which it cannot  --  the proposal fails utterly to comply with procedural requirements for establishing NAAQS, and in particular, for amending the air quality criteria on which the NAAQS are based. The law requires EPA to set NAAQS (as well as virtually every other significant CAA rule) through a structured dialogue with the public. CAA § 307(d)(3). First, EPA must issue a proposal setting out the "factual data" relied on the "methodology used in obtaining the data and in analyzing the data," and the major legal and policy considerations underlying the proposal. The public can then comment on every aspect of this proposal and can submit its own data. When EPA takes final action, it must respond, "to each of the significant comments, criticisms, or new data submitted." Ib.
These provisions largely mirror the requirements of the APA though they are more elaborately stated. Other specific provisions, however, go well beyond APA requirements. Specifically, any change in criteria documents must by law be reviewed by CASAC. CAA §109(d). In addition, criteria documents must also be reviewed by EPA's SAB under 42 USC 42 USC section §4365 (c)(1) (the Environmental Research Development Demonstration Authorization Act). 
EPA's proposal would specifically and indisputably amend the air quality criteria, adopted pursuant to section 108 (b) of the CAA, for PM and lead. See proposed rule footnote 3 final sentence which states that the proposed rule would "preclude" using the Lanphear study which is part of the criteria for the NAAQS for lead, and the Harvard Six Cities and American Cancer Society II studies which are part of the air quality criteria for the PM NAAQS, and presumably a number of informative studies that update and expand these original works by adding more recently collected health and air pollution information. For these reasons, the statement at proposal 18773 saying the rule "does not establish an environmental health or safety standard" is false.
The proposal furthermore affects the current review of the PM NAAQS by proposing to regulate -- in fact, dictate -- the type of science that can be used in that review, and removing studies which were part of the air quality criteria in the past reviews, which would again directly alter the air quality criteria. Air quality criteria cannot be amended without review by CASAC. See CAA §109 (d)(2)(B). EPA consequently must submit its proposal to CASAC for its review, following all procedural requirements for public meeting and deliberations in doing so. CASAC must then submit its recommendations to the Administrator (see §109 (d)(2)(B) final clause), and the Administrator must consider these recommendations and provide a reasonable explanation for any actions that deviate significantly from those recommendations (CAA §307(d)(3)). 
EPA cannot proceed with this action until these requirements are satisfied. And EPA cannot short-circuit the role the law assigns CASAC by using this type of broad rulemaking, effectively dictating how science can be used in future rule makings, to restrict CASAC's authority. 
The proposal would amend the substantive standards for decision-making for a host of actions covered by CAA section 307 (d), among them the NAAQS (§307 (d)(1)(A)), residual risk determinations for HAP (§307 (d)(1)(C)), standards for mobile source air toxics (§307 (d)(1)(K)), and residual risk standards for municipal solid waste combustors (§307(d)(1)(D)). Therefore, CAA §307 (d)(5)(ii) and (iv)require the Administrator to hold a public hearing on his proposal and to keep the record open for an additional thirty days, "to provide an opportunity for submission of rebuttal and supplementary information."
EPA should have submitted its proposal to the SAB pursuant to the requirements of 42 USC§ 4365 (c)(1) (the Environmental Research Development Demonstration Authorization Act), which requires the Administrator to submit to the SAB any "proposed criteria document, standard, limitation, or regulation, together with relevant scientific and technical information in the possession of the Environmental Protection Agency (EPA) ... on which the proposed action is based" at the time it provides that proposal to another agency of the government for formal review. The SAB is then to review and comment on the proposal, which the Administrator is to consider, although the Administrator is not required to obtain Science Advisory Board approval for any final action. See H. Rep. No. 95-722 (95th Cong. 1st Sess. (1977) (Conference Report). 
EPA and the SAB have adopted procedures to implement this statutory requirement, whereby EPA provides SAB with a description (including a pertinent summary of potential issues of scientific concern) of planned major actions not yet proposed and the SAB determines, in a public forum, which of these actions merits its consideration and comment. See Memorandum of December 27, 2012, "Identifying EPA Planned Actions for Science Advisory Board Consideration of the Underlying Science" from Michael Goo, Assistant Administrator for Policy, Glenn Paulsen, EPA Science Advisor, and Vanessa Vu, SAB Office Director, and Memorandum of November 12, 2013 from SAB Chair James Mihelic to Members of the Chartered SAB and Liaisons. EPA has failed to comply with its statutory obligations and the requirements of its internal procedures.
In short, in its evident zeal to advance purported "transparency," EPA has thus ignored a variety of statutory and regulatory requirements which provide actual transparency to agency rulemakings, including the CAA's scientific review process. 
Comment [6155-2630]: Commenter (6125) states that it is clear from even a brief look that these provisions require EPA to consider all the best available science, without authorizing, or even mentioning any of the per se restrictions based on "transparency" that EPA has proposed to use as a universal overriding rule. For these reasons, EPA must justify its proposal as a proper exercise of its authority under each of the statutory provisions whose operation it would affect - provisions that the agency does not even cite. It is the job of the proposer, not the commenters, to survey these provisions and provide the needed justification. 
In several of the listed statutes Congress has explicitly told EPA how to use science in decision-making. As an example, the commenter notes that CAA 108(a)(2) specifies that "air quality criteria for an air pollutant ​shall accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air." 
These provisions, and others, all use mandatory language directing EPA to use reliable and relevant scientific information -- and further specify how to do so. They all direct that decisions regarding protection of public health be based on the full range of relevant scientific information, in some cases specifying how to identify what information to use. None of them give EPA discretion to ignore such information for any of the reasons set forth in the proposal.
Comment [6161-1504]: Commenter (6131) notes that the EPA itself seems to question whether it has the authority to promulgate this rule. The FR notice asks commenters to provide information on what areas of the CAA they feel may demonstrate this type of authority. The EPA should have a solid foundation before it proposes rules and should not have to seek answers for questions about its own authority.
Comment [6163-1102]: Commenter (6133) notes that the EPA cites 42 U.S.C. § 7601(a) of the CAA as one basis for the Proposal. But that section merely authorizes the Administrator to "prescribe such regulations as are necessary to carry out his functions under this chapter." The courts have made clear that "EPA cannot rely on its gap-filling authority to supplement the CAA's provisions when Congress has not left the agency a gap to fill." NRDC v. EPA., 749 F.3d 1055, 1064 (D.C. Cir. 2014); see also American Petroleum Institute v. EPA, 52 F.3d 1113, 1119 (D.C. Cir. 1995) ("the general grant of rulemaking power to EPA cannot trump specific portions of the CAA"); NRDC v. Reilly, 976 F.2d 36, 41 (D.C. Cir. 1992) (EPA cannot use its general rulemaking authority as justification for adding to a statutorily specified list); Sierra Club v. EPA, 719 F.2d 436, 453 (D.C. Cir. 1983) (same); Gonzales v. Oregon, 546 U.S. 243, 264 - 65 (2006) ("It would go . . . against the plain language of the text to treat a delegation for the `execution' of [the Attorney General's] functions as a further delegation to define other functions well beyond the statute's specific grants of authority."). Here, not only is there no statutory gap to fill, as explained further below, the Proposal is in direct conflict with other provisions of the Act. EPA cannot rely on 42 U.S.C. § 7601(a) to support this rule.
EPA also cites 42 U.S.C. § 7403, which requires the Administrator to establish a national research and development program for air pollution, among other things. EPA does not state specifically which of the many subsections it believes authorizes this proposed rule. Thus, the citation fails to provide sufficient notice for the public to comment on the proposed rule. 
Nothing in the Proposal establishes or even purports to establish the type of national research and development program for air pollution discussed in subsection (a). But that subsection is nonetheless revealing about congressional intent concerning "studies relating to the causes, effects (including health and welfare effects) extent, prevention, and control of air pollution." 42 U.S.C. § 7403(a)(1). There is no indication that Congress intended to allow EPA to ignore or refuse to consider studies on the health and welfare effects of air pollution only if raw data or `regulatory science underlying EPA's actions [were] publicly available in a manner sufficient for independent validation." See 83 Fed. Reg. at 18,773 (proposed §§ 30.1 - 30.3). Indeed, the absence of any such congressional conditions or criteria makes it all the more obvious that EPA invented and added those criteria and conditions as a matter of its own policy preferences, contrary to the Act. This EPA may not do.
Subsection (b) authorizes EPA to collect and make available information about such research, but nothing in that subsection allows EPA to restrict which types of data it considers in regulatory decisions. Nor does subsection (b) draw any distinction between dose-response data and other types of data. Again, the absence of any such congressional distinction makes it all the more obvious that EPA invented and added that distinction as a matter of its own policy preferences, contrary to the Act. This EPA may not do. None of the other subsections in 42 U.S.C. § 7403 address this issue either. There is no support in the CAA for the Proposal.
Comment [6163-1128]: Commenter (6133) notes that in CAA section 101(b), Congress directs EPA "to protect and enhance the quality of the Nation's air resources so as to promote the public health and welfare and the productive capacity of its population." 42 U.S.C. § 7401(b). The Proposal prevents EPA from doing so by blocking the agency from considering information that also is the best available, peer-reviewed, independent, credible science that could persuade or cause the agency to better protect the "public health and welfare and the productive capacity of [the Nation's] population." In this way, the Proposal thwarts the leading purpose of the CAA. CAA section 101 shows the Proposal is arbitrary and capricious and an abuse of EPA's discretion. 
CAA section 103(a)(1) directs EPA to "conduct, and promote the coordination and acceleration of, research, investigations, experiments, demonstrations, surveys, and studies relating to the causes, effects (including health and welfare effects), extent, prevention, and control of air pollution." 42 U.S.C. § 7403(a)(1). There is nothing in these congressional directives restricting these tasks ("research, investigations, experiments, demonstrations, surveys, and studies") to materials based only on data that are "publicly available in a manner sufficient for independent validation." 83 Fed. Reg. at 18,773/2 (proposed § 30.1). Nor is there any authorization in these congressional directives to distinguish between "research, investigations, experiments, demonstrations, surveys, and studies" that involves "dose response data and models," and science that does not, for imposing regulatory restrictions on what science EPA must consider. The absence of any such congressional restrictions, authorizations or distinctions makes it clear that EPA invented and added the Proposal's limitations and conditions as a matter of its own policy preferences, contrary to the Act. This EPA may not do. CAA section 103(a)(1) shows that the Proposal is arbitrary and capricious and an abuse of EPA's discretion. 
CAA subsection 103(a)(4) directs EPA to "establish technical advisory committees composed of recognized experts in various aspects of air pollution to assist in the examination and evaluation of research progress and proposals and to avoid duplication of research." 42 U.S.C. § 7403(a)(4). CAA section 103(a)(5) directs EPA to "conduct and promote coordination and acceleration of training for individuals relating to the causes, effects, extent, prevention, and control of air pollution." Id. § 7403(a)(5). There is nothing in these congressional directives restricting these tasks to materials based only on data that are "publicly available in a manner sufficient for independent validation." 83 Fed. Reg. at 18,773/2 (proposed § 30.1). Nor is there any authorization in these congressional directives to distinguish between research or science that involves "dose response data and models" on the one hand, and science that does not, on the other hand, for imposing regulatory restrictions on what science EPA must consider. The absence of any such congressional restrictions, authorizations, or distinctions makes it clear that EPA invented and added the Proposal's limitations and conditions as a matter of its own policy preferences, contrary to the Act. This EPA may not do. CAA subsections 103(a)(4) & (5) show that the Proposal is arbitrary and capricious and an abuse of EPA's discretion. 
CAA section 103(b) is significantly titled "Authorized Activities of Administrator in Establishing Research and Development Program." 42 U.S.C. § 7403(b) (emphasis added). It states that: 
In carrying out the provisions of the preceding subsection the Administrator is authorized to -- 
      (1) collect and make available, through publications and other appropriate means, the results of and other information, including appropriate recommendations by him in connection therewith, pertaining to such research and other activities;
      (2) cooperate with other Federal departments and agencies, with air pollution control agencies, with other public and private agencies, institutions, and organizations, and with any industries involved, in the preparation and conduct of such research and other activities;
      (3) make grants to air pollution control agencies, to other public or nonprofit private agencies, institutions, and organizations, and to individuals, for purposes stated in subsection (a)(1) of this section;
      (4) contract with public or private agencies, institutions, and organizations, and with individuals, without regard to section 3324(a) and (b) of title 31 and section 6101 of title 41;
      (5) establish and maintain research fellowships, in the EPA and at public or nonprofit private educational institutions or research organizations;
      (6) collect and disseminate, in cooperation with other Federal departments and agencies, and with other public or private agencies, institutions, and organizations having related responsibilities, basic data on chemical, physical, and biological effects of varying air quality and other information pertaining to air pollution and the prevention and control thereof;
      (7) develop effective and practical processes, methods, and prototype devices for the prevention or control of air pollution; and
      (8) construct facilities, provide equipment, and employ staff as necessary to carry out this chapter. 
Id. There is nothing in these congressional directives restricting these tasks, research or data to materials based only on data that are "publicly available in a manner sufficient for independent validation." 83 Fed. Reg. at 18,773/2 (proposed § 30.1). Nor is there any authorization in these congressional directives to distinguish between research, science, data, or models that involves "dose response data and models" on the one hand, and science that does not, on the other hand, for imposing regulatory restrictions on what science EPA must consider. The absence of any such congressional restrictions, authorizations or distinctions makes it clear that EPA invented and added the Proposal's limitations and conditions as a matter of its own policy preferences, contrary to the Act. This EPA may not do. CAA subsections 103(b) shows that the Proposal is arbitrary and capricious and an abuse of EPA's discretion. 
CAA section 103(d) addresses Environmental Health Effects Research: 
      (1) The Administrator, in consultation with the Secretary of Health and Human Services (HHS), shall conduct a research program on the short-term and long-term effects of air pollutants, including wood smoke, on human health. In conducting such research program, the Administrator -- 
      (A) shall conduct studies, including epidemiological, clinical, and laboratory and field studies, as necessary to identify and evaluate exposure to and effects of air pollutants on human health;
      (B) may utilize, on a reimbursable basis, the facilities of existing Federal scientific laboratories and research centers; and
      (C) shall consult with other Federal agencies to ensure that similar research being conducted in other agencies is coordinated to avoid duplication. 
      42 U.S.C. § 7403(d). 
There is nothing in these congressional directives restricting these tasks, research, studies, or data to materials based only on data that are "publicly available in a manner sufficient for independent validation." 83 Fed. Reg. at 18,773/2 (proposed § 30.1). Nor is there any authorization in these congressional directives to distinguish between research, science, data, or models that involves "dose response data and models" on the one hand, and science that does not, on the other hand, for imposing regulatory restrictions on what science EPA must consider. The absence of any such congressional restrictions, authorizations, or distinctions makes it clear that EPA invented and added the Proposal's limitations and conditions as a matter of its own policy preferences, contrary to the Act. This EPA may not do. CAA subsection 103(d) shows that the Proposal is arbitrary and capricious and an abuse of EPA's discretion. 
CAA subsection 103(d)(2) directs "[i]n conducting the research program under this subsection, the Administrator shall develop methods and techniques necessary to identify and assess the risks to human health from both routine and accidental exposures to individual air pollutants and combinations thereof." 42 U.S.C. § 7403(d)(2). Subsection 103(d)(2) then says, "such research program shall include the following elements," listing subsections (A)-(C). Id. Subsection 103(d)(2)(B) & (C) are especially relevant and revealing: 
      (B) An evaluation, within 12 months after November 15, 1990, of each of the HAP listed under section 7412(b) of this title, to decide, on the basis of available information, their relative priority for preparation of environmental health assessments pursuant to subparagraph (C). The evaluation shall be based on reasonably anticipated toxicity to humans and exposure factors such as frequency of occurrence as an air pollutant and volume of emissions in populated areas. Such evaluation shall be reviewed by the Interagency Task Force established pursuant to subparagraph (A). 
      (C) Preparation of environmental health assessments for each of the HAP referred to in subparagraph (B), beginning 6 months after the first meeting of the Interagency Task Force and to be completed within 96 months thereafter. No fewer than 24 assessments shall be completed and published annually. The assessments shall be prepared in accordance with guidelines developed by the Administrator in consultation with the Interagency Task Force and the SAB of the EPA. Each such assessment shall include --  
      (i) an examination, summary, and evaluation of available toxicological and epidemiological information for the pollutant to ascertain the levels of human exposure which pose a significant threat to human health and the associated acute, subacute, and chronic adverse health effects;
      (ii) a determination of gaps in available information related to human health effects and exposure levels; and
      (iii) where appropriate, an identification of additional activities, including toxicological and inhalation testing, needed to identify the types or levels of exposure which may present significant risk of adverse health effects in humans."
      42 U.S.C. § 7403(d)(2)(B) & (C) (emphases added). 
There is nothing in these congressional directives restricting these tasks, research, studies, or data to materials based only on data that are "publicly available in a manner sufficient for independent validation." 83 Fed. Reg. at 18,773/2 (proposed § 30.1). Nor is there any authorization in these congressional directives to distinguish between research, science, data, or models that involves "dose response data and models" on the one hand, and science that does not, on the other hand, for imposing regulatory restrictions on what science EPA must consider. The absence of any such congressional restrictions, authorizations, or distinctions makes it clear that EPA invented and added the Proposal's limitations and conditions as a matter of its own policy preferences, contrary to the Act. This EPA may not do. CAA subsection 103(d)(2) shows that the Proposal is arbitrary and capricious and an abuse of EPA's discretion. 
Equally damning for the Proposal, when Congress directs EPA to evaluate pollutants and their health effects, Congress uses broad and capacious terms:
 "on the basis of available information" (§ 103(d)(2)(B), 42 U.S.C. § 7403(d)(2)(B)); 
 "available toxicological and epidemiological information for the pollutant to ascertain the levels of human exposure which pose a significant threat to human health and the associated acute, subacute, and chronic adverse health effects" (§ 103(d)(2)(C)(i), 42 U.S.C. § 7403(d)(2)(C)(i)); and 
 "available information related to human health effects and exposure levels" (§ 103(d)(2)(C)(ii), 42 U.S.C. § 7403(d)(2)(C)(ii)).
These instructions to EPA are prefaced with the mandatory language, "[s]uch research program shall include the following elements." (§ 103(d)(2), 42 U.S.C. § 7403(d)(2)). Congress went out of its way not to authorize EPA to ignore "available toxicological and epidemiological information" to ensure that the agency would be "ascertain[ing] the levels of human exposure which pose a significant threat to human health and the associated acute, subacute, and chronic adverse health effects." (§ 103(d)(2)(C)(i), 42 U.S.C. § 7403(d)(2)(C)(i)). 
This shows clear congressional concern with all available science related to human health effects from air pollution -- not some restricted, politicized subset of science where underlying, confidential data are "publicly available in a manner sufficient for independent validation." CAA subsection 103(d)(2) shows that the Proposal is arbitrary and capricious and an abuse of EPA's discretion. 
When Congress directs EPA to conduct an ecosystem research program in subsection 103(e), Congress says that such program "shall include" "[e]valuation of risks to ecosystems exposed to air pollutants, including characterization of the causes and effects of chronic and episodic exposures to air pollutants and determination of the reversibility of those effects." 42 U.S.C. § 7403(e). Subsections (e)(3) -(e)(6) address other effects on water quality, crops, soils, and other elements of ecosystems. 
There is nothing in these congressional directives restricting these tasks, research, studies, or data to materials based only on data that are "publicly available in a manner sufficient for independent validation." 83 Fed. Reg. at 18,773/2 (proposed § 30.1). Nor is there any authorization in these congressional directives to distinguish between research, science, data, or models that involves "dose response data and models" on the one hand, and science that does not, on the other hand, for imposing regulatory restrictions on what science EPA must consider. The absence of any such congressional restrictions, authorizations, or distinctions makes it clear that EPA invented and added the Proposal's limitations and conditions as a matter of its own policy preferences, contrary to the Act. This EPA may not do. CAA subsection 103(e) shows that the Proposal is arbitrary and capricious and an abuse of EPA's discretion. 
In section 108(a)(2) of the CAA, Congress required air quality criteria for air pollutants to "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air," CAA § 108(a)(2), 42 U.S.C. § 7408(a)(2) (emphases added). In American Trucking v. Whitman, 531 U.S. at 457, the Supreme Court said that NAAQS must be based on "published air quality criteria that reflect the latest scientific knowledge."
The Proposal violates these statutory requirements by prohibiting EPA from considering available science to discharge the agency's statutory responsibility to "protect the public health," with "an adequate margin of safety." CAA § 109(b)(1), 42 U.S.C. § 7409(b)(1). The Proposal does this by subverting and supplanting the congressional criteria in CAA § 108(a)(2), 42 U.S.C. § 7408(a)(2) with a restrictive standard driven by whether raw data are "publicly available in a manner sufficient for independent validation," 83 Fed. Reg. at 18,773/2 (proposed § 30.1).
With this unlawful maneuver, the Proposal prevents EPA from adopting air quality criteria that "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare." CAA § 108(a)(2), 42 U.S.C. § 7408(a)(2) (emphases added). First, the Proposal thwarts the congressional directives for "accurate reflection" of the "latest scientific knowledge." It does so by compelling or allowing EPA to ignore the "latest scientific knowledge," and to fail to "accurately reflect" that science, if raw data are not "publicly available in a manner sufficient for independent validation." 83 Fed. Reg. at 18,773/2 (proposed § 30.1).
Moreover, the Proposal thwarts the congressional directives for science that is "useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air." CAA §108(a)(2), 42 U.S.C. § 7408(a)(2) (emphases added). It does so, again, by compelling or allowing EPA to ignore the "latest scientific knowledge," and to fail to accurately reflect that science, if raw data are not "publicly available in a manner sufficient for independent validation." 83 Fed. Reg. at 18,773/2 (proposed § 30.1). 
Further, section 208 contains the mandatory term "shall" -- which does not give the agency latitude. It does not matter if that "scientific knowledge" is "publicly available" in the way EPA contemplates in the Proposal, it must simply inform the effects of air pollution on public health or welfare. Further, American Trucking considered the requirements of this section and specifically concluded that "the CAA imposes no" "general requirement that EPA obtain and publicize the data underlying published studies on which the Agency relies." 283 F.3d at 372. 
In these statutory provisions, obviously there is no mention of the necessity, or even relevance, of raw data being "publicly available in a manner sufficient for independent validation" before EPA must consider studies based on that data. Equally plain, there is no authorization for EPA to fail to "accurately reflect" that science when issuing air quality criteria. 
There is nothing in these congressional directives restricting EPA's responsibilities, or the research, studies, or data it must consider, to materials based only on data that are "publicly available in a manner sufficient for independent validation." 83 Fed. Reg. at 18,773/2 (proposed § 30.1). Nor is there any authorization in these congressional directives to distinguish between research, science, data, or models that involve "dose response data and models" on the one hand, and science that does not, on the other hand, for imposing regulatory restrictions on what science EPA must consider. The absence of any such congressional restrictions, authorizations, or distinctions makes it clear that EPA invented and added the Proposal's limitations and conditions as a matter of its own policy preferences, contrary to the Act. This EPA may not do. CAA subsection 108(a) shows that the Proposal is arbitrary and capricious and an abuse of EPA's discretion. 
The Proposal also violates section 109 of the CAA and contravenes the Supreme Court decision in American Trucking v. Whitman. The Proposal's conception of "pivotal regulatory science" turns on, among other things, "analyses that drive the magnitude of the benefit-cost calculation," and "studies, models and analyses" that are "critical to the calculation of a final regulatory standard or level, or to the quantified costs, benefits, risks and other impacts on which a final regulation is based." 83 Fed. Reg. at 18,770/2; id. at 18,773/2 (proposed § 30.2 (dose response data and models)). CAA section 109(b)(1) requires EPA to promulgate or revise health-based NAAQS that are "requisite to protect the public health," "allowing an adequate margin of safety." 42 U.S.C. § 7409(b)(1).
As noted, in American Trucking v. Whitman, a unanimous Supreme Court said that NAAQS must be based on "published air quality criteria that reflect the latest scientific knowledge." 531 U.S. at 457. Moreover, the Court held that CAA section "109(b), interpreted in its statutory and historical context and with appreciation for its importance to the CAA as a whole, unambiguously bars cost considerations from the NAAQS-setting process." 531 U.S. at 471. The Court also squarely rejected arguments appealing to statutory language concerning "adverse public health, welfare, social, economic, or energy effects which may result from various strategies for attainment and maintenance" of NAAQS. The justices made clear such language and concerns have "no bearing upon whether cost considerations are to be taken into account in formulating the standards." Id. 
The Proposal violates CAA section 109(b)(1) and the governing Supreme Court interpretation in American Trucking by purporting to allow the "magnitude of a benefit-cost calculation" and "quantified costs and benefits" to impact or govern (1) EPA's consideration of peer-reviewed science relevant to reviewing, setting or revising health-based NAAQS; and (2) EPA's review, revision or establishment of health-based NAAQS. This is unlawful. 
CAA sections 109(b)(1), (2), & (c) require EPA to protect Americans' "public health" with an adequate margin of safety, and America's "welfare" from "any known or anticipated adverse effect." 42 U.S.C. § 7409(b)(1), (2) & (c). The Proposal prevents EPA from doing so by blocking the agency from considering information that also is the best available, peer-reviewed, independent, credible science that could persuade or cause the agency to better protect Americans' public health and welfare, based on the statutory criteria in section 109. In this way, the Proposal thwarts the central role and fundamental right to health-based air quality standards under the CAA. CAA section 109 shows the Proposal is arbitrary and capricious and an abuse of EPA's discretion. 
CAA section 111(a)(1) defines a standard of performance as: 
      a standard for emissions of air pollutants which reflects the degree of emission limitation achievable through the application of the best system of emission reduction which (taking into account the cost of achieving such reduction and any non-air quality health and environmental impact and energy requirements) the Administrator determines has been adequately demonstrated." 42 U.S.C. 7411(a)(1). 
There is nothing in these congressional directives restricting EPA's establishment of "standards of performance," or its determinations of "achievability" or "best system of emission reduction" or "adequate demonstration," to information based only on data that are "publicly available in a manner sufficient for independent validation." 83 Fed. Reg. at 18,773/2 (proposed § 30.1). Nor is there any authorization in these congressional directives to distinguish between research, science, data, or models involving "dose response data and models" on one hand, and science that does not, on the other hand, for imposing regulatory restrictions on what science EPA must consider. 
EPA cannot fulfill the congressional directive to establish the "best system of emission reduction" if EPA is artificially and unlawfully restricting its consideration of data and information to those that are "publicly available in a manner sufficient for independent validation." Id. Nor may EPA fulfill the "adequately demonstrated" directive if systems of emission reduction that have been adequately demonstrated require EPA to consider data, science, or information that are not "publicly available in a manner sufficient for independent validation." Id. 
CAA section 111(b)(1)(A) requires EPA to establish a list of stationary sources to be subject to section 111 standards of performance: 
      [The Administrator] shall include a category of sources in such list if in his judgment it causes, or contributes significantly to, air pollution which may reasonably be anticipated to endanger public health or welfare.
      42 U.S.C. 7411(b)(1)(A) (emphasis added). 
There is nothing in the Act restricting EPA's consideration of which categories of sources "may reasonably be anticipated to endanger public health or welfare" to information based only on data that are "publicly available in a manner sufficient for independent validation." 83 Fed. Reg. at 18,773/2 (proposed § 30.1). There is no indication of congressional intent that what "may reasonably be anticipated to endanger public health or welfare" may be modified or constrained by ignoring science and data concerning endangerment if that information is not "publicly available in a manner sufficient for independent validation." 
The absence of any such congressional restrictions, authorizations, or distinctions concerning what EPA may consider makes it clear that EPA invented and added the Proposal's limitations and conditions as a matter of its own policy preferences, contrary to the Act. This EPA may not do. CAA subsections 111(a)(1) and 111(b)(1)(A) show that the Proposal is arbitrary and capricious and an abuse of EPA's discretion. 
CAA section 112(b) provides a list of toxic air pollutants for which industrial sources must limit their emissions. The statute then directs the Administrator to periodically review that list of HAP and, where appropriate, revise this list by rule. In particular, the Administer is directed to add pollutants which:
      present, or may present, through inhalation or other routes of exposure, a threat of adverse human health effects (including, but not limited to, substances which are known to be, or may reasonably be anticipated to be, carcinogenic, mutagenic, teratogenic, neurotoxic, which cause reproductive dysfunction, or which are acutely or chronically toxic) or adverse environmental effects whether through ambient concentrations, bioaccumulation, deposition, or otherwise, but not including releases subject to regulation under subsection (r) as a result of emissions to the air. 42 U.S.C. § 7412(b)(2). 
There is nothing in these congressional directives restricting EPA's establishment of this list nor of the pollutants that should be added to it based only on data that are "publicly available in a manner sufficient for independent validation." 83 Fed. Reg. at 18,773/2 (proposed § 30.1). Nor is there any authorization in these congressional directives to distinguish between research, science, data, or models involving "dose response data and models" on one hand, and science that does not, on the other hand, for imposing regulatory restrictions on what science EPA must consider. 
EPA cannot fulfill the congressional directive to establish section 112(b)(2)'s pollutant list if the agency is artificially and unlawfully restricting its consideration of data and information to those that are "publicly available in a manner sufficient for independent validation." Id. Nor will EPA be able to fully analyze pollutants for inclusion on this list if determining inclusion would require EPA to consider data, science or information that are not "publicly available in a manner sufficient for independent validation." Id. 
Similarly, Section 112(b)(3) lays out a petition process to add chemicals to the Section 112 list that similarly require the petitioner to submit to EPA proof that "the substance is an air pollutant and that emissions, ambient concentrations, bioaccumulation or deposition of the substance are known to cause or may reasonably be anticipated to cause adverse effects to human health or adverse environmental effects." 42 U.S.C. § 7412(b)(3)(B). Here, the straitjacket that the Proposal would place on this statutory language would similarly prevent the agency from carrying out its statutory directive.
Section 112(b)(3)(C) provides criteria for delisting pollutants from the list. This section would nonetheless be hamstrung if the agency were limited exclusively to data that are "publicly available in a manner sufficient for independent validation." 83 Fed. Reg. at 18,773/2 (proposed § 30.1). Nor is there any authorization in the section's congressional directives to distinguish between research, science, data, or models involving "dose response data and models" on one hand, and science that does not, on the other hand, for purposes of listing or delisting pollutants from section 112's hazardous pollutant list. 
Nearly every subsection of Section 112, including standards for major and area sources, reporting requirements, and accidental release provisions, touch on protecting "public health," weighing "risks," or assessing how "hazardous" a "substance" or "pollutant" may be. EPA cannot fulfill the congressional directives of any of these sections if the agency is artificially and unlawfully restricting its consideration of data and information to those that are "publicly available in a manner sufficient for independent validation." Id. Nor will EPA be able to fully analyze risks to or impacts on human health and set section 112 standards accordingly if making such determinations would require EPA to consider data, science, or information that are not "publicly available in a manner sufficient for independent validation." Id. 
The absence of any such congressional restrictions, authorizations, or distinctions concerning what EPA may consider makes it clear that EPA invented and added the Proposal's limitations and conditions as a matter of its own policy preferences, contrary to the Act. This EPA may not do. CAA section 112 makes exceedingly clear that the Proposal is arbitrary and capricious and an abuse of EPA's discretion. 
The sections listed above merely represent a sampling of some examples in Title 1 of the Act that exemplify the extent to which the Proposal is arbitrary, capricious, an abuse of EPA's discretion, and a violation of clear congressional directives. The Act's five other Titles are no different, and the list provided here is not exhaustive -- the CAA is rife with examples of statutory language that the Proposal would distort with its adherence to data that are "publicly available in a manner sufficient for independent validation" 83 Fed. Reg. at 18,773/2 (proposed § 30.1) and research, science, data, or models involving "dose response data and models."
Comment [6165-1550]: Commenter (6135) states that they agree with the findings of the U.S. Court of Appeals for the D.C. Circuit in American Trucking Associations v Environmental Protection Agency that the CAA imposes no obligation to obtain and make public the data underlying certain "key studies" and similarly reject arguments that a general requirement be imposed for EPA to obtain and publicize the data underlying published studies on which the agency relies. We also agree with the court's finding that "requiring agencies to obtain and publicize the data underlying all studies on which they rely 'would be impractical and unnecessary.'"
Comment [6168-4099]: Commenter (6178) raises the following concerns with the Transparency Rule proposed by the NPRM:
1. The Agency cites specific sections of statutes it administers as the basis of the Agency's authority to adopt the Transparency Rule; specifically CAA sections 103, 301(a), 42 U.S.C. 7403, 7601(a). The Agency rightly cites this act of Congress in an attempt to support the NPRM as such authority is necessary for the Agency to adopt regulations pursuant to the CAA. 
2. However, Congress, in each section cited, never instructs the Agency to consider transparency to the public of supporting data and/or models of the scientific research when deciding to rely on such research as support for regulations passed pursuant to the CAA. Consequently, the Transparency Rule requires the EPA to rely on factors which Congress has not intended it to consider when promulgating regulations pursuant to the CAA, rendering the Transparency Rule arbitrary and capricious under the APA. 
3. Further, not only does the Transparency Rule lack sufficient Congressional intent to avoid being deemed arbitrary and capricious, implementing the Rule would actually contradict Congressional instruction to conduct scientific research with a focus on `promotion' and `coordination' among itself and other institutions that produce scientific research material to the objectives of the CAA Congress requires the EPA to enforce. This provides additional cause for finding the Transparency Rule arbitrary and capricious under the APA.
Comment [6168-4102]: Commenter (6178) states that the CAA does not instruct the Agency to consider Transparency to the public when evaluating scientific data. 
It is a basic principle of separation of powers enshrined in the American system of government that only Congress passes laws, and then only the executive branch enforces such laws. With respect to executive agencies like the EPA, this principle translates into limitations on such agencies: the agencies are inherently restricted by the specifics of the laws they enforce. 
The EPA's NPRM violates this basic principle of separation of powers. This is evident when reviewing each of the Acts the Agency cited in the NPRM as each instruct the EPA to conduct scientific research with very specific goals in mind, none of which however concern transparency to the public of the underlying data and models supporting the conclusions of such research. 
To illustrate this lack of Congressional intent, section 7403 of the CAA, one of two cited by the Agency to support the Transparency Rule as authorized by the CAA, Congress instructs the EPA administrator to "establish a national research and development program for the prevention and control of air pollution and as part of such program shall...conduct, and promote the coordination and acceleration of, research, investigations, experiments, demonstrations, surveys, and studies relating to the causes, effects (including health and welfare effects), extent, prevention, and control of air pollution; encourage [and] cooperate with... air pollution control agencies and other appropriate public or private agencies, institutions, and organizations, and individuals in the conduct of such activities; conduct investigations and research and make surveys concerning any specific problem of air pollution in cooperation with any air pollution control agency with a view to recommending a solution of such problem...; establish technical advisory committees composed of recognized experts in various aspects of air pollution to assist in the examination and evaluation of research progress and proposals and to avoid duplication of research; and conduct and promote coordination and acceleration of training for individuals relating to the causes, effects, extent, prevention, and control of air pollution." [emphasis added] 
The other section cited, CAA section 7601, states "the Administrator [of the EPA] is authorized to prescribe such regulations as are necessary to carry out his functions under this chapter." 
Each of the emboldened phrases above highlights Congress's instructions to the EPA on how to conduct scientific research when determining the appropriate regulations to adopt pursuant to the CAA. Nowhere in such instructions does Congress direct the Agency to consider whether the underlying data and/or models supporting the conclusions of such scientific research is transparent such that, in the words of the NPRM, "the data and models underlying the science [pivotal to its significant regulatory actions] is publicly available in a manner sufficient for validation and analysis." Therefore, the Transparency Rule does not follow from the Congressional instruction found in the CAA. 
Furthermore, Congress instructs the EPA in the CAA to focus on `promoting cooperation' between it and other institutions, public and private, researching air pollution such that `acceleration' of necessary research is achieved, resulting in required EPA regulations being enacted as soon as possible. Focusing on transparency as described in the NPRM would very likely delay promulgation of such regulations as coordination between critical research institutions that utilize nonpublic information (i.e. research participant's personal data or proprietary research databases) would inevitably be hindered, if not entirely eliminated. Accordingly, not only would the Transparency Rule's adoption lack basis in the Congressional instructions cited by the Agency, it would also contradict the basic Congressional instructions that the Agency Administrator is only `authorized to prescribe such regulations as are necessary to carry out his functions'. 
Each of the other Acts' sections cited by the Agency in support of the Transparency Rule contain substantially similar clauses, therefore the above analysis applies to each cited Act.
Comment [6334-3168]: Commenter (6356) asserts that the regulation should not be applied retroactively. EPA proposes on page 18,771 that this regulation, if finalized, would apply prospectively. But it solicits comment regarding how the proposed rule should apply to future rulemakings which are based on the administrative record from previous reviews  -  particularly where the governing statute requires repeated review on a fixed date-certain cycle and dose-response data and models underlying pivotal regulatory science if those were developed prior to the effective date of this rule. Id. at 18,772/col. 1. The commenter submits that to the extent that future regulations rely on old data and/or dose-response relationships, then the proposed regulation should be transparent on both sets of information--the data and/or dose-response models must be available for review and independent validation in the context of the new rulemaking. However, the information could not form the basis for a challenge of a standard or pollutant limitation that EPA promulgated prior to adoption of this final agency action. In other words, prior regulations should not be affected by this rulemaking, even if EPA has not made their basis available for independent validation. But if that information forms the basis for a new standard or pollutant limitation, then the data and dose-response relationships that were the foundation of the historic requirements would be subject to this regulation if adopted. To the extent that these methods of analyses have evolved or been replaced, and particularly if they are distinguishable, it may change the outcome of future regulations, even if the old regulations adopted prior to this rulemaking continue to be implemented.
EPA should explain, in the context of this rulemaking, that many environmental statutes, including but not limited to the CAA, contain anti-backsliding provisions regarding the relaxation of standards. See, e.g., CSS Section 172(e), 42 U.S.C. §7502(e). Courts have interpreted this CAA provision to mean that while EPA can revoke a NAAQS for a criteria pollutant, the non-backsliding provisions of the Act prohibit the agency from revoking regulatory requirements that were required for implementation of a prior standard. See South Coast Air Quality Management District v. EPA, 882 F.3d 1138 (D.C. Cir. (2018), petition for rehearing pending; South Coast Air Quality Management District v. EPA, 472 F.3d 882 (2006), rev'd. 489 F.3d 1245 (D.C. Cir. 2007). Both decisions involved revocation of a standard and its replacement with a more stringent standard, so NEDA/CAP expresses no opinion of revision of standard to a less stringent standard, because we consider that unlikely to happen. It may be helpful for EPA to flag similar statutory provisions in its final action on this Notice.
Comment [6340-1454]: Commenter (6362) states that the provisions cited by EPA under the CAA, the CWA, the SDWA, the CERCLA, and the EPCRA in support of its authority to develop and implement its proposed rule all provide broad regulatory authority to promulgate regulations "as are necessary to carry out [the Administrator's] functions" under the statute. 
Comment [6343-2707]: Commenter (6365) expresses that they are committed to meeting their obligations to operate in a responsible manner to avoid adversely impacting the environment and public health and safety. Both lead itself and the production of lead are extensively regulated by the EPA, including under the CAA. As a result of this program and other actions taken by industry, "[t]he United States have made tremendous progress in reducing lead concentrations in the outdoor air," with the national average concentrations dropping "more than 90 percent since 1980," and "EPA expects that most of the areas previously designated as nonattainment will have attained or are on track to attain by the end of 2016."
In order to ensure that the regulatory standards that the commenter's members are subject to are supported by the best possible science, the ABR supports EPA's objective of strengthening transparency in regulatory science by ensuring that the data and models underlying regulatory science is publicly available in a manner sufficient for validation and analysis. Furthermore, EPA should establish clear criteria for data and model availability so that stakeholders can effectively determine whether the data and models underlying each regulatory proposal are available for review.
Comment [6353-2326]: Commenter (6375) notes that the EPA solicits comment on whether alternative or additional regulatory or other policy vehicles are appropriate to establish and implement these policies, and whether further regulatory or other policy vehicles at the programmatic or statutory level would be appropriate as alternative or additional steps the agency may take to further the policies articulated in Section II of the NPRM's preamble. (pg. 18771).
The regulatory language that EPA proposes provides general requirements that apply to the Agency's use of science under the many statutes that it implements. The generality of the proposed regulations may be a function of the diverse purposes of those statutes Specifically, some of these statutes focus on the introduction of new substances into the marketplace (e.g., FIFRA and TSCA), while others primarily concern regulation of emissions or releases to (e.g., CAA and CWA), or removal from (e.g., CERCLA), the environment. The scientific analyses required for these different programs and the sources of the scientific information underlying those analyses differ. The Agency should consider supplementing the general language that it has proposed with statutory specific regulations. A statutory-specific rulemaking would provide greater certainty going forward.
Comment [6355-2849]: Commenter (6377) notes that CAA Section 108(a)(2) requires NAAQS to be based on air quality criteria that "shall accurately reflect the latest scientific knowledge" useful in identifying potential health and welfare effects, including, "to the extent practicable," specific information on variable factors, pollutant interactions and welfare effects. These statutory provisions impose three distinct requirements for the scientific information contained in criteria documents (now known as ISA): (1) it must be accurate; (2) it must reflect the latest relevant information available; and (3) specific data must be provided where practicable.
In adopting these provisions in 1970, the Senate Committee expressly noted the absence of key scientific information relevant to NAAQS development and adopted an ambitious research program to fill the gaps, including research on: (1) the contribution of air pollution to the etiology of disease; (2) synergistic effects; (3) tissue accumulation; (4) functional impairment; (5) contributing factors such as age, occupation and smoking; and (6) improved predictive models. The Committee stated that "This effort should be directed toward the accelerated development of more comprehensive air quality criteria" (id.). Accordingly, as new information is developed, Congress directed EPA to include it in more comprehensive versions of the ISA.
This requirement was strengthened in the 1977 CAA Amendments, which included the rulemaking provisions that now appear in Section 107(d) of the Act. Those provisions originated in the House bill, and in adopting them the House Committee found that "appropriately broad administrative discretion to promulgate regulations to protect health or the environment must be restrained by thorough and careful procedural safeguards that ensure an effective opportunity for public participation in the rulemaking process." Two provisions were adopted to provide such assurance. With respect to proposed rules, Section 307(d)(3) provides:
All data, information, and documents . . . on which the proposed rule relies shall be included in the docket on the date of publication of the proposed rule. 
Section 307(d)(6)(C) imposes a similar requirement on final rules:
The promulgated rule may not be based (in whole or in part) on any information or data which has not been placed in the docket as of the date of such promulgation.
As the 1977 House Committee noted, these provisions were necessary because Congress gave EPA broad discretion in developing NAAQS, rejecting the "substantial evidence" standard for judicial review in favor of the more lenient "arbitrary and capricious" standard, under which courts give substantial deference to agency findings of fact. In adopting this approach, Congress wanted to be sure that all of the scientific information on which the agency relies is available for public review at the proposal stage and for judicial review in the wake of a final standard. The House Committee went on to state that "Of course, the agency has at least as great an obligation to include any such documents that contradict its position as it does to include those that support it" (p. 321).
To recap, Section 108(a) of the Act requires NAAQS to be based on scientific information that is accurate and the latest relevant information available, including specific data where practicable. Section 307(d) requires all data and information on which proposed and final rules are based to be placed in the underlying dockets. These provisions amount to a statutory requirement for EPA to provide scientific transparency to the maximum extent practicable in developing ISA and proposed and final NAAQS rules.
Comment [6355-2851]: Commenter (6377) states that the substantial costs of the NAAQS program have been documented extensively. In enacting the CAA Amendments of 1970 and 1977, Congress prohibited EPA from considering costs in establishing NAAQS, but also adopted a series of provisions designed to ensure that the potential benefits of the program would be justified by the scientific data in NAAQS rulemaking dockets. EPA's transparency proposal is a reasonable effort to implement those requirements while providing adequate protections for privacy, confidentiality, and other potential problem areas. Far from being inconsistent with the applicable CAA provisions, EPA's transparency proposal is required by them and is long overdue. The Coalition urges EPA to implement the proposal expeditiously and apply it to all currently pending NAAQS reviews.
Comment [6424-1252]: Commenter (6446) notes that the proposed rule deviates from accepted practices that EPA routinely uses, and is required to use, to evaluate science it relies on for regulatory decisions. EPA has long recognized that proprietary CBI submitted to it cannot be shared with the public, yet it routinely relies on that information, including scientific studies and other scientific information, as the foundation for important regulatory decisions. EPA also adheres to specific Congressional directives regarding the science it should consider for regulatory action, including directives to consider whether the scientific information is reasonable, clear, complete and whether it has been peer reviewed. (See, e.g., the TSCA at 15 U.S.C. § 2625, subds. (h) and (i), requiring EPA to consider several factors to evaluate science relating to toxic chemicals and requiring that decisions be based on "the weight of the scientific evidence," and the CAA at 42 U.S.C. § 7408, subd. (a)(2), requiring EPA to rely on "the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices.")
Comment [6870-4001]: Commenter (6892) states that, in their view, the federal statutes not only fail to support the proposal but instead require consideration of all relevant scientific evidence. The provisions cited by the EPA proposal provide flimsy support  -  thus the plaintive cry for legal assistance from public commenters. More importantly, the proposal flies in the face of EPA's primary responsibility to base its decisions on consideration of all relevant scientific evidence.
It would be tedious to discuss the statutes in each of the different areas that EPA regulates, and not very useful, since the provisions cited by EPA in each area are similar. Air pollution is probably the most important area covered by the proposal in public health terms, so I will use that as an example. 
The problem is not just that the CAA fails to provide clear authority for EPA's proposed exclusionary rule. There are also provisions in the Act that cut strongly against the proposal.
Like many recent EPA actions, the "transparency" proposal is an effort to elevate the concerns of regulated businesses at the expense of EPA's primary mission: protecting public health and safety. Transparency is a worthy goal, but it should not serve as an excuse to ignore inconvenient evidence or to compromise public safety.
Comment [6872-3439]: Commenter (6894) expresses that an unintended negative outcome, the community of scientists, which has increasingly grown to share the essential data used in research, would be thwarted by those who may be unable or unwilling to share their data, especially post-facto data that were part of the current publications already having gone through the peer review and publication process. The result would be a crippling, if not an elimination of the strong, crucial science that has been the backbone of existent, validated regulatory reviews per the CAA mandate.
Comment [6873-1716]: Commenter (6895) notes that, in one example of the need for valid science-based standards, the PM-2.5 NAAQS set limits based on daily and annual exposure to PM-2.5 in ambient air. The regulation was supported by studies that were performed in many communities by many scientists and those scientists were able to calculate similar linkages between a concentration of PM2.5 and an acute or chronic health effect.
The Health Effects Institute (HEI) examined two of the sentinel studies that were pivotal to the development of the PM-2.5 NAAQS. The Harvard Six Cities Study and the American Cancer Society Study of Particulate Air Pollution and Mortality were selected for independent validation because the development of the NAAQS relied on their findings. A team of independent analysts started from the individual health and air quality data to see if the author's conclusions were supported. They corroborated the findings of the original studies.
Further, as noted above the re-validation of the Harvard Six Cities Study and the American Cancer Society Study of Particulate Air Pollution and Mortality as well as the re-analysis of 40 studies all used to support the development of the PM-2.5 NAAQS shows clearly that the authors of these pivotal studies provided sufficient data so that the work could be replicated and validated. There is no need for revisiting these studies through the application of additional restrictions or requirements for public data availability nor from the release of personally identifying information.
In 1971 President Richard Nixon signed into law the ban on lead in paint in public housing, followed later by a ban in all paint sold in the United States. In 1973, the newly formed EPA reviewed blood lead levels among urban adults and children grouped by location and found blood lead levels which exceeded concentrations that were known to be associated with adverse health effects. The data was pooled and masked by sex, age, location, and potential exposure category. EPA's review led to the removal of lead from gasoline in 1995. All these actions directly resulted in lower ambient lead levels and the prevention of adverse neurological effects in children and adults from lead exposure. If this situation were to occur today, in accordance with this proposed rule EPA would be required to replicate these findings, making each individual's results available for further review by non-scientific individuals (lawyers and politicians), which would have further delayed the removal of lead from gasoline and jeopardized the health of millions of children living in urban areas across the United States. EPA requests comments on alternatives to redoing studies upon which current regulations are based. The Departments are strongly opposed to any retroactive application of the proposed rule. The pivotal studies used in many of the past regulatory actions including the promulgation of the PM-2.5 NAAQS provided sufficient data so that the work could be replicated and validated in a manner widely accepted by the scientific community in the lead in gasoline matter, an alternative would be to put lead back in gasoline. That would of course be unconscionable, however, EPA's reasoning could well lead to this type of result.
Comment [6891-2149]: Commenter (6921) states that numerous laws such as the CAA require the use of the best available science when implementing their provisions; the proposed rule would run afoul of such laws and hinder their proper implementation .
Comment [6894-3604]: Commenter (6915) states that the CAA requires that air quality criteria "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare." § 108(a)(2), 42 U.S.C. § 7408(a)(2) (emphases added).
Comment [6895-3068]: Commenter (6916) states that the EPA's attempt to justify its action on the basis that courts previously have found it has the discretion to rely on non-public data in decision making, see 83 Fed. Reg. 18,769 n.3, simply does not explain why this proposed action  -  to cut off access to studies based on non-public data - is justified. The cases, upholding earlier Agency decisions not to release sensitive data underlying regulatory decisions, were decided based on a hard look at the statute, on grounds "that `the CAA imposes no such obligation' and that `requiring agencies to obtain and publicize the data underlying all studies on which they rely would be impractical and unnecessary.'" Nor is EPA's suggestion that various third-party independent organizations have called for "open science" at all persuasive, as discussed below. This issue has been debated  -  and transparency has improved, over many years  -  but even commentators in the initial rounds of the conversation, noted that there is no empirical evidence supporting the idea that there is a "bad science" problem.
The commenter further states that the proposal is impractical and unnecessary because existing laws and guidelines provide for controlled release of underlying data to permit reanalysis where there are questions about study results.
Comment [6895-3151]: Commenter (6916) notes that, by statute, the public may not access raw underlying data - or protected intellectual property. To the extent that the Proposal would, if finalized, require the public release of such data, it is unlawful. 
EPA's CAA rulemaking history illustrates that the Proposal is not necessary: scientific results and methods can be questioned and validated without exposing protected underlying data to public scrutiny.
The Proposal seeks comment on whether its new public data release paradigm should be applied even to "dose response data and models...developed prior to the effective date." To be clear (which the Proposal is not) this would have the potential effect, on CAA rulemakings, of prohibiting the use of the well-developed dose response information resulting from two landmark studies, the Harvard Six Cities Study and the American Cancer Society Study of Particulate Air Pollution and Mortality, that have been called "the poster children of good-science reforms." We strongly oppose such an outcome, as it would remove from EPA's rulemaking toolbox the "best available science" on the link between air quality and health. That is both unnecessary and unjustified.
The Proposal's focus on public release of the underlying data used in studies  -  particularly "dose response data" and studies, evinces a clear lack of understanding of the way science is conducted and verified. While replicating the results of a study with the same data set certainly verifies it, as the National Research Council said in 2002, "[f]or large epidemiological studies, repeating a study is seldom either possible or desirable." More effective, instead, is some combination of new studies testing the same question in different places, and/or an independent analysis with the same data. Formal peer review, whether or not prior to publication, also validates the outcome of scientific studies, as reflected in the OMB Guidelines presumption of objectivity and usefulness for scientific studies that have undergone formal peer review, described above. 
An example of this is the series of studies, re-analyses, verifications, and extensions of work begun in 1974 on the prospective "respiratory health effects of respirable particles and sulfur oxides on a sample of adults and children in six U.S. cities." This work, first undertaken by researchers at the Harvard School of Public Health, became known as the Harvard Six Cities Study. The Six Cities Study, published in 1993, tracked 20,000 people in six U.S. cities for two years and found a definitive link between PM pollution and serious health effects, including death. The American Cancer Society Study, published in 1995, linked air pollution to cardiovascular disease, respiratory disease, and lung cancer. Since their publication the studies have been reanalyzed and extended and consistently validated. EPA has found that these additional studies provide "consistent and stronger evidence of an association with mortality at lower air quality distributions than had previously been observed."
Some of the early peer reviewed Harvard Six Cities work was relied on by EPA in setting the PM NAAQS in 1987. Industry challenged the 1987 24-hour PM standards in part by questioning that work. The court held that EPA must "take into account all the relevant studies revealed in the record," and noted the precautionary nature of the CAA. 
The Harvard researchers and others continued to explore the effects of ambient PM exposures on human health and mortality, in the six cities, and various other places around the country and the world. As those results came out, industry-sponsored researchers again challenged them, in professional meetings and journals, arguing that because they could not reproduce or use the (confidentially provided) human health data set, the results could not be relied on. They claimed to come up with different results using other data. 
The HEI was brought in to manage an evaluation of the studies. HEI, funded by industry and EPA, is an independent arbiter, which chose a new team of scientists to reconstruct the Six Cities data set and verify the original results. Subsequently, over 100 other studies were peer-reviewed and published by 2004, building on, expanding, corroborating, and verifying the relationship between PM exposures and adverse respiratory health effects and even early death. These included follow up using "the sample of adults in the Harvard Six Cities Study," that found an approximately two-year reduction in life expectancy for those living in the most polluted areas.
These results, and the methodologies underlying them, were then validated by the original Harvard Six Cities researchers, in collaboration with a team funded by the American Cancer Society, using new cohort data  -  a combined data sample of more than 500,000 adults. The collaborative work was peer-reviewed and published. That work then became the focus of new debate over the 1997 PM NAAQS standard-setting, including new industry attacks based on industry's asserted lack of access to the health data in which they were grounded  -  raising the same questions that are still underlying this 2018 Proposal  -  namely, arguing that EPA needed to make public the data underlying any study on which it relied. Again, however, HEI was brought in to validate the results, and the resulting work was published in 2000. The D.C. Circuit Court reviewing the 1997 standard determined in 2002 that releasing the data was neither required by the Act, nor was it practical or necessary.
 The HEI review validated the previous work, and "demonstrated the robustness of the PM-mortality risk estimates to many alternative model specifications....[and] also made a number of innovative methodological contributions that...substantially contributed to subsequent analyses."
Comment [6895-3185]: Commenter (6916) states that the EPA's Proposal to limit the scientific basis for the Agency's decision making is not authorized by the CAA.
EPA, like all federal agencies, is a creature of statute, and "may act only pursuant to authority delegated ... by Congress." But the Proposal fails to state clearly under what authority EPA is proposing these rules, providing only a list of statutes that require the Agency affirmatively to take scientific information into account, but not explaining why they would offer the Agency authority to disregard certain relevant studies.  Indeed, EPA seems unsure of its authority, requesting comment on "additional or alternative sources of authority." 
Nor does the EPA Administrator's authority "to prescribe such regulations...as are necessary to carry out the Agency's functions" present an opening to disregard relevant information, as EPA suggests. To the contrary, courts have held that the Administrator must take into account all relevant studies in the record. Other CAA sections, not cited by the Agency, lay out and define some of the Administrator's rulemaking "functions under the Act" as requiring for example, the use of "the latest scientific knowledge useful in indicating the kinds and extent of all identifiable effects on public health or welfare..." So, the Proposal to permit him to disregard some studies is contrary not only to his general rulemaking authority, but to his specific authority to, inter alia, set NAAQS based on the "latest scientific knowledge."
The seminal CAA air quality standard-setting provisions have twice been held not to require the release of data underlying studies relied on by the Agency, as discussed above. Indeed, in both situations, the court has described such release as "unnecessary and unjustified."
The Proposal also is inconsistent with the overall precautionary nature of the CAA. Congress did not intend EPA to limit itself to considering only `sure things' but asks the Administrator to protect the public against uncertain dangers due to exposure to air pollution. Primary NAAQS are to be set with an "adequate margin of safety," meaning that "the Administrator need not regulate only the known dangers to health but may `err' on the side of overprotection." This precautionary approach is reflected also in the various technology-forcing sections of the Act, which Senator Muskie said "may [require] that people and industries will be asked to what seems to be impossible at the present time." Under this scheme, the Administrator must consider all relevant studies placed in the record, and "may apply his expertise to draw conclusions from suspected, but not completely substantiated, relationships between facts, from theoretical projections from imperfect data, from probative preliminary data not yet certifiable as `fact' and the like," in order to protect public health as Congress intended. This context simply does not authorize the Administrator to decline to consider any studies  -  but particularly not those representing the best available science  -  because the underlying data is confidential.
Comment [6895-3204]: Commenter (6916) states that CAA section 307(d)(6)(B) requires that when finalizing a proposed rule, and after taking comment, an Agency must "respon[d] to each of the significant comments, criticism, and new data submitted in written or oral presentations during the comment period." And, while both proposed and final rules under the CAA must be accompanied by a statement of basis that summarizes studies and data on which the Agency relies, in no way does that requirement authorize the public release of the raw data. Nor does the Act's requirement that the record underlying rules include a "summary of ... the factual data on which the ... rule is based," require the public release of such data. Nor does the Act elsewhere require or allow data to be released to the public. Where "new data" is submitted by commenters, that must be responded to by the Agency and that response made part of the record, but the statute does not require public release of new data either. Indeed, Agency proposals, including this one, describe measures by which commenters can submit so-called CBI  -  which the Agency can consider but protect from public release. 
Therefore, to the extent that the Proposal envisions authorizing EPA to ignore or disregard or not to consider some studies submitted during a rulemaking comment period, whether or not the data is publicly released, such agency action would be taken in direct violation of the CAA. Nor is there any basis elsewhere in the CAA for EPA to decide to exclude or prohibit the consideration of otherwise relevant matter because it is based on confidential information not available to the general public.
The Proposal does not meet CAA procedural requirements. As noted supra, the CAA requires, with any proposed rule, a summary of "the factual data on which the proposed rule is based; the methodology used in obtaining the data and in analyzing the data; and the major legal interpretations and policy considerations underlying the proposed rule." But this Proposal is not accompanied by any record in support. Nor has the Agency even attempted to explain in the preamble why current statutes and guidelines do not adequately provide for the public to question scientific studies, seek validation, replication (or reproducibility) of study results, and to comment on methodologies, without "public" release of confidential data. It therefore does not satisfy the CAA's requirements or provide the public with a meaningful opportunity to fully comment on the proposal.
Comment [6895-3227]: Commenter (6916) contends that the Proposal does not clearly identify the authority under which EPA has promulgated it. As discussed above, the CAA does not authorize the Proposal, which is clearly aimed at limiting the studies EPA can rely on and consider in CAA rulemaking. Certainly that is the case for the CAA sections listed by the Agency in the Proposal, including the general rulemaking authority under the Act which requires the evaluation of all material in the record. EPA's suggestion that it has "general authority" to make rules to comply with its obligations is similarly unavailing here, as the general authority does not exist to make rules that are outside EPA's statutory obligations, or in violation of them  -  and a rule that would enable EPA to disregard relevant studies in a CAA rulemaking record is clearly outside the Agency's authority. Nowhere does the Act permit EPA to limit its own access to relevant science. 
The Proposal's terms are so vague as to fail to afford an opportunity for meaningful comment. Significant issues are presented in the Proposal only in the broadest terms, and with enough specificity "to permit interested parties to comment meaningfully." As noted supra, we assume (as we must, given no other information from the Agency), that the Proposal is aimed at precluding EPA's reliance on studies for which the data are not made public. That is the natural reading of the proposed regulatory language, which is the most choate information offered in the Proposal. But even that reading is not certain, given the very broad discretionary nature of the exemptions provided for in the rule text (see, e.g., 83 Fed. Reg. 18,774, proposed 40 C.F.R. § 30.9, giving the Administrator the ability to grant an exemption to the data publication requirement if that is "impracticable," which is undefined in the Proposal). As a result, it is unclear how this rule would apply to EPA decision making, and how it would be different than the Agency's current protocol for considering studies relevant to specific rulemaking proposals. 
As one example, EPA alludes to "transparency" multiple times, but without defining what that concept means, or explaining why it believes existing statutes, regulations, guidelines and protocols fail to meet it. While asserting an interest in "ensur[ing] that the data and models underlying science is [sic] publicly available in a manner sufficient for validation and analysis," there is no more specific statement explaining how that would be implemented or why releasing confidential health and business information is necessary. Nor is the Agency's rationale completely clear in the Proposal. When commenters are left to guess, that is not lawfully sufficient notice of the subjects and issues involved. 
Comment [6895-3296]: Commenter (6916) states that the EPA has failed to follow required and customary rulemaking process in issuing this Proposal. The rulemaking process set forth in the APA, and in more detail in the CAA, provide a Congressionally-directed framework to assure that the Agency's rulemaking decisions are not arbitrary  -  that they are made transparently and based on an adequate record that is subject to public scrutiny. Similarly, the Environmental Research, Development and Demonstration Authorization Act of 1976 (ERDDAA) requirement to submit CAA regulatory proposals to SAB review serves as a check to ensure that the scientific basis underlying the rule is robust.
As discussed above, however, EPA failed to submit the rule to its SAB, and additionally did not prepare a regulatory impacts analysis of the Proposal as required by Executive Order 12,866. Nor did the Administrator provide sufficient time for a meaningful OMB Office of Information and Regulatory Affairs (OIRA) review, having signed the rule on April 24, 2018 after having submitted the Proposal to OIRA on April 19, 2018  -  despite the requirement to provide OIRA with at least 10 working days to decide whether to waive review. 
EPA's failure to subject the Proposal to these required analyses denies the public "transparency"  -  because information is withheld on which the affected public might base meaningful comments  -  as well as denying OMB the information it needs to complete its reviews.
EPA's Proposal claims to be "consistent with" Executive Order 13,777, which ordered a Regulatory Reform Task Force to identify regulations that "rely in whole or in part on data, information, or methods that are not publicly available or are insufficiently transparent to meet the standard for reproducibility," and 13,783, which required a focus on energy independence, but also the "achieve[ment of] environmental improvements for the American people." But the Executive Order 13,777 Task Force report does not identify any regulations that rely on insufficiently transparent data, nor is this Proposal - which would deny access to scientific studies - consistent with the goal of advancing environmental improvements for the American public.
Comment [6897-1893]: Commenter (6918) supports the Proposed Rule's underlying values, which originate from EPA's statutory authority and principles of good government.
Transparency, validity, and public participation are especially important when EPA issues rules under the CAA. Section 307(d)(3) of the Act, which applies to certain important categories of EPA rules as well as to other rules to which the EPA Administrator decides to apply it, imposes even more stringent requirements than does the APA, 5 U.S.C. § 553, which specifies general requirements for notice-and-comment rulemaking. Specifically, CAA section 307(d)(3) requires:
      In the case of any rule to which this subsection applies, notice of proposed rulemaking shall be published in the Federal Register, as provided under section 553(b) of Title 5, shall be accompanied by a statement of its basis and purpose and shall specify the period available for public comment . . . . The notice of proposed rulemaking shall also state the docket number, the location or locations of the docket, and the times it will be open to public inspection. The statement of basis and purpose shall include a summary of -- 
      (A) the factual data on which the proposed rule is based;
(B) the methodology used in obtaining the data and in analyzing the data; and
(C) the major legal interpretations and policy considerations underlying the proposed rule.
As the D.C. Circuit has explained, section 307(d) requires EPA to issue a proposed rule and to provide a detailed explanation of its reasoning at the proposed rule stage. See, e.g., Small Refiner Lead Phase-Down Task Force v. EPA, 705 F.2d 506, 519 (D.C. Cir. 1983). CAA § 307(d)(3) "requires a much more detailed notice of rulemaking" than the APA. Id. at 518; see also Union Oil Co. v. EPA, 821 F.2d 678, 681-82 (D.C. Cir. 1987) (noting that Congress's 1977 amendments to the CAA "provide new procedural requirements for EPA rulemaking" that are "more stringent than those previously applicable under the Administrative Procedure Act," including a requirement that "EPA . . . establish a detailed `rulemaking docket' that . . . must provide the entire basis for the final rule.") (quoting 42 U.S.C. § 7607(d)(3) (1982)). The public process values that underlie notice-and-comment rulemaking, both under the CAA and under the APA, amply justify the notice requirement. If poor notice is given, "[t]he agency will not get the informed feedback it needs, the parties will feel unfairly treated, and there will be a meager record for [the courts] to review." Small Refiner Lead Phase-Down Task Force, 705 F.2d at 550.
The Proposed Rule represents another instance of EPA seeking to build on its existing policies, principles, and statutory obligations. EPA recognizes that "[m]eaningful public participation requires much more than simply holding public meetings or hearings or collecting public comment." And, consistent with EPA's long history of seeking to increase transparency, promote public participation, and ensure validity of its rules, increasing access to underlying data and models will help satisfy EPA's specific obligations under the CAA.
Comment [6901-4441]: Commenter (6922) provides that, in the CAA: 
      The Administrator shall issue air quality criteria for an air pollutant within 12 months after he has included such pollutant in a list under paragraph (1). Air quality criteria for an air pollutant shall accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air, in varying quantities.
Comment [8750-1431]: Commenter (9227) asserts that, not only is there no authority for EPA's pan-statutory Proposal, but the Proposal would also violate explicit statutory commands. Though EPA admits that "[t]he best available science must serve as the foundation of EPA's regulatory actions," proposed section 30.5 would prohibit EPA from considering high quality and critically important scientific studies -- precisely that "best available science" -- when undertaking regulatory actions. Specifically, section 30.5 would prevent EPA from considering any scientific study for which the underlying "dose response data and models" are not "publicly available in a manner sufficient for independent validation." The commenter notes that in CAA, 42 U.S.C. § 7408(a), EPA shall establish air quality criteria that "shall accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air, in varying quantities."). And, by excluding science that meets these statutory criteria from supporting regulations to protect public health and welfare, the Proposal would frustrate Congress's policy in these statutes and frustrate EPA from achieving its fundamental mission.
Comment [8750-1432]: Commenter (9227) states that the EPA's statutory authorities generally require the agency to consider all available data when undertaking significant rulemakings. The EPA's statutory authorities mandate a variety of requirements for what scientific information EPA must consider in rulemaking. These statutes are discussed in detail, infra at Section I.B.3. To take one example that appears in numerous statutes, including TSCA, CAA, SDWA, and the ESA, Congress has often required agencies to act on the "best available science." For an agency to comply with this obligation, the agency must at least consider all available scientific information. "Best" means "of the most excellent, effective, or desirable type or quality." "Available" means "able to be used or obtained." And "science" means "the intellectual and practical activity encompassing the systematic study of the structure and behavior of the physical and natural world through observation and experiment." Assessing which science is "best" requires consideration of the overall quality of the science, and the public availability of underlying data is, at best, one of many aspects that should inform that assessment of overall quality.
An agency "cannot ignore available. . . information." Numerous courts have indicated that a plaintiff or petitioner can establish a violation of the "best available science" requirement by "point[ing] to any scientific evidence that the agency failed to consider." "The best available data requirement. . . prohibits [an agency] from disregarding available scientific evidence that is in some way better than the evidence [it] relies on.'" "An agency does. . . have an obligation to deal with newly acquired evidence in some reasonable fashion." EPA's proposal will result in EPA precluding itself from considering certain studies that are "available," thus violating the requirement that EPA rely on the best available science.
In addition, the requirement that agencies use "best available" science or information often means that the agency must act even if the available science or information is imperfect.
"Even if the available scientific and commercial data were quite inconclusive, [the agency] may -- indeed must -- still rely on it" when the agency has a duty to act. "[W]here the information is not readily available, we cannot insist on perfection." Just as the Courts have recognized that they cannot expect perfection, agencies cannot choose to ignore certain studies or sources of information based solely on whether the data is publicly available -- especially where the validity of those studies has been established using techniques that do not rely on public availability of underlying data.
EPA cannot reasonably elevate the interest in public availability of all underlying information above all other factors in assessing the "best available science." Textually, EPA's approach is unlawful.
Comment [8750-1448]: Commenter (9227) notes that, as EPA acknowledges in footnote 3 of the Proposal, in at least two instances the D.C. Circuit Court of Appeals has recognized that studies for which underlying data is not publicly available may constitute "best available science." The D.C. Circuit's decisions in these cases further demonstrate that the public unavailability of a study's underlying data does not render a study incapable of constituting "best available science" otherwise unworthy of EPA's consideration.
In American Trucking Associations v. EPA, the petitioner challenged EPA's reliance on scientific studies for which underlying data was not publicly available in deciding to strengthen the NAAQS for PM. The Court held that the CAA did not require EPA to make public underlying data where EPA relied on the study itself and not the raw data underlying the study. The Court agreed with EPA's position that requiring agencies to obtain and publicize the data underlying all studies on which they rely "would be impractical and unnecessary." Importantly, the Court concluded that:
If EPA and other governmental agencies could not rely on published studies without conducting an independent analysis of the enormous volume of raw data underlying them, then much plainly relevant scientific information would become unavailable to EPA for use in setting standards to protect public health and the environment. . . . Such data are often the property of scientific investigators and are often not readily available because of. . .proprietary interests. . . or because of [confidentiality] arrangements [with study participants].
The court accordingly recognized that ignoring relevant scientific information simply because the underlying data is not available would violate EPA's obligations to consider "best available science." Coalition of Battery Recyclers Association v. EPA involved another challenge to EPA's reliance on a scientific study for which the underlying data was not publicly available.In that case, EPA had relied upon the study in question to determine the "concentration-response relationship between blood lead levels and IQ changes." The D.C. Circuit again upheld EPA's reliance on studies without making the underlying data publicly available and explained, "raw data often is unavailable due to proprietary interests of a study's scientific investigators or confidentiality agreements with study participants." Likewise, in City of Waukesha v. EPA the D.C. Circuit concluded that agency peer review satisfies the requirement to use best, peer reviewed science and supporting studies.
Comment [8750-1453]: Commenter (9227) notes that, by prohibiting EPA from considering all valid and relevant studies when undertaking significant rulemakings, the proposed rule would prevent EPA from complying with an array of statutory provisions governing EPA's consideration of available science.
Under CAA section 108(a), EPA must establish air quality criteria for each air pollutant that serves as the basis for setting the NAAQS. Such criteria "shall accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air, in varying quantities." As explained above, the scientific community, EPA, and the courts have all concluded that lack of public availability of underlying data does not render the study invalid. And consideration of such studies can be essential for EPA to fulfill CAA section 108(a)'s directive that it consider "the latest scientific knowledge" in establishing air quality criteria, that it consider studies "useful" in indicating effects of pollutants on ambient air, and in providing an adequate margin of safety in the standard itself. Thus, EPA's proposal to bar EPA from considering such studies would prevent EPA from complying with its statutory obligation under CAA section 108(a).
Section 108(a)(2) says nothing about excluding information -- its evident purpose is to be inclusive as to information to be considered. EPA's historic practice reflects this broad directive: each NAAQS review evaluates virtually all studies in the area, excluding none, but assigning appropriate weight based on study-by-study evaluation. Since the NAAQS provisions were enacted in 1970, EPA has conducted many NAAQS rulemakings. The agency does not establish per se, a priori rules regarding study inclusion or exclusion, but rather evaluates each of the individual studies -- and there are thousands typically evaluated for each NAAQS review -- on their merits based on reasoned criteria. While details of the development and review of the criteria and standards have evolved over time, in practice, EPA has endeavored to include all relevant scientific studies in the process, even providing provisional assessments of relevant literature that appears after the formal scientific review has been completed. Over the years, tens of thousands of peer-reviewed studies of health effects, exposure, and atmospheric interactions, and monitoring have been included in reviews of criteria and standards. A requirement that they must be excluded from consideration unless the raw data and full methodologies are made available for all of them is inconsistent with the legislative mandate and EPA's practice over the last 40 years.
Thus, a science regulation that applies to the NAAQS is unlawful unless EPA can show that the new standard can be established and implemented consistent with the applicable statutory requirements. To do so, EPA must prove that public unavailability of data means that a study does not constitute "latest scientific knowledge useful" in indicating effects on human health or welfare. EPA's Proposal neither acknowledges this requirement nor explains how the Proposal would not violate this statutory command.
For example, in past NAAQS reviews, EPA has considered the Harvard Six Cities study and American Cancer Society studies, despite the fact that the data underlying these studies is not publicly available. These studies, however, are plainly "useful in indicating the kind and extent of all identifiable effects on public health or welfare." These seminal studies have been part of the air quality criteria since the mid-1990s -- they have thus been accepted as "useful" by separate panels of CASAC, and by EPA, in three separate NAAQS reviews. Their use has been upheld by the D.C. Circuit. Both studies have been reanalyzed and validated by highly competent third-party reviewers (the HEI) with access to the underlying data. The study results have been reproduced many times over. Extended follow-up analyses of the ACS and Harvard Six Cities studies provide consistent and stronger evidence of an association with PM 2.5 and mortality at even lower air quality distributions than had previously been observed. This type of cumulative weight of evidence is highly probative in assessing both causality and in establishing the level of the NAAQS. The proposal says almost nothing about any of these other attributes that not only make these studies "useful," but indeed make them particularly high quality and reliable. The primary ozone NAAQS provides further examples of the pernicious effects the proposal would have. Among the key controlled human exposure studies demonstrating that exposure to ozone causes adverse health effects in even healthy subjects at levels below the level of the then-current NAAQS are Adams (2006) and Schelegle (2009). These studies were sponsored by the American Petroleum Institute, which controls access to the underlying data. The American Petroleum Institute refused an EPA researcher access to the data of a related Adams study it sponsored (Adams (1998)). So not only would these evidently "useful" (under CAA section 108(a)(1)) studies be barred from consideration under the Proposal, but the Proposal creates a perverse incentive for industry to refuse access to study data. The published studies --  peer reviewed -- would obviously be providing information "useful" in indicating effects of air pollution, but the Proposal would not only bar their consideration but create an incentive for industry never to provide underlying data for any industry-sponsored study with a result not to industry's liking.
The most recent premiere long-term cohort study for PM is Domenici (2017) which found even greater effects of fine particles at levels below EPA's current standards. This study used a Medicare database available to any research group that can guarantee confidentiality of personal data. Yet the proposal could evidently bar consideration of this powerful study.
NAAQS must be requisite to protect the public health, and to provide an "adequate margin of safety" in doing so. The proposal violates this central statutory requirement. NAAQS are required to provide this margin of safety "to build a buffer to protect against uncertain and unknown dangers to human health." EPA's Proposal would build a buffer against using the very studies necessary to guard against these dangers.
Comment [8750-1770]: Commenter (9227) contends that the proposed rule fails to meet even the most basic procedural and substantive obligations. The APA requires that the "opportunity for comment must be a meaningful opportunity," and "[t]hat means enough time with enough information to comment and for the agency to consider and respond to the comments." Prometheus Radio Project v. FCC, 652 F.3d 431, 450 (3d Cir. 2011) (internal citation and quotation marks omitted). See also Am. Hosp. Ass'n v. Bowen, 834 F.2d 1037, 1044-45 (D.C. Cir. 1987) (noting the "obvious importance of the [APA's] policy goals of maximum participation and full information."). For its part, the CAA "requires a much more detailed notice of proposed rulemaking than does the APA." Union Oil Co. of Cal. v. EPA, 821 F.2d 678, 682 (D.C. Cir. 1987); see Small Refiner Lead Phase-Down Task Force v. EPA, 705 F.2d 506, 550 (D.C. Cir. 1983) ("[T]he additional notice requirements in § 307(d)(3) suggest that Congress intended agency notice under the CAA to be more, not less, extensive than under the APA."). Executive Order 13563 underscores these obligations requiring that to promote "open exchange of information and perspectives among State, local, and tribal officials, experts in relevant disciplines, affected stakeholders in the private sector, and the public as a whole," agencies "shall endeavor to provide the public with an opportunity to participate in the regulatory process."
Moreover, notice has to be provided by the agency; it cannot be bootstrapped from the public comments. The reasons are evident: there is no requirement for parties to monitor all of the thousands or tens of thousands of submitted comments in order to guess the issues on which to comment. A contrary rule "would turn notice into an elaborate treasure hunt, in which interested parties, assisted by high-priced guides (called `lawyers'), must search the record for the buried treasure of a possibly relevant comment."
Drafting these comments has entailed a great deal of guesswork. The comments of EDF or any other commenter on a particular issue thus should not be taken to mean that EPA provided sufficient notice of that issue.
The proposed rule lacks essential elements needed to understand it, rendering the opportunity for comment meaningless. The Proposal contains vague and contradictory statements about its actual substance and effect, fails entirely to analyze and disclose its costs and benefits, and is littered with vague references to entire websites and executive branch departments. The cursory reasoning and wholly inadequate record offered in support of the proposed rule prevents stakeholders from engaging with the agency on its rationale for the proposed action and its costs and benefits or offering contrary evidence. Finally, EPA has not provided any basis whatsoever to warrant the gross inadequacies of the proposed rule and the process to consider it. With such a deeply deficient basis for action, the only legally viable course is to withdraw the Proposal.
Comment [8750-1777]: Commenter (9227) Section 307(d) applies to "such. . . actions as the Administrator may determine." EPA claims to take this action under "authority of the statutes it administers. . . including CAA sections 103, 301(a)." By issuing this Proposal through notice and comment procedures, Administrator Pruitt appears to have determined that 307(d) procedures apply.
Even without that invocation, the proposed rule is subject to these procedural requirements because it materially impacts many of the actions delineated in 307(d)(1) to which the CAA rulemaking procedures explicitly apply. The Proposal applies to "significant regulatory actions," which many of these actions are. The CAA requires science-based decision-making that the Proposal will materially affect. For example, by restricting the science EPA may rely on in regulatory actions, the Proposal materially impacts residual risk determinations for HAP (§ 307(d)(1)(C)), standards for mobile source air toxics (§ 307 (d)(1)(K)), and residual risk standards for municipal solid waste combustors (§ 307(d)(1)(D)).
This proposed rule directly affects EPA's setting and review of NAAQS, the promulgation or revision of which is subject to the CAA rulemaking requirements. Section 108(a) of the CAA requires the Administrator to set air quality criteria for air pollutants that "reflect the latest scientific knowledge." This Proposal amends the science EPA can consider for air quality criteria. Under CAA section 109 EPA must use the air quality criteria to set primary and secondary NAAQS and periodically review them --  which EPA is currently doing for PM. In the Proposal, EPA cites Am. Trucking Ass'ns v. EPA, 283 F.3d 355, 358 (D.C. Cir. 2002) as an example of an instance where EPA relied on a scientific study where the underlying data was not publicly available. EPA states that under the Proposal use of such science would be "preclude[d]" In Am. Trucking Ass'ns the Court upheld EPA's use of key studies underlying the NAAQS for PM. Under the Proposal, EPA would not have been permitted to use those studies, and it is unclear how the Proposal will affect EPA's reliance on these studies as it undertakes its review. This demonstrates how this Proposal would have an immediate impact on EPA NAAQS-setting under the CAA. EPA is thus subject to the CAA 307(d) procedural requirements for this Proposal.
Comment [8750-1778]: Commenter (9227) asserts that the EPA has failed to provide a properly developed record in support of the proposed rule. EPA has not identified sufficient supporting evidence in the Proposal or in its docket and has failed to provide adequate notice of the supporting evidence for the public to respond to meaningfully, as the APA, the CAA, and other substantive statutes require.
Rulemaking under the CAA is subject to the same general requirements of statutory conformity and reasoned decision-making derived from the APA and basic principles of administrative law. CAA rules cannot be "arbitrary, capricious, an abuse of discretion, or otherwise not in accordance with law," "in excess of statutory jurisdiction, authority, or limitations, or short of statutory right," or "without observance of procedure required by law."
The commenter notes that the citations for support in the Proposal are vague and uninformative, and even where the particular citation can be identified and located, it is often not clear how EPA thinks the citation supports the Proposal. This does not meet the standards of the APA and CAA.
Additionally, EPA has failed to meet the docket requirements of the CAA. CAA section 307(d)(3) requires that publication of the proposed rule in the Federal Register include a summary of the factual data on which the proposed rule is based, the methodology used in obtaining the data and in analyzing the data, and the major legal interpretations and policy consideration underlying the proposed rule. It also requires the agency to place "[a]ll data, information, and documents. . . on which the proposed rule relies" in the rulemaking docket on the date of publication of the proposed rule. The undifferentiated citation of articles and policies, most of which contradict the Proposal or otherwise offer no support for it, fails abjectly to satisfy these requirements. Any document that becomes available after the proposed rule has been published and that is of central relevance to the rulemaking must also be placed in the docket as soon as possible after its availability. The agency must allow enough time for participants in the rulemaking to respond to those documents with comments.
As of the date of the publication of the Proposal, the docket at regulations.gov contained only the following 12 documents: (1) OIRA Review Start Document (Apr. 17, 2018); (2) OIRA Review Conclusion Document (Apr. 23, 2018); (3) White House Memorandum on Scientific Integrity (Mar. 9, 2009); (4) Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies; Republication, 67 Fed. Reg. 8,452 (Feb. 22, 2002); (5) Exec. Order 13,777, Enforcing the Regulatory Reform Agenda, 82 Fed. Reg. 12,285 (Feb. 24, 2017); (6) EPA, Plan to Increase Access to Results of EPA-Funded Scientific Research (Nov. 29, 2016); (7) OMB Memorandum M-05-03 on Issuance of OMB's "Final Information Quality Bulletin for Peer Review" (Dec. 16, 2018); (8) EPA, Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the EPA (Oct. 2002); (9) Exec. Order 13,563, Improving Regulation and Regulatory Review, 76 Fed. Reg. 3,821 (Jan. 18, 2011); (10) Exec. Order 16, 093, Promoting Energy Independence and Economic Growth, 82 Fed. Reg. 16,093 (Mar. 28, 2017); (11) OMB Memorandum M-13-13: Open Data Policy-Managing Information as an Asset (May 9, 2013); (12) Commission on Evidence-Based Policymaking, The Promise of Evidence-Based Policymaking (Sep. 2017).
This clearly is not enough to meet the CAA's requirements. Aside from the drafts of the proposed rule submitted to OIRA, each of these documents was a pre-existing memorandum, policy document, or executive order that contains no specific analysis -- factual, legal, policy or otherwise -- that pertains to the impacts of or at all justifies this proposed rule. While EPA in the proposed rule cites to some of these documents as purportedly being consistent with these prior policies, see, e.g., 83 Fed. Reg. at 18,769-70, as the discusses elsewhere in their comment letter, these policies do not in fact provide any basis for the Proposal. The record that EPA provides clearly fails to support its proposed action. Some of the factual data, legal interpretations, and policy considerations that EPA has not sufficiently provided evidence for include: the number of scientific studies that would be precluded from consideration under the Proposal; whether there are fields of research where the Proposal would result in insufficient scientific information available for EPA to meet its statutory duties; how EPA will address the substantial privacy concerns implicated by the Proposal; how application of this Proposal will impact substantive agency actions; what the costs of implementing this Proposal are if EPA intends to not just exclude studies from consideration where too costly to provide access, etc.
EPA, for instance, includes Executive Order 13,563 in the docket to support its statement that "[t]he best available science must serve as the foundation of EPA's regulatory actions." While Executive Order 13,563 makes that statement, it does not support EPA's Proposal, which as explained above, hinders EPA's use of the best available science. EPA provides no evidence or explanation in the docket or Proposal for why EPA believes this policy would further that goal. The executive order only states that agencies should make available to the public the scientific or technological findings or conclusions on which rules rely, as opposed to underlying raw data that EPA has targeted with this Proposal. Meanwhile, EPA blatantly violates the executive order's provisions requiring agencies to weigh costs and benefits; to write regulations that are easy to understand; and to provide the scientific and technical findings underlying the rule for the public to comment on.
Section 307(d)(3) of the CAA requires that "[a]ll data, information, and documents ... on which the proposed rule relies shall be included in the docket on the date of publication of the proposed rule." Many items that EPA cites to in the Proposal as providing a basis for the proposed rule do not appear in the docket. For example, EPA states: "The proposed rule takes into consideration the policies or recommendations of third party organizations who advocated for open science." In a footnote, EPA provides: "These include policies and recommendations from: The Administrative Conference of the United States' Science in the Administrative Process Project; National Academies' reports on Improving Access to and Confidentiality of Research Data, Expanding Access to Research Data, and Access to Research Data in the 21st Century; the Health Effects Institute; Center for Open Science; members of the Risk Assessment Specialty Section of the Society of Toxicology, the Dose Response Section of the Society for Risk Analysis, and the International Society for Regulatory Toxicology and Pharmacology; and the Bipartisan Policy Center's Science for Policy Project." Many of these policies and recommendations did not appear in the docket on the data of publication of the Proposal and still do not appear in the docket -- a clear violation of the CAA -- nor are the specific documents or reports even identified or properly cited so that they may be tracked down. This is evidently prejudicial to commenters -- it undermines commenters ability to submit meaningful feedback when the agency is hiding the ball in this manner.
These policies and recommendations are not easily identifiable on their own either, even after significant internet research. This is also true of footnote 16, where EPA lists a number of agencies to support its claim that the federal government is already implementing solutions to data disclosure. EPA cites, for example, the National Institute of Standards of Technology. NIST has numerous policy documents on protecting privacy concerns and keeping data secure as well as its own internal policies on releasing data. It is hard to see how any are relevant here, but without a particular cite the public is denied even a chance to respond to whatever EPA is trying to use as support -- or must respond to everything that might be being referenced, creating a burdensome task. Throughout these comments, as we attempt to respond to EPA's Proposal, we have been very practically limited by our inability, even after much research and consideration, to be fully certain we have identified the appropriate policies to respond to. This presents a situation that the CAA's docket requirement was exactly formulated to prevent. 
On May 25, 2018, EPA added a memorandum to the docket for this rulemaking. This memorandum contains hyperlinks apparently intended to accompany various citations in the footnotes of the Proposal. This document does not cure the former procedural defect, as the CAA requires information the proposed rule relies on to be placed in the docket on the day the proposed rule is published. Further, these hyperlinks still link ambiguously to various documents and agency websites without providing any information about what specifically EPA intends to cite or how the cited information is being used or considered by EPA. Additionally, simply adding such a document to the docket does not provide adequate notice to the public. Someone who had access only to the proposed rule and was not carefully monitoring the docket would have no indication or notice of this new document. Either EPA is failing to comply with the CAA's requirements by failing to include in the docket factual data, legal interpretations, and policy considerations that support the Proposal, or these supporting items do not exist, deeming this rulemaking completely arbitrary -- in either case the Proposal fails to meet the standards of the APA and CAA. Under the CAA the rulemaking docket "must provide the entire basis for the final rule and the exclusive record for judicial review," this docket clearly cannot support a final rule.
Comment [8750-1804]: Commenter (9227) states that the CAA requires that the Federal Register notice be accompanied by a statement of basis and purpose that includes a summary of the factual data on which the proposed rule is based, the methodology used in obtaining the data and in analyzing the data; and the major legal interpretations and policy considerations underlying the proposed rule. As discussed above, all data, information, and documents on which the proposed rule relies must be included in the docket on the date of publication of the proposed rule.
These core requirements are "designed (1) to ensure that agency regulations are tested via exposure to diverse public comment, (2) to ensure fairness to affected parties, and (3) to give affected parties an opportunity to develop evidence in the record to support their objections to the rule and thereby enhance the quality of judicial review." In addition, "a chance to comment ... [enables] the agency [to] maintain[] a flexible and open-minded attitude towards its own rules,"] and "avoid[s] the inherently arbitrary nature of unpublished ad hoc determinations." The failure to include critical documents relevant to the proposed rule in the docket, as required by the CAA, itself constitutes a notice violation because "absence of those documents, or of comparable materials. . . makes impossible any meaningful comment on the merits of EPA's assertions." By failing to provide a more developed docket, EPA is frustrating the terms and purposes of these statute's notice requirements. These procedures are in place to form a "specific" proposal that can serve as a "focus for comments," Small Refiner Lead Phase Down Task Force v. EPA, 705 F.2d 506, 548-49 (D.C. Cir. 1983); see Home Box Office, Inc. v. FCC, 567 F.2d 9, 36 (D.C. Cir. 1977) (agency must "make its views known . . . in a concrete and focused form so as to make criticism or formulation of alternatives possible"). Because EPA has not provided supporting evidence, has not included key items it points to as major considerations underlying the Proposal, and has generally presented a vague and unspecified proposed rule and docket, EDF and the public are hindered in our ability to provide specific comment focused on the underpinnings of the Proposal, because we do not know and can only guess as to what they are.
Even the text of EPA's proposed rule and the statement of basis and purpose fails to provide the requisite notice to allow meaningful comment. At the most fundamental level, it contains vague and contradictory statements about the actual effect of the Proposal. The Proposal generally appears to make its requirements mandatory -- i.e., failure to make information publicly available will preclude the agency from relying on the study at all. See 83 Fed. Reg. at 18,769 n. 3 ("EPA is proposing to exercise its discretionary authority to establish a policy that would preclude it from using such data in future regulatory actions."); id. at 18,771 ("the regulatory text would impose requirements"); see also id. at 18,769 ("EPA will ensure that the data and models underlying the science is publicly available...") (emphasis added) and proposed section 30.5 ("When promulgating significant regulatory actions, the Agency shall ensure that does response data and models underlying pivotal regulatory science are publicly available in a manner sufficient for independent validation"). In a few places, however, the Proposal makes it sound as if its aims are more aspirational. See id. at 18,770 ("Where available and appropriate, EPA will use peer-reviewed information, standardized test methods, consistent data evaluation procedures, and good laboratory practices to ensure transparent, understandable, and reproducible scientific assessments.") (emphasis added); id. at 18,772 ("The proposed rule directs EPA to make all reasonable efforts to" make data publicly available, but "does not compel the Agency to make that information available where it concludes after all such reasonable efforts that doing so in way [sic] that complies with the law and appropriate protections is not possible.") (emphasis added); see also id. at 18,768 ("EPA should ensure that the data underlying those are publicly available...") (emphasis added). The difference between a requirement precluding use of science and making all best efforts to make data publicly available is enormous.
To the extent EPA intends to propose a rule that would preclude use of science, as it appears the Proposal would do, the proposed rule is further flawed because it contains no analysis of how that would affect regulations. How many studies does EPA typically rely on in promulgating regulations? What percentage of these would meet EPA's new requirements? For those that do not, how many could not meet these requirements for patient privacy, CBI, or other reasons? How would EPA set standards if it must rely on many fewer studies? Would EPA be precautionary in the face of less evidence? Would EPA delay promulgating regulations in order to comply with this new mandate? How does this mandate interact with statutory deadlines or statutory requirements that EPA look at a wide range of science? None of these very basic questions are addressed in the proposed rule and without answering them, it is impossible for the public to assess the import and likely consequences of the Proposal. Even more basically, the agency gives no notice as to the Proposal's impacts, its costs, its benefits, why it applies only to regulatory requirements but not to any regulatory actions (like licensing or permitting) that confer a benefit, substantive and procedural criteria for adjudicating waivers, or even the legal theory under which the Proposal issues -- the plaintive solicitation for comment as to "additional or alternative sources" of authority, 83 Fed. Reg. at 18771, does not suffice.
To the extent the Proposal is intended to solicit comment on how EPA may make reasonable efforts to make data publicly available it is also unlawfully vague. The proposed rule includes numerous footnotes referencing entire websites or even Departments of the Executive Branch. For example, the Proposal claims that "EPA believes that concerns about access to confidential or private information can, in many cases, be addressed through the application of solutions commonly used across some parts of the Federal government." To support this proposition, EPA remarkably cites (without any further elaboration or explanation in the proposal itself) to "examples from the U.S. Department of Health and Human Services, National Institute of Standards and Technology, U.S. Department of Education, and the U.S. Census Bureau." See Small Lead Refiner Phase Down, 705 F. 2d at 548 (requirement that comments are to raise issues with "reasonable specificity" applies equally to the agency giving notice). For example, it is not possible to identify whether the sources referenced support EPA's claim that there are approaches available to address the serious privacy issues raised by the Proposal --  without providing the specific policies and recommendations, a public commenter has no way of knowing whether they are consistent or why EPA believes them to be consistent. It is impossible to respond in a meaningful way without significant guesswork. 
Similarly, in footnote 10 of the proposal, where EPA lists a number of organizations whose "policies and recommendations" the Proposal allegedly took under consideration -- no explanation is provided. In addition, in the proposed rule EPA fails to adequately define key terms like "validation", "independence", "reproducibility," "replication," and "uncertainty," while also citing a "replication crisis" in science. It is important that these terms are defined clearly as these terms are not defined consistently across the scientific community nor governments -- which has implications for the scope and purview of the proposed rule.
This amount of information is wholly insufficient to allow a public commenter to provide meaningful comments about these issues.
Comment [8750-1807]: Commenter (9227) states that the Proposal impacts EPA's process for setting NAAAQs in material ways by amending the scientific information that can be used as air quality criteria. Under the CAA air quality criteria cannot be amended without review by the Clean Air Science Advisory Committee (CASAC). Thus, EPA must submit this proposal to CASAC for review, consider their recommendations, and provide reasonable explanation for deviation from those recommendations.
Comment [8750-1943]: Commenter (9227) contends that the EPA arbitrarily failed to consider the far-reaching implications this Proposal could have on inter-agency coordination and consultation given that other agencies normally rely on research potentially excluded by the Proposal and cites consultation provisions in the CAA.
Comment [8750-2635]: Commenter (9227) notes that the EPA cites CAA, § 301, 42 U.S.C. § 7601, as purportedly granting authority for the Proposal. The authority granted by section 301(a), however, applies only to the CAA and, in any event, is not broad enough to encompass this Proposal. Section 301 provides that "[t]he Administrator is authorized to prescribe such regulations subject to section 307(d) as are necessary to carry out his [or her] functions under this Act." The courts have consistently "decline[d] to read ... open-ended power into section 301," and instead have required that regulations promulgated under section 301 be both necessary and appropriate. As discussed in more detail below, EPA's Proposal here is not necessary, and instead directly conflicts with several other provisions of the CAA. It is axiomatic that a "general grant of authority cannot trump specific statutory provisions."
[9066-1932] Commenter (6188) contends that the Proposed Rule contradicts the requirements of Federal environmental law, including cited authority for the rulemaking. The Proposed Rule's requirements are at odds with statutory mandates contained in the very same environmental laws it points to as providing a basis for the rule. They also directly controvert authority it cites as support for the rulemaking.
The requirements of the Proposed Rule conflict with EPA's statutory obligations under federal environmental laws. CAA § 109 mandates that EPA set air quality standards based on "air quality criteria," which must "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare." 42 U.S.C. §§ 7409, 7408(a)(2). 
Exclusion of relevant studies from consideration based not on the quality of the science but on the public availability of underlying data directly contradicts mandates. EPA itself has defined "best available science" without reference to the public availability of data:
Use of best available science involves the use of supporting studies conducted in accordance with sound and objective scientific practices, including, when available, peer reviewed science and supporting studies and data collected by accepted methods or best available methods (if the reliability of the method and the nature of the decision justifies use of the data).
Simply put, the "best available science" is science that follows standards accepted by the scientific community, not science that EPA arbitrarily selects based upon the public availability of data -- a criterion irrelevant to scientists' standards for determining whether research represents the best science.
Specifically, scientists have observed that the best quality science oftentimes relies upon data that is not publicly available due to significant privacy considerations. It is for that very reason that courts have recognized EPA's need to rely upon studies based on publicly undisclosed underlying data when considering the best science. American Trucking Ass'ns, 283 F.3d at 372 (explaining that curtailing EPA's ability to rely on published studies would exclude "plainly relevant scientific information" from regulatory decision-making processes); see also Coalition of Battery Recyclers Ass'n, 604 F.3d at 623 (finding that EPA is entitled to rely on published study results as "raw data is often unavailable due to proprietary interests of a study's scientific investigators or confidentiality agreements with study participants"). Forcing EPA to ignore high quality science controverts EPA's statutory obligations and will impair EPA's ability to protect public health and the environment.
Comment [9066-1894]: Commenter (6188) states that the EPA purports to derive authority for the Proposed Rule from CAA § 103. That provision simply authorizes EPA to establish a national research and development program for the prevention and control of air pollution and, as part of that program, to conduct and promote the coordination of research and studies relating to the causes, effects, extent, prevention, and control of air pollution. 42 U.S.C. § 7403(a)(1). The section authorizes specific activities of the Administrator in establishing such a program, none of which includes limiting the scope of reviewable data, research, or studies when undertaking regulatory action. The section has no bearing on how or to what extent EPA utilizes research in regulatory decision-making processes.
Comment [9217-3559]: Commenter (9225) contends that the proposed rule is not authorized by any authority delegated to EPA by Congress and is contrary to a number of statutes under EPA's authority. This includes, but is not limited to, the CAA; CWA; TSCA; LCSA; SDWA; FIFRA; and more. 
Comment [9287-3739]: Commenter (8281-PH16) Because of the importance of having access to such scientific data in order to protect the public's health, we oppose the "Strengthening Transparency and Regulatory Science" proposed rule. Research and evidence are the foundation of EPA's policies and has been necessary for success of laws like the CAA and improving and in saving lives from the dangers of air pollution.
Congress intentionally directed EPA to consider peer reviewed research under the CAA and mandates regular reviews of the science to ensure that EPA is reviewing and considering the most up to date science. The commenter believes that the proposal would prevent EPA from using the best science to inform decision-making, and the result would be weaker standards at the expense of American's health. For example, the proposal would exclude several landmark air quality studies from the evidence base that EPA is permitted to consider, largely on the basis that these studies include confidential patient information that would make them less transparent under the constructs of the proposed rule. 
Comment [9290-3812]: Commenter (8281-PH24) states that the EPA has administered successfully, the CWA, and the CAA, and these acts should be expanded based on scientific research. The EPA should not be working to undermine scientific research. 
Comment [5167-4751]: Commenter (5178) provides that the CAA requires that EPA set NAAQS to protect the public health, and to do so with an adequate margin of safety. NAAQS must be based on air quality criteria that "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air." EPA must comprehensively consider the state of scientific knowledge, and set standards to "protect individuals who are particularly sensitive to the effects of pollution." These susceptible groups include children; the elderly; and those with heart disease, lung disease, or asthma. Recently, Patricia Koman and colleagues have suggested that pregnant women should be considered an at-risk population due to physiological changes, noting, "Women with preexisting conditions, women experiencing poverty, and groups that suffer systematic discrimination may be particularly susceptible to cardiac effects of air pollutants during pregnancy."
Two landmark prospective cohort studies have informed the NAAQS for fine particulate pollution (PM2.5). The Harvard Six Cities Study followed 8,111 residents of six U.S. cities from recruitment in 1974-1977 through 1991 and collected air sampling data and health information. After adjusting for other health risk factors, the investigators found that mortality rates were associated with the level of air pollution, and that the association was particularly strong for fine particulate pollution. The American Cancer Society (ACS) study linked air pollution data from 151 U.S. metropolitan areas to data on risk factors and mortality in 552,138 adults residing in these areas who enrolled in 1982 and were followed through 1989. The authors found increased mortality to be associated with sulfate and fine particulate air pollution "at levels commonly found in U.S. cities."
Both studies were instrumental in setting the 1997 NAAQS, and some stakeholders called for the data to be made public. However, study participants had been assured confidentiality. Instead of making data public, investigators agreed to supply it to the HEI, which used a competitive process to select a team to reanalyze the data. In 2000, HEI reported that the re-analyses "assured the quality of the original data, replicated the original results, and tested those results against alternative risk models and analytic approaches without substantively altering the original findings of an association between indicators of PM air pollution and mortality."
Since publication of these two landmark studies, dozens of additional peer-reviewed studies, as well as follow-up research involving the Six Cities and ACS cohorts, have replicated and expanded upon their findings and provided insights into specific effects of particulate pollution on certain groups, In a look back at the evolution of research on the health effects of particulate pollution, lead Six Cities investigator Douglas W. Dockery commented:
True replication requires not reanalyzing existing work but independent investigators producing independent data. The HEI process made important contributions in evaluating, advancing, and applying new methods which led to significant additional analyses. But the HEI work reanalyzing an existing body of evidence was time-consuming and expensive. It would not be a good use of resources to require this approach for all studies contributing to the regulatory actions, or even to a few key studies.
EPA's 2011 policy assessment for review of the particulate NAAQS noted that in addition to epidemiologic studies, toxicological and controlled human exposure studies identify potential biological pathways for cardiovascular and respiratory effects associated with particulate exposures. The analysis also summarizes research on susceptible populations: "the currently available epidemiological and controlled human exposure evidence expands our understanding of previously identified susceptible populations (i.e., children, older adults, and individuals with pre-existing heart and lung disease) and supports the identification of additional susceptible populations (e.g., persons with lower socioeconomic status (SES), genetic differences)." It reports that evidence is accumulating on the effects of PM2.5 on low birthweight and infant mortality.
Some of the most recent research adds to knowledge on susceptible populations. For example, Qian Di and colleagues analyzed 2000-2012 data on 61 million Medicare beneficiaries and zip-code level information on PM2.5 and ozone pollution. They found increased mortality risks  --  including higher risks for black beneficiaries and those eligible for Medicaid, who are disproportionately women with low incomes  --  at pollutant levels below the current limits.
Mingyu Zhang and colleagues analyzed local air pollution data as well as data from the Boston Birth Cohort, which began recruiting pregnant women in 1998 and followed their children for several years after birth. They found that the children of women who were exposed to the highest PM2.5 levels during their third trimesters had a greater risk of elevated blood pressure at ages three to nine. Both of these studies involve data that can be obtained by investigators who apply to use it and are approved by the relevant IRB or Privacy Board,, but their data sets are not publicly available. Such an approach recognizes recent advances in identifying individuals from data that do not appear to contain PII. For decades, public release of data collected on individuals has relied on anonymization to protect privacy, but researchers have demonstrated the potential of "reidentification" techniques, which combine seemingly innocuous data from multiple publicly available sources, to identify individuals.,  
Failure to consider these studies and others like them in setting NAAQS because their data are not publicly available would conflict with responsibilities to advance children's environmental health and environmental justice, as well as the requirement that EPA set standards that protect susceptible subgroups. 
Comment [6141-2554]: Commenter (6166) asserts that the proposed rule is not authorized by any authority delegated to EPA by Congress and is contrary to a number of statutes under EPA's authority. This includes, but is not limited to, the CAA; CWA; TSCA; LCSA; SDWA; FIFRA; and more. Substantively, the rule violates numerous public health and environmental provisions contained in these laws, as well as requirements to use the best available science or to consider all available information,
Comment [6857-2945]: Commenter (6879) notes that EPA rules and regulations have been critical to the protection of wildlife, natural habitats, and the ecosystem services they provide to communities and the economy. This includes protection of people from air pollution and climate disruption under the CAA.
These laws require EPA to use the "best available" science, or all "reasonably available" science and information. By excluding high-quality peer-reviewed and validly published science from the regulatory decision-making process, the proposed regulation directly contravenes the specific mandate of these statutes. And by deliberately excluding the best science available, the rule also contravenes the Congressional mandates for these statutes and undermines their public health and environmental protection purposes.
Comment [6873-1712]: Commenter (6895) states that by mandating this irrelevant requirement, the proposed rule will restrict the use of best available science in public health and environmental protection decision making, which contradicts EPA's own "Peer Review Handbook." For example, EPA is required to use the "latest scientific knowledge" in setting NAAQS and historically EPA follows that requirement in making risk management decisions based on the best available science. The Presidential/Congressional Commission on Risk Assessment and Risk Management was mandated by Congress in the CAA Amendments of 1990 "to make a full investigation of the policy implications and appropriate use of risk assessment and risk management in regulatory programs under various Federal laws to prevent cancer and other chronic human health effects which may result from exposure to hazardous substances"., After years of work the Commission proposed a systematic, comprehensive framework that can address various contaminants, media, and sources of exposure, as well as public values, perceptions, and ethics, and that keeps the focus on the risk management goals. For this framework to be successful, the use of science cannot be restricted or limited. EPA should adhere to the well-reasoned process which the Commission on Risk Assessment and Risk Management developed.
Comment [7869-3372]: Commenter (8272) asserts that the proposed rule is not authorized by any authority delegated to EPA by Congress and is contrary to a number of statutes under EPA's authority. This includes, but is not limited to, the CAA; CWA; TSCA; LCSA; SDWA; and the FIFRA. The rule violates numerous public health and environmental provisions contained in these laws, as well as requirements to use the best available science or to consider all available information.
Comment [6106-875]: Commenter (6116) notes that, on April 30, 2018, the EPA published a notice in the Federal Register of a proposed rule that would drastically reduce the types of scientific studies that can be used to inform EPA regulations protecting public health under the guise of improving transparency. The commenter notes that they are troubled by this proposal, as we believe it would vastly undermine the mission of the EPA, which is to protect human health and the environment. It is curious that this proposal champions "transparency" when it seems certain to serve the opposite purpose. The commenter opposes this proposal for several reasons, including, but not limited to the following: 
 The EPA itself seems to question whether it has the authority to promulgate this rule. The FR notice asks commenters to provide information on what areas of the CAA they feel may demonstrate this type of authority. The EPA should have a solid foundation before it proposes rules and should not have to seek answers for questions about its own authority. 
Comment [6116-1117]: Commenter (6141) states that Ground the New Regulatory Framework based on the Agency's Statutory Authority and Longstanding Values and Principles of Good Government.
The Proposal for scientific transparency is consistent with the objectives and requirements of existing federal environmental statutes and administrative policies. Most importantly, the CAA and other major federal environmental statutes provide EPA with broad authority to establish procedures, requirements, and criteria for adopting rules, regulations, and policies for how the Agency adopts new regulations, rules, and guidance. For example, section 301(a) of the CAA provides the EPA Administrator with broad authority to establish regulations "as are necessary to carry out ... the functions" under the CAA, while section 307(d) provides extensive authority on the manner in which EPA may conducting notice and comment rulemakings, including provisions for ensuring a robust rulemaking record and adequate opportunities for public participation. The other relevant federal environmental statutes also provide EPA broad authority for the establishment of procedures, requirements, and criteria for how it establishes implementing rules, regulations, and policies. Oglethorpe Power [the commenter] supports EPA's proposal to use this authority to establish new rules for ensuring the transparency and public availability of the underlying scientific data and other important supporting documentation used by EPA in its regulatory actions.
Comment [6163-1276]: Commenter (6133) notes that, in addition to addressing how and whether the agency will consider science, the Proposal also contains a seemingly unrelated provision regarding agency peer review. The Proposal, in § 30.7, reads: 
      What role does independent peer review in this section?
      EPA shall conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein. 
      Because transparency in regulatory science includes addressing issues associated with assumptions used in models, EPA shall ask peer reviewers to articulate the strengths and weaknesses of EPA's justification for the assumptions applied and the implications of those assumptions for the results. 
      83 Fed. Reg. at 18,774. 
There is no statutory authority for EPA to "conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein." The federal statutes that EPA lists as putative authority for the Proposal provide no authority for proposed § 30.7. See 83 Fed. Reg. at 18,769/2 (citing CAA sections 103, 301(a), 42 U.S.C. 7403, 7601(a)). The claimed authorities that EPA lists do not mention peer review or even allude to the concept. Id. Neither the Proposal nor accompanying docket materials identify any provision of any federal statute that authorizes EPA to promulgate proposed § 30.7. Our own research revealed no provision of any federal statute that authorizes EPA to promulgate proposed § 30.7. 
When Congress writes federal statutes, Congress knows how to create legal authority for peer review, who shall conduct that peer review, what role, if any, that EPA or other parties will play, and how that peer review may be conducted. None of the provision in the law authorize EPA's Proposal in § 30.7. Instead, the statutes require that EPA use peer-reviewed science, regardless of whether it would meet EPA's definition of the term in § 30.7. And in cases where the law requires EPA to conduct the review, the statutes often spell out specifically how that should happen. 
The commenter cites statute provisions on Peer Review under the CAA (as well as other statutes): 
      § 7511b. Federal ozone measures
      (g) Ozone design value study The Administrator shall conduct a study of whether the methodology in use by the EPA as of November 15, 1990, for establishing a design value for ozone provides a reasonable indicator of the ozone air quality of ozone nonattainment areas. The Administrator shall obtain input from States, local subdivisions thereof, and others. The study shall be completed, and a report submitted to Congress not later than 3 years after November 15, 1990. The results of the study shall be subject to peer and public review before submitting it to Congress. 
      42 U.S.C. § 7511b (emphasis added).
      § 7412. HAP
      (p) Mickey Leland National Urban Air Toxics Research Center (NUATRC) (3) Scientific Advisory Panel (SAP) The Board of Directors shall be advised by a SAP, the 13 members of which shall be appointed by the Board, and to include eminent members of the scientific and medical communities. The Panel membership may include scientists with relevant experience from the NIEHS, the CDC, the EPA, the National Cancer Institute (NCI), and others, and the Panel shall conduct peer review and evaluate research results. The Panel shall assist the Board in developing the research agenda, reviewing proposals and applications, and advise on the awarding of research grants. 
      42 U.S.C. § 7412 (emphasis added). 
Under longstanding federal case law, when Congress authorizes an approach in one section of a statute using specific language but does not do the same in other sections of a statute, courts presume that Congress acted purposefully and did not mean to address or authorize that approach in those other statutory sections. See, e.g., Dean v. United States, 556 U.S. 568 (2009) ("It is generally presumed that Congress acts intentionally when including particular language in one section of a statute but not in another." (citing Russello v. United States, 464 U.S. 16, 23 (1983)). Not only are there no implied grants of authority to an agency in other statutory sections, but the congressional decisions to authorize the approach elsewhere in the statute give even greater force to the conclusion that the agency has not been given authority where Congress did not use the same or similar authorizing language. 
1.3.1.2 CAA (SNPRM Comments)
Comment [29-2180]: Commenter (11184) states that, by way of example, in CAA. Section 108(a)(2), Congress requires EPA to issue air quality criteria for air pollutants to "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air." 42 U.S.C. § 7408(a)(2). The SNPRM does not claim that studies that have not publicly released all their underlying data, such as those linking air pollution to mortality and disease, do not accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air. If EPA were to exclude such studies from consideration when issuing air quality criteria, it would be in violation of the CAA.
Comment [48-518]: Commenter 12727 states that the expanded proposal would irreconcilably conflict with EPA's obligation to regulate emissions of HAP under the CAA. Section 112 of the Act requires EPA to set emission standards for existing major sources of HAP that are typically no less stringent than "the average emission limitation achieved by the best performing 12 percent of the existing sources (for which the Administrator has emissions information)." This provision does not allow EPA to ignore information that the agency possesses but that qualifies as CBI. Indeed, EPA has historically taken such CBI into account in calculating the minimum limitation. Now, however, EPA has proposed to define "data and models" to encompass "data on environmental releases" and to eliminate any such data that would underlie pivotal regulatory science driving the level of a standard, such as emission standards under section 112. EPA's proposed rule, as expanded, patently conflicts with the CAA's well established requirements and cannot be finalized.
Comment [48-2181]: Commenter (12727) Several environmental statutes require EPA to ground its decision-making in scientific evidence. The CAA, for example, mandates that "[a]ir quality criteria . . . accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare," 42 U.S.C. § 7408(a)(2), and the TSCA requires the Administrator to "make decisions . . . based on the weight of the scientific evidence," 15 U.S.C. § 2625(i)." 
The originally proposed regulatory language (§§ 30.3, 30.5) violated these statutory commands by preventing EPA from relying on a study as "pivotal regulatory science . . . used to justify significant regulatory decisions" if dose-response data or models underlying the study were not publicly available, without regard to whether the study had been validated by other means. The Supplemental Notice exacerbates these violations by expanding the proposed regulation's scope (§§ 30.3, 30.5) to not just pivotal regulatory science used to justify significant regulatory decisions, but also "pivotal science supporting influential scientific information." 
Comment [38-40]: Commenter (12727) Just as EPA is required to consider all available science when making its regulatory decisions, governing environmental and public health statutes likewise require EPA to consider all available science when releasing ISI, which often forms the basis for the agency's regulatory decisions. For example:
      CAA section 108 instructs EPA to establish air quality criteria that "accurately reflect the latest scientific knowledge," which criteria, in turn, must inform NAAQS.
A rule that prohibits EPA from considering (or that downgrades) valid, high quality scientific studies when generating ISI such as the products identified above would contravene the above- noted statutory directives regarding EPA's obligation to consider all available science.
Comment [53-2006]: Commenter (11360) Congress intentionally and wisely embedded peer-reviewed research in the foundation of the CAA, including requiring regular reviews of the science, explicitly recognizing that EPA needs the most current, peer-reviewed data to protect the public health. Limiting scientific evidence does not strengthen regulations or policies; rather, it is paramount that the full suite of relevant science, vetted through peer-review, inform the landscape of decision-making. Excluding relevant studies simply because they do not meet unnecessarily rigid transparency standards will adversely affect decision-making processes.
Comment [73-421]: Commenter (11576) notes that, as previously explained, the current rulemaking process has also violated the requirements of other statutes the EPA may or may not be relying on in attempting to issue the proposed rule. Under the CAA, the EPA is directed to "give interested persons an opportunity for the oral presentation of data, views, or arguments, in addition to an opportunity to make written submissions[,]" and to "keep the record of such proceeding open for thirty days after completion of the proceeding to provide an opportunity for submission of rebuttal and supplementary information." The EPA's refusal to satisfy these requirements cannot be allowed to stand.
Comment [81-2189]: Commenter (12715) asserts that the EPA is arbitrarily barring consideration of scientific studies or information with nonpublic underlying data or data that are not able to be independently validated, or arbitrarily ascribing less weight to such studies or information, neither approach constitutes "housekeeping," and neither comports with statutory requirements that EPA use the best available science. To reiterate from the 2018 Comments, EPA's obligation with respect to the use of scientific information is clear: it must base its decisions on such criteria as the latest scientific knowledge, the best available, peer-reviewed science, and/or generally accepted scientific principles or laboratory tests. To cite an example, in performing its duties, EPA must rely on: "the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices," CAA of 1970, 42 U.S.C. § 7408(a)(2). EPA has repeatedly emphasized the importance of this fundamental precept, including in its 2018-2022 strategic plan, which states that one of the agency's priorities is to "identify, assess, conduct, and apply the best available science to address current and future environmental hazards."
These statutory requirements apply to all aspects of EPA's decision-making, including its scientific evaluations, adoption of regulatory standards, and development of policies. They do not permit what EPA proposes here: to either exclude or give less weight to scientific studies or information based on criteria -- availability of the underlying data and ability to independently validate underlying data -- that are not determinative of whether the studies or information constitute the best available science. Agencies may not adopt or implement regulations that conflict with the statutes under which they are promulgated, and an agency's interpretation of those statutes must always at least be reasonable. See Chevron, U.S.A., Inc. v. Nat. Res. Def. Council, Inc., 467 U.S. 837, 842-44 (1984). Furthermore, neither 5 U.S.C. § 301 nor any other purported housekeeping authority allows EPA to negate these statutes' fundamental, substantive, and common-sense commands. And even assuming EPA has some inherent housekeeping authority, it cannot rely on a general grant of rulemaking authority to promulgate regulations or develop policies that are otherwise inconsistent with more specific statutory directives. Glob. Van Lines, Inc. v. Interstate Commerce Comm'n, 714 F.2d 1290, 1293-97 (5th Cir. 1983).
Finally, EPA acknowledges that the statutes it administers would control in the event of a conflict between the proposed rule and those statutes, see 85 Fed. Reg. at 15,398, but that caveat cannot save the proposed rule. First, the core principle of the proposed rule is facially invalid, i.e., there is no circumstance in which it would be acceptable for EPA to exclude or give less weight to relevant, probative scientific studies, models, or other information that have been validated through peer review, on the sole basis that the underlying data are not publicly available or are not able to be independently validated. EPA cannot save an invalid rule by including what amounts to a waiver procedure, because the "essence of waiver is the assumed validity of the general rule." Alltel Corp. v. Fed. Commc'n Comm'n, 838 F.2d 551, 561 (D.C. Cir. 1988). Second, the agency's essential duty in rulemaking is to exercise its subject matter expertise to enact rules that implement and further the purposes of the statutes Congress has assigned it to administer. Chevron, 467 U.S. at 842-44. This does not include proposing rules without first determining the source of legal authority but instead asking commenters to supply that information. Third, the agency must propose its rules with specificity and be clear as to the programs, regulations, and information to which they will apply. 5 U.S.C. § 553(b)(3); Home Box Office, Inc., supra, 567 F.2d at 35-36.
Comment [82-2192]: Commenter (12464) Third, both alternatives are inconsistent with the statutes directing which data EPA must consider in certain contexts. As we explained in a previous comment letter, many statutes require EPA to consider science in its decision-making, without any reference to whether such science is based on studies with raw data that is publicly available. For example, under the CAA, EPA must set and review the NAAQS for six common pollutants. In setting these standards, the CAA instructs EPA to use "the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects [of air pollution] on public health or welfare." Thus, a conflict would arise between the statutory language and the reduced-weight alternative if EPA gave less weight to a study that was "the latest scientific knowledge" on a relevant question merely because the raw data from that study is not publicly available. The same conflict would exist with the proposed tiered-access system, if the "latest scientific knowledge" came from a study with raw data to which researchers have not or could not grant restricted access.
Comment [89-2193]: Commenter (11403) states that the EPA's 2018 version of this rule cited a hodgepodge of statutory authority, none of it valid. EPA at least corrected the obvious typographical errors from the previous version. However, those edits still do not grant EPA the authority to restrict science from all regulatory decision-making processes within the agency. EPA is attempting to use general grants of authority to create a completely novel approach to assessing science. However, as the Supreme Court has stated, "Congress . . . does not alter the fundamental details of a regulatory scheme in vague terms or ancillary provisions -- it does not, one might say, hide elephants in mouseholes." EPA does not have the authority to alter the fundamental details of all of its regulatory schemes based on such vague provisions of law.
Section 103 of the CAA directs EPA to "conduct, and promote the coordination and acceleration of, research, investigations, experiments, demonstrations, surveys and studies" on air pollution. Curtailing the use of science based on arbitrary notions of transparency does the opposite of what Section 103 requires.
Section 301(a) of the CAA gives the Administrator authority to "prescribe such regulations as are necessary to carry out his functions" relating to the CAA. This catch-all provision certainly provides the authority to prescribe regulations, but EPA has not provided a single rationale explaining how the proposed rule is "necessary" to the carrying out of functions under the CAA
In short, none of the statutory provisions cited by EPA provide authority for the proposed rule. The proposed rule fails at step one under Chevron v. Natural Resources Defense Council since it is unreasonable to believe that any statutory general grant of authority or any mention of EPA's oversight of other science or research programs would enable EPA to conduct rulemaking in a way that is altogether inconsistent with the rule of law. None of the cited provisions provide the authority sought by EPA, and as such the proposed rule should be withdrawn.
Comment [93-2088]: Commenter (11395) states that while similar language does not specifically appear in the CAA, American Trucking Assns. v. EPA [283 F.3d 355 (D.C. Cir. 2002)] supported EPA's reliance on peer-reviewed scientific studies, without conducting an independent analysis of the raw data underlying those studies, in setting the 1997 ozone and fine PM ambient air quality standards as follows:
      If EPA and other governmental agencies could not rely on published studies without conducting an independent analysis of the enormous volume of raw data underlying them, then much plainly relevant scientific information would become unavailable to EPA for use in setting standards to protect public health and the environment ... [S]uch data are often the property of scientific investigators and are often not readily available because of ... proprietary interests ... or because of [confidentiality] arrangements [with study participants].
Comment [97-2149]: Commenter (11479) states that the proposed rule, as modified in the supplemental notice, would clearly interfere with the agency's use of the best available science. The commenter notes that the CAA has requirements to update pollution standards that provide for an "adequate margin of safety" to protect public health. This determination can only be reliably made using the best available science. This proposal would, however, prevent EPA from using the best available science to set pollution standards that would provide for an adequate margin of safety, because the proposal excludes studies where the raw data and other underlying information cannot be made public. Because of this, EPA should not cite the CAA as a source of statutory authority. Further, the CAA encourages a close partnership with the NAS and other independent scientific institutions. When EPA is considering a proposal to reinterpret how it uses science in regulatory decisions, it should first and foremost consider comments from and engage with NAS and similar bodies, which, for this rule, EPA did not.
In fact, virtually all of EPA's statutory mandates require the use of the best available scientific evidence. Congress did not state in any of these laws that the agency shall only consider scientific evidence derived from studies where the underlying unanalyzed data is publicly available, nor that the evidence considered should be solely at the Administrator's discretion.
Comment [115-1431]: Commenter (11393) appreciates the SNPRM's recognition that "in the event the procedures outlined in the proposed rulemaking conflict with the statutes that EPA administers, or their implementing regulations, the statutes and regulations will control." 85 Fed. Reg. at 15398. This statement alone should lead EPA to withdraw the proposed rule and SNPRM; the requirements conflict with every statute EPA administers and would cause EPA to unlawfully exclude valid scientific research from its work. 
By way of example, in CAA Section 108(a)(2), Congress requires EPA to issue air quality criteria for air pollutants to "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air." 42 U.S.C. § 7408(a)(2). The SNPRM does not claim that studies that have not publicly released all their underlying data, such as those linking air pollution to mortality and disease, do not accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air. If EPA were to exclude such studies from consideration when issuing air quality criteria, it would be in violation of the CAA.
Comment [118-1437]: Commenter (11407) states that they appreciate the SNPRM's recognition that "in the event the procedures outlined in the proposed rulemaking conflict with the statutes that EPA administers, or their implementing regulations, the statutes and regulations will control." 85 Fed. Reg. at 15398. This statement alone should lead EPA to withdraw the proposed rule and SNPRM; the requirements conflict with every statute EPA administers and would cause EPA to unlawfully exclude valid scientific research from its work.
By way of example, in CAA Section 108(a)(2), Congress requires EPA to issue air quality criteria for air pollutants to "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of a4 identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air." 42 U.S.C. § 7408(a)(2). The SNPRM does not claim that studies that have not publicly released all their underlying data, such as those linking air pollution to mortality and disease, do not accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air.
If EPA were to exclude such studies from consideration when issuing air quality criteria, it would.be in violation of the CAA.
Comment [125-1091]: Commenter (11495) notes that the EPA attempts to support the Supplemental Notice, in part, with the same environmental statutes that require it to consider the best available science. For example, in the Proposed Rule, EPA cites section 103 of the CAA, which provides that the Administrator shall "conduct . . . research . . . and studies relating to the causes, effects (including health and welfare effects), extent, prevention, and control of air pollution" and is authorized to "collect and make available . . . the results of and other information . . . pertaining to such research and other activities." EPA also cites CAA section 301(a), which provides that the Administrator is authorized to "prescribe such regulations as are necessary" to carry out EPA's functions under the Act.] EPA points to similar provisions for all major environmental statutes, each of which contain the same requirement that regulations promulgated under the Administrator's general rulemaking authority must be necessary.
Courts have consistently "decline[d] to read open-ended power" into general statutory grants of rulemaking authority, such as CAA section 301(a). Rather, the authority imparted by such provisions is limited to actions that are necessary and appropriate for enforcing and carrying out the specific statutory mandates enacted by Congress. In the case of the CAA, this means that any rule promulgated under the authority of section 301(a) must be connected back to the actual purpose and goals of the Act -- to protect and enhance public health and welfare through air pollution prevention. Although EPA relies on such grants of general rulemaking authority, it fails to articulate any connection between the Strengthening Transparency in Regulatory Science Rule and the purposes or goals of each individual substantive statute.
EPA fails to -- and indeed cannot -- demonstrate that the Strengthening Transparency in Regulatory Science rule is both necessary and appropriate for carrying out a particular statutory directive. As discussed in detail above, the rule is not only unnecessary to carry out EPA's functions, but also in direct opposition to a number of specific statutory "quality of information" provisions. It is well established that agencies may not rely on general statutory grants of rulemaking authority to promulgate regulations that are inconsistent with more specific statutory 
Comment [125-2202]: Commenter (11495) states that the EPA's Supplemental Notice is at odds with provisions of multiple environmental statutes which require the Agency to consider certain high-quality scientific information. The commenter notes that the CWA and the CAA require that water and air quality criteria determinations be based on the "latest scientific knowledge."]
Each of these provisions directs EPA to incorporate a certain quality of scientific information into its rulemaking decisions made pursuant to the statute. In particular, these requirements are designed to ensure the scientific validity of the information upon which EPA bases its actions. Mandating a high-quality scientific basis for regulatory actions is necessary to guarantee the best possible protections for public health and welfare and to minimize partisan influence in scientific decision-making.
The Supplemental Notice, however, would impermissibly prohibit the agency from considering of relevant and high-quality scientific studies whenever the data underlying the study are not publicly available -- in direct contravention of the statutory directives that mandate, for example, consideration of the "best available science." None of the provisions requires, or even suggests, that EPA only consider information that is publicly available. It strains the plain meaning of words like "best," "latest," and even "appropriate" to suggest that they permit the unscientific requirement that all data be publicly available.
EPA "cannot ignore available . . . information." Courts have held that "best available data" requirements "prohibit[] [an agency] from disregarding available scientific evidence that is in some way better than the evidence [it] relies on." Furthermore, a plaintiff may establish a violation of a "best available science" requirement by "point[ing] to any scientific evidence that the agency failed to consider." The Supplemental Notice would undoubtedly preclude EPA from considering certain studies and scientific information that is "better" or the "best available," simply because the study's underlying data is not publicly available, thereby failing to satisfy Congressional directives and undermining the quality of regulatory decisions.
Likewise, EPA cannot avoid the statutory directives to consider high-quality scientific information simply by assigning a lesser weight to such information when the underlying data are not publicly available, as the Agency alternatively proposes. The Supplemental Notice's alternative proposal is designed to have largely the same pernicious effects as flatly prohibiting consideration of studies lacking publicly available data. Just as with the primary proposal, EPA's alternative proposal would limit the agency's consideration of the best available science in making important regulatory decisions, and is, therefore, arbitrary, and capricious.
The effect of the Proposed Rule is inconsistent with EPA's view that the Supplemental Notice is "intended to be consistent with the statutes that EPA administers and EPA plans to implement this procedural rulemaking in accordance with all applicable statutory and regulatory requirements." EPA attempts to resolve this inconsistency by stating that "in the event the procedures outlined in the proposed rulemaking conflict with the statutes that EPA administers, or their implementing regulations, the statutes and regulations will control." However, as discussed below, EPA lacks the statutory authority to issue a rule this broad, even when it claims that it will avoid statutory conflicts. 
Comment: Comment [151-1349]: Commenter (12405) notes that the CAA authorizes EPA to establish NAAQS to protect public health and to regulate emissions of HAP. EPA works with local governments to reduce air pollution and uses scientific studies that could be impacted by the proposed rule to revise its national air quality standards and NAAQS on a regular basis.
In addition to concerns stated in the previous section about the usefulness of data if enough information is redacted to protect privacy, these studies raise additional challenges because they were performed internationally. In the Danish study, participants are protected by EU law. Going forward, an EU study's compliance with the proposed rule will need to be reconciled with the new General Data Protection Regulation, which is seen as more restrictive than the United States' HIPAA of 1996. The privacy directive raises many issues for science not discussed here, but it is clear that many studies involving people located in the EU will have a difficult time both complying with the new directive and providing enough information to EPA to be considered.
Studies coming from other countries are vital to health determinations in the Unites States because people in other countries are exposed to chemicals at different rates than in the U.S. The ability to compare and contrast exposure to health outcomes can be enlightening. For example, average PM concentrations in South Korea and China are several times higher than in the United States, making relatively subtle effects stand out more easily. Studies done in other countries can also help researchers more clearly understand whether an effect is dependent on dose or length of exposure. The inability to review and use international research in determinations will guarantee EPA is missing major findings and important data.
Comment [157-1390]: Commenter (12422) states that many of laws authorizing the activities of the EPA require the agency to use the best science in protecting the public's health and environment. For example, the CAA mandates that air quality criteria "accurately reflect" the "latest scientific knowledge." In the past, the EPA has considered all available studies in issuing these criteria without consideration of the availability of the underlying data. Promulgation of this proposed rule would be an arbitrary and capricious violation of the provisions of these laws.
Comment [159-2089]: Commenter (12430 notes that, in the SNPRM, U.S. EPA proposes to consider only (or, in the alternative, more heavily weight) studies and models with publicly available data. However, restricting agency consideration of scientific studies and models would run afoul of U.S. EPA's obligations under many of its operating statutes to rely on the best available science.
For example:
The CAA requires, "Air quality criteria for an air pollutant shall accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air, in varying quantities."
The CAA and other statutes require U.S. EPA to make decisions based on many of the kinds of science that the SNPRM would prevent U.S. EPA from considering. The SNPRM, if finalized, would violate these statutes, and would prevent U.S. EPA from discharging its duties under the related laws.
Comment [161-2094]: Commenter (12434) states that the Proposal directly conflicts with requirements under the core federal environmental laws, each of which requires that decisions be made using the "best available science" or some similar restriction. By requiring that publicly available data be prioritized over other scientific data, regardless of any other merits, the EPA is arbitrarily constrained and will be unable to comply with the statutory requirements to meet the "best available science."
Similarly, the CAA requires that air quality criteria "shall accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare . . ." 42 U.S.C. 7408(a)(2). The Supreme Court ruled that NAAQS must be based on "published air quality criteria that reflect the latest scientific knowledge." American Trucking v. Whitman, 531 U.S. 457, 457 (2001). Like the CWA, the CAA details the information that should be considered when making regulatory decisions but does not limit this scientific information to only that which is publicly available. Again, the Proposal would eliminate key scientific information that informs the public health impacts of air pollution and allows the EPA to properly establish guidelines and restrictions.
The standards set under the CAA necessarily rely on public health data. Significant environmental justice impacts will result if confidential or only partially publicly available public health data is ignored. For example, in Connecticut v. Pruitt, in which Save the Sound (then known as Connecticut Fund for the Environment) was an intervenor, a district court judge required the EPA to hold a hearing to determine whether a Pennsylvania coal-fired power plant should install controls. Connecticut v. Pruitt, No. 3:17-cv-00796 (D. Ct. Feb. 7, 2018). Pollution from the facility contributed to Connecticut's greenhouse gas and ozone pollution, resulting in nonattainment of NAAQS, which are set based on public health studies. Consequently, the burning of coal will be limited to emergencies by 2023 and phased out by 2028. Similarly, it is these data sets that can inform our understanding of disparate impacts of the siting of polluting activities. NAAQS are used by EPA and state agencies when conducting cumulative impact modeling to determine the additional public health impacts of the siting of a new facility on a community that is already burdened with pollution. These standards cannot be appropriately set or the impacts of the siting of polluting activities properly examined if these studies are ignored, potentially aggravating the disparate impacts on communities. In order to conduct robust public health studies, it is common for members of the public to be assured that the data sets will remain partially or fully confidential so as to protect the sensitive health information of study participants, allowing for large data sets from which to draw conclusions.
Comment [162-1475]: Commenter (12435) states that the proposed framework under which the EPA Administrator would have discretion to exempt certain studies from these requirements lacks criteria tied to the actual scientific merit of the studies and/or the Acts that EPA administers. This is inconsistent with existing statutes, including the CAA 109(d), which specifies that an independent scientific review committee shall "advise the Administrator of areas in which additional knowledge is required to appraise the adequacy and basis of existing, new, or revised NAAQS" and "describe the research efforts necessary to provide the required information." This independent scientific review should not be limited in the research efforts reviewed by the Administrator.
Comment [177-1595]: Commenter (12596) states that the Administrators have used this authority to make substantive decisions regarding consideration of scientific studies when setting NAAQS. In the same way, this discretion would allow an Administrator to consider transparency assessments arising from a procedural framework during NAAQS reviews.
In these circumstances, a procedural framework would not be interpretative of the CAA's substantive authority. It would not modify the use or criteria for consideration of scientific studies under the statute, nor would it implicate public rights to participate under the CAA's rulemaking processes. Indeed, the public would retain the opportunity to provide input on transparency assessments, and EPA's substantive determinations concerning them, through a CAA rulemaking's notice-and-comment process. Transparency assessments produced under a procedural framework would simply be a data point for the Administrator to consider within the confines of the CAA and its jurisprudence.
Furthermore, the Final Transparency Rule would not affect requirements in the CAA to predicate regulatory action on the best science available. The opposite is true. In the face of growing concerns about the replicability of scientific studies, the absence of a transparency assessment framework undermines the reasonableness of future CAA rulemakings, and validity of underlying claimed health benefits.
One survey of 1,500 scientists found that 52% agreed that there is a significant "crisis" of reproducibility. These challenges could implicate the Administrator's ability to make a reasonable determination of adverse impacts under the CAA. Without a formal framework, assessments of scientific replicability, and therefore the transparency of pivotal regulatory science, could be arbitrary and capricious. The Final Transparency Rule would increase replicability and verification in the scientific process, thereby allowing the testing of critical methodological assumptions and mitigating biases in key studies that inform significant regulations. In other words, it would foster review of best science.
Comment [181-1623]: Commenter (12699) notes that a further scan through these statutes would find additional language that would suggest a need for accurate information. Looking at just the criteria pollutant language in the CAA, "[a]ir quality criteria for an air pollutant shall accurately reflect the latest scientific knowledge." [Emphasis added]. The NAAQS shall be "based on such criteria and allowing an adequate margin of safety, are requisite to protect the public health." It is impossible to know if something is "requisite" (or "necessary") if that decision is based on inaccurate science. The science in a given situation very well may be accurate, but it is incumbent upon the EPA (and other agencies) to take necessary steps to ensure this accuracy.
Beyond the specific statutory sections, the need for accuracy should be simply inferred. After all, did Congress delegate regulatory power to the EPA so that it can develop regulations that are based on inaccurate information?
Comment [181-2207]: Commenter (12699) states that, to put it succinctly, the answer is yes. The EPA should rely on both the substantive environmental statutes and the housekeeping authority.
The EPA would not be implementing these statutes properly if it were using flawed science in the promulgation of its rules. To highlight one example, it is useful to reexamine the U.S. Supreme Court case Michigan v. EPA. In discussing the "appropriate and necessary" language in 112(n)(1)(A) of the CAA, the U.S. Supreme Court argued "[o]ne would not say that it is even rational, never mind "appropriate," to impose billions of dollars in economic costs in return for a few dollars in health or environmental benefits."
Similarly, it would not be rational for the EPA to use science for these very costly policies without even knowing whether the science is credible. Nor would it be rational for the agency to fail to take every possible step to ensure scientific accuracy, including making underlying information available to the public (especially since the EPA is well aware of the problems that pervade the science). Making the information available to the public is not some nice gesture to appease the public. It is a critical step to help the agency itself get necessary feedback to better assess the science.
Comment [184-2093]: Commenter (12707) notes that the supplemental proposal undermines the use of best available science as called for in numerous environmental statutes including the CAA.
The CAA, for example, has requirements to update pollution standards that provide for an adequate margin of safety for public health. This determination can only be reliably made using the best available science. However, this proposal would prevent EPA from using that information to set science-based pollution standards that would provide for an adequate margin of safety. EPA should not cite the CAA as an appropriate source of statutory authority. The added confusion of the supplemental proposal, including so-called tiered access or EPA's ability to consider some research over what has long been considered "best available science" based upon the availability of data, does not legitimize the proposal.
In sum, there is no statutory authority, with or without the Housekeeping Statute, for EPA to rely on to censor or constrain science. Further, finalizing this effort will only serve to undermine EPA's essential role and create confusion over which studies will be considered and how they will be valued to set standards that directly impact public health and the quality of our air and water. Many of these uncertainties are recognized in EPA's own SAB's April 28, 2020 letter.
Comment [187-1702]: Commenter (12720) provides that the EPA cites 42 U.S.C. § 7601(a) of the CAA as one basis for the Proposal. But that section merely authorizes the Administrator to "prescribe such regulations as are necessary to carry out his functions under this chapter." The courts have made clear that "EPA cannot rely on its gap-filling authority to supplement the CAA's provisions when Congress has not left the agency a gap to fill." NRDC v. EPA., 749 F.3d 1055, 1064 (D.C. Cir. 2014); see also American Petroleum Institute v. EPA, 52 F.3d 1113, 1119 (D.C. Cir. 1995) ("the general grant of rulemaking power to EPA cannot trump specific portions of the CAA"); NRDC v. Reilly, 976 F.2d 36, 41 (D.C. Cir. 1992) (EPA cannot use its general rulemaking authority as justification for adding to a statutorily specified list); Sierra Club v. EPA, 719 F.2d 436, 453 (D.C. Cir. 1983) (same); Gonzales v. Oregon, 546 U.S. 243, 264 - 65 (2006) ("It would go . . . against the plain language of the text to treat a delegation for the `execution' of [the Attorney General's] functions as a further delegation to define other functions well beyond the statute's specific grants of authority."). Here, not only is there no statutory gap to fill, as explained further below, the Proposal and Supplemental Proposal are in direct conflict with other provisions of the Act. EPA cannot rely on 42 U.S.C. § 7601(a) to support any final rule.
EPA also cites 42 U.S.C. § 7403, which requires the Administrator to establish a national research and development program for air pollution, among other things. EPA does not state specifically which of the many subsections it believes authorizes this proposed rule. Thus, the citation fails to provide sufficient notice for the public to comment on the rulemaking.
Nothing in the Proposal (or Supplemental Proposal) establishes or even purports to establish the type of national research and development program for air pollution discussed in subsection (a). But that subsection is nonetheless revealing about congressional intent concerning "studies relating to the causes, effects (including health and welfare effects) extent, prevention, and control of air pollution." 42 U.S.C. § 7403(a)(1). There is no indication that Congress intended to allow EPA to ignore or refuse to consider studies on the health and welfare effects of air pollution only if raw data or `regulatory science underlying EPA's actions [were] publicly available in a manner sufficient for independent validation." See 83 Fed. Reg. at 18,773 (proposed §§ 30.1 - 30.3). Indeed, the absence of any such congressional conditions or criteria makes it all the more obvious that EPA invented and added those criteria and conditions as a matter of its own policy preferences, contrary to the Act. This EPA may not do.
Subsection (b) authorizes EPA to collect and make available information about such research, but nothing in that subsection allows EPA to restrict which types of data it considers in regulatory decisions. Nor does subsection (b) draw any distinction between dose-response data, "data and models underlying pivotal regulatory science and pivotal science," and other types of data. Again, the absence of any such congressional distinction makes it all the more obvious that EPA invented and added that distinction as a matter of its own policy preferences, contrary to the Act. This EPA may not do. None of the other subsections in 42 U.S.C. § 7403 address this issue either. There is no support in the CAA for the Proposal or the Supplemental Proposal.
Comment [198-2211]: Commenter (13549) states that the SNPRM changes the authority EPA claims to rely on for the proposal, however either way, the proposal remains unlawful. The text of the SNPRM proposes to rely solely on 5 U.S.C. § 301. 85 Fed. Reg. at 15404. The preamble says EPA is taking comment on whether to use 5 U.S.C. § 301, which the SNPRM calls EPA's housekeeping authority, independently or in conjunction with statutory provisions cited in the proposed rule and corrected in the SNPRM's preamble. 85 Fed. Reg. at 15398. The fact that EPA is still considering and asking what authority it can rely on for the proposed rule, twenty-two months after the initial proposed rule, shows that EPA does not have such authority. The SNPRM's statements on authority continues EPA's pattern of making rules first and searching for a justification later (which it still has not provided for this rulemaking). As the NTAA explained in its 2018 comments: In order to protect human health and environment, EPA must have a solid foundation before it proposes rules and rulemaking as such portrays the agency's naiveté in seeking answers for questions about its own authority.
First, none of the initial or corrected environmental statutes provide EPA authority for the proposed rulemaking. In most cases, EPA simply cites its general authority for rulemaking under the statutes. Still that general authority alone cannot provide a basis for the rule, especially when, as here, the rule would conflict with the requirements of each of the statutes. Courts have made clear that "EPA cannot rely on its gap-filling authority to supplement [a statute's] provisions when Congress has not left the agency a gap to fill." NRDC v. EPA, 749 F.3d 1055, 1064 (D.C. Cir. 2014); see also New York v. U.S. EPA, 413 F.3d 3, 40-42 (D.C. Cir. 2005). EPA otherwise cites to statutes describing research programs established by the agency, with no relation to or authorization for the requirements the SNPRM seeks to create. The Band also disagrees with EPA's reliance on 5 U.S.C. § 301 because 1) it is questionable whether § 301 applies to EPA as EPA is not one of the fifteen "Executive Departments" listed at 5 U.S.C. § 101, and 2) the proposed rule and SNPRM are not merely internal rules of agency procedure -- they would significantly impact every aspect of EPA's scientific and regulatory work, the individuals who contribute to that work, and the Tribes and communities who rely on it.
The Band appreciates the SNPRM's recognition that "in the event the procedures outlined in the proposed rulemaking conflict with the statutes that EPA administers, or their implementing regulations, the statutes and regulations will control." 85 Fed. Reg. at 15398. This statement alone should lead EPA to withdraw the NPRM and SNPRM; the requirements conflict with every statute EPA administers and would cause EPA to unlawfully exclude valid scientific research from its work.
By way of example, in CAA Section 108(a)(2), Congress requires EPA to issue air quality criteria for air pollutants to "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air." 42 U.S.C. § 7408(a)(2). The SNPRM does not claim that studies that have not publicly released all their underlying data, such as those linking air pollution to mortality and disease, do not accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air. If EPA were to exclude such studies from consideration when issuing air quality criteria, it would be in violation of the CAA.
Comment [199-2213]: Commenter (12403) notes that the proposal would hamper implementation of CAA requirements.
Comment [200-2214]: Commenter (12412) notes that the SNPRM changes the authority EPA claims to rely on for the proposal, however either way, the proposal remains unlawful. The text of the SNPRM proposes to rely solely on 5 U.S.C. § 301. 85 Fed. Reg. at 15404. The preamble says EPA is taking comment on whether to use 5 U.S.C. § 301, which the SNPRM calls EPA's housekeeping authority, independently or in conjunction with statutory provisions cited in the proposed rule and corrected in the SNPRM's preamble. 85 Fed. Reg. at 15398. The fact that EPA is still considering and asking what authority it can rely on for the proposed rule, twenty-two months after the initial proposed rule, shows that EPA does not have such authority. The SNPRM's statements on authority continues EPA's pattern of making rules first and searching for a justification later (which it still has not provided for this rulemaking). As the NTAA explained in its 2018 comments: In order to protect human health and environment, EPA must have a solid foundation before it proposes rules and rulemaking as such portrays the agency's naiveté in seeking answers for questions about its own authority.
First, none of the initial or corrected environmental statutes provide EPA authority for the proposed rulemaking. In most cases, EPA simply cites its general authority for rulemaking under the statutes. Still that general authority alone cannot provide a basis for the rule, especially when, as here, the rule would conflict with the requirements of each of the statutes. Courts have made clear that "EPA cannot rely on its gap-filling authority to supplement [a statute's] provisions when Congress has not left the agency a gap to fill." NRDC v. EPA, 749 F.3d 1055, 1064 (D.C. Cir. 2014); see also New York v. U.S. EPA, 413 F.3d 3, 40-42 (D.C. Cir. 2005). EPA otherwise cites to statutes describing research programs established by the agency, with no relation to or authorization for the requirements the SNPRM seeks to create. The commenter disagrees with EPA's reliance on 5 U.S.C. § 301 because 1) it is questionable whether § 301 applies to EPA as EPA is not one of the fifteen "Executive Departments" listed at 5 U.S.C. § 101, and 2) the proposed rule and SNPRM are not merely internal rules of agency procedure -- they would significantly impact every aspect of EPA's scientific and regulatory work, the individuals who contribute to that work, and the Tribes and communities who rely on it.
The commenter appreciates the SNPRM's recognition that "in the event the procedures outlined in the proposed rulemaking conflict with the statutes that EPA administers, or their implementing regulations, the statutes and regulations will control." 85 Fed. Reg. at 15398. This statement alone should lead EPA to withdraw the proposed rule and SNPRM; the requirements conflict with every statute EPA administers and would cause EPA to unlawfully exclude valid scientific research from its work.
By way of example, in CAA Section 108(a)(2), Congress requires EPA to issue air quality criteria for air pollutants to "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air." 42 U.S.C. § 7408(a)(2). The SNPRM does not claim that studies that have not publicly released all their underlying data, such as those linking air pollution to mortality and disease, do not accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutants in the ambient air. If EPA were to exclude such studies from consideration when issuing air quality criteria, it would be in violation of the CAA.
Comment [201-2218]: Commenter (12474) notes that the SNPRM changes the authority EPA claims to rely on for the proposal, however either way, the proposal remains unlawful. The text of the SNPRM proposes to rely solely on 5 U.S.C. § 301. 85 Fed. Reg. at 15404. The preamble says EPA is taking comment on whether to use 5 U.S.C. § 301, which the SNPRM calls EPA's housekeeping authority, independently or in conjunction with statutory provisions cited in the proposed rule and corrected in the SNPRM's preamble. 85 Fed. Reg. at 15398. The fact that EPA is still considering and asking what authority it can rely on for the proposed rule, twenty-two months after the initial proposed rule, shows that EPA does not have such authority. The SNPRM's statements on authority continues EPA's pattern of making rules first and searching for a justification later (which it still has not provided for this rulemaking). As the NTAA explained in its 2018 comments: In order to protect human health and environment, EPA must have a solid foundation before it proposes rules and rulemaking as such portrays the agency's naiveté in seeking answers for questions about its own authority.
First, none of the initial or corrected environmental statutes provide EPA authority for the proposed rulemaking. In most cases, EPA simply cites its general authority for rulemaking under the statutes. Still that general authority alone cannot provide a basis for the rule, especially when, as here, the rule would conflict with the requirements of each of the statutes. Courts have made clear that "EPA cannot rely on its gap-filling authority to supplement [a statute's] provisions when Congress has not left the agency a gap to fill." NRDC v. EPA, 749 F.3d 1055, 1064 (D.C. Cir. 2014); see also New York v. U.S. EPA, 413 F.3d 3, 40-42 (D.C. Cir. 2005). EPA otherwise cites to statutes describing research programs established by the agency, with no relation to or authorization for the requirements the SNPRM seeks to create.
The NTAA also disagrees with EPA's reliance on 5 U.S.C. § 301 as:
      1) It is questionable whether § 301 applies to EPA as EPA is not one of the fifteen "Executive Departments" listed at 5 U.S.C. § 101, and
      2) The proposed rule and SNPRM are not merely internal rules of agency procedure --  they would significantly impact every aspect of EPA's scientific and regulatory work, the individuals who contribute to that work, and the Tribes and communities who rely on it.
The NTAA appreciates the SNPRM's recognition that "in the event the procedures outlined in the proposed rulemaking conflict with the statutes that EPA administers, or their implementing regulations, the statutes and regulations will control." 85 Fed. Reg. at 15398. This statement alone should lead EPA to withdraw the proposed rule and SNPRM; the requirements conflict with every statute EPA administers and would cause EPA to unlawfully exclude valid scientific research from its work.
By way of example, in CAA Section 108(a)(2), Congress requires EPA to issue air quality criteria for air pollutants to "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air." 42 U.S.C. § 7408(a)(2). The SNPRM does not claim that studies that have not publicly released all their underlying data, such as those linking air pollution to mortality and disease, do not accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air. If EPA were to exclude such studies from consideration when issuing air quality criteria, it would be in violation of the CAA.
Comment [205-2036]: Commenter (13788) states that this rule is not procedural. If finalized, it will have substantial effects on EPA decisions that impact regulated entities. For example, under the CAA's NAAQS program, EPA is required to issue air quality criteria for pollutants that "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air." If EPA were to ignore or diminish the value of the latest scientific studies because they involve human health data that cannot be made publicly available, that would significantly impact the health-protectiveness of the resulting ambient air quality standards, which would have a substantial effect on both regulatory beneficiaries and regulated industries. 
In addition to the Chrysler Court's decision, the D.C. Circuit held in Pickus v. United States Bd. Of Parole, that rules of practice or procedure "should not be deemed to include any action which goes beyond formality and substantially affects the rights of those over whom the agency exercises authority," and that "adherence to congressional purpose counsels a construction of this exemption [for procedural rules] that excludes from its operation action which is likely to have considerable impact on ultimate agency decisions." By restricting EPA's consideration of science in regulatory decision-making based only on the public availability of underlying data, this rule would have a clear and substantial effect on ultimate agency decisions, including decisions that affect the rights of regulated entities, and therefore it is not a procedural rule.
Nor is this a proposed "internal rule of agency procedure" that "exclusively pertains to the internal practices of the EPA." According to details provided to staff of the House Committee on Science, Space, and Technology, EPA's proposed tiered access system would not be implemented internally at EPA, but rather the Agency expects that the bulk of the burden of implementing such a system would fall on the researchers who conduct (or conducted, as this relates to retroactive application of this new policy) the study. According to information shared with committee staff, EPA anticipates that researchers would be responsible for managing the logistics of making data and models publicly available in compliance with the rule, for judging the sensitivity of study data and models and what information can or cannot be made publicly available through tiered access, and what tier of access should be designated for different types of information. It is unclear how EPA would ensure compliance with this rule, or whether after this rule is finalized, it will in effect ban all studies based on confidential human health data. However, what is clear is that this rule would have significant impacts beyond the Agency's internal practices and procedures.
Comment [205-2051]: Commenter (13788) states that under the APA and CAA, Agency actions that are "arbitrary, capricious, an abuse of discretion, or otherwise not in accordance with law" are to be held unlawful and set aside. Agencies are required to "articulate a satisfactory explanation for [the] action including a `rational connection between the facts found and the choice made.'" Here, however, EPA is proposing to exclude or devalue scientific studies based on criteria that are unrelated to scientific value or merit, and that would eliminate the Agency's ability to rely on research that may be the best available science.
The Agency's newly claimed statutory authority under the housekeeping statute does not save this rule, and actually undermines any transparency justification for the rule. Many of the studies EPA relies on are done outside of the Agency, and therefore must be beyond the reach of the housekeeping statute. The Agency's conflation of transparency in EPA's processes with data availability in scientific research outside the Agency cannot be used to provide satisfactory justification for this rule.
EPA seems to believe that by wielding the terms "validation" and "reanalysis" it shows concern about the quality of studies relied on by the Agency. But EPA's real intention in the SNPRM, as in the Proposal, is to limit the Agency's access to public health science, not to ensure the quality of the studies the Agency relies on. That is made abundantly clear by the Agency's statement that if "multiple high-quality studies exist but only two studies have data and models that are available for independent validation and reanalysis, EPA would only include those two studies as pivotal regulatory science and/or pivotal science." By acknowledging that the highest quality studies will be ignored if confidential data and models are not made available for validation and reanalysis, EPA is admitting that this rule is not concerned with improving the quality of studies relied on by the Agency. The SNPRM is instead laser-focused on information availability, even if it comes at the expense of the quality of scientific work. The best studies may not rely on available underlying data -- precisely because they are human health studies that requires the maintenance of the privacy of the study participants  -- and in those situations this rule would clearly prioritize data availability over quality.
Unlike the 2018 Proposal, the SNPRM does not include any acknowledgement that EPA must use the "best available science" for regulatory actions -- but it must. This lack of concern for quality is evident throughout the SNPRM and is particularly alarming in an agency that relies heavily on science -- as it must, given its statutory mandates. Additionally, the suggestion that the rule could apply to studies retroactively if the Administrator does not grant an exemption further illustrates the intention to undermine longstanding reliance on the human health studies that have done the most to advance air quality and public health. The SNPRM states that the "proposal would apply to reviews of data, models, and studies at the time a rule is developed or ISI is finalized, regardless of when the data and models were generated." EPA has provided no reasoning to explain how retroactive application to past studies -- including the Harvard Six Cities and American Cancer Society PM health studies and subsequent work that links exposures to air pollution with declines and damages to human health -- is justified. It certainly has not provided any information to suggest that this rule would encourage the release of more data from those studies. This proposal's focus is clearly on arbitrarily excluding or diminishing the value of quality scientific research rather than any real commitment to transparency.
Comment [205-2126]: Commenter (13788) claims that the rule conflicts with EPA's performance of its statutory duties, including under the CAA, by preventing the Agency from relying on the latest and best available science for its rulemakings.
Comment [205-2186]: Commenter (13788) states that not one of the other environmental statutes cited as potential authority by EPA actually authorizes this rule. At most, many of the provisions cited by EPA generally allow the agency to promulgate regulations in order to implement various statutes, with many requiring that the regulations be necessary. However, EPA has not shown how this rule would further the implementation of any statute, much less how the rule is necessary to fulfill any statutory responsibilities, and the fact that the Agency has not considered such a rule necessary until now suggests the opposite. Under the CAA, the most important scientific studies and models for use in evaluating the public health effects of air pollution are those that are based on actual human health data -- much of which cannot be publicly available due to the confidentiality requirements imposed by other federal statutes and the agreements under which the data is provided to researchers. For CAA rulemaking then, any of the approaches EPA is proposing actively undermine the Agency's use of the best science in making decisions that promote and protect the public health and welfare -- the fundamental purpose of the CAA.
If Congress had intended EPA to prioritize data transparency over the quality of scientific information in its regulatory decision-making, it could have written that requirement into the statutes. In the absence of such a requirement and as required by law, EPA should prioritize quality in accordance with the various "mandates requiring that its decisions rest on various formulations of the best available science."
Comment [205-2216]: Commenter (13788) asserts that, Nor the Federal Housekeeping Statute cannot be used to authorize rules that limit the scope of EPA's responsibilities under other statutes. The Chrysler Court was clear that "[section] 301 does not authorize regulations limiting the scope of [the Trade Secrets Act]" (the statute under consideration in that case), or of any other substantive statutory authority. Similarly, the Federal Housekeeping Statute cannot authorize regulations limiting the scope of environmental statutes. If finalized, this rule would limit EPA's ability to evaluate and use the best available science to support its regulatory decisions, thereby undermining its ability to fully implement its responsibilities as mandated under environmental statutes. It therefore cannot be a valid exercise of the authority granted by the Federal Housekeeping Statute.
Neither the Federal Housekeeping Statute nor the IQA provide authority or justification for a rule that would significantly impede and disrupt EPA's ability to fulfill its statutory mandates. For example, the air quality criteria EPA is required to issue when reviewing the NAAQS under the CAA takes the form of an ISA, in which the Agency evaluates the latest science on public health and environmental effects of criteria pollutants. The ISA document is considered a HISA, which is a subset of the ISI governed by the IQA guidance. Subjecting the ISA to the requirements of this rule and SNPRM would result in the exclusion of important, longstanding public health studies from consideration, because the human health data on which they are based, and which makes them of value, is not publicly available as contemplated by the proposed rule. Such a result would directly conflict with the CAA's mandate that the air quality criteria reflect the latest scientific knowledge. The Agency claims that "[i]n the event the procedures outlined in this part conflict with statutes that EPA administers, or their implementing regulations, the statutes and regulations will control." EPA fails, however, to explain how the Agency will define and identify such conflicts and ensure that staff do not follow the proposed rule. This is a significant question that EPA entirely fails to discuss. 
1.3.2 Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA) (NPRM and SNPRM Comments).
Comment: Sections 1.3.2.1 and 1.3.2.2 of this Chapter present comments received on the NPRM (Section 1.3.2.1) and SNPRM (Section 1.3.2.2) related to the EPA's legal authority for the rule under the FIFRA. The EPA provides the following general response to these comments:
Response: EPA appreciates your comments. As discussed in Section I.C of the final rule preamble and in this RTC document in Section 1.2, EPA has decided it is most appropriate to rely on its authority to promulgate housekeeping regulations, rather than any of the substantive environmental statutes cited in the 2018 proposal or the 2020 supplemental proposal as legal authority for the rule. EPA intends to promulgate future science transparency rules under its substantive environmental statutes. Additionally, the final rule states in § 30.3 that "[i]n the event the procedures outlined in this rule conflict with statutes that EPA administers, or their implementing regulations, the statutes and regulations will control."
1.3.2.1 FIFRA (NPRM Comments)
Comment [6103-517]: Commenter (6113) asserts that the proposed regulation turns the statutory scheme of FIFRA upside down. Very limited amounts of pesticides on food products are allowed only if EPA determines the amount to be safe. Slip op. p. 7. See also slip op. at p. 9 (explaining how the EPA must meet FDA standards on this). The proposed regulation seeks to exclude much information that establishes the pesticide's harm whereas the statute requires the manufacturer to prove that the amount is safe.
Comment [6112-1556]: Commenter (6137) provides that EPA also cites to two specific provisions in the Federal Insecticide, Fungicide, Rodenticide Act ("FIFRA") as statutory support for the Proposed Rule. Upon closer examination, these provisions do not provide the necessary authority for this Rule.
First, EPA points to Section 20(a), 7 U.S.C. § 136r(a), which provides:
      (a) Research
      The Administrator shall undertake research including research by grant or contract with other Federal agencies, universities, or others as may be necessary to carry out the purposes of this subchapter, and the Administrator shall conduct research into integrated pest management in coordination with the Secretary of Agriculture. The Administrator shall also take care to ensure that such research does not duplicate research being undertaken by any other Federal agency.
By its plain language, this provision authorizes research, not the use of scientific studies in regulating pesticides. Thus, EPA's reliance on this provision as support for this rule is misplaced. 
Second, the Proposed Rule cites Section 25(a)(1), 7 U.S.C. § 136w, which authorizes the EPA Administrator "to prescribe regulations to carry out the provisions of this subchapter." However, this broad authority is expressly limited to regulating "in accordance with the procedure" prescribed in FIFRA itself. Id. Yet, as discussed more fully in Section V, EPA failed to comply with these requisite procedures. Accordingly, this provision provides no authority for EPA's issuance of this rule.
Comment [6112-1567]: Commenter (6137) contends that not only does FIFRA not provide authority for the Proposed Rule, but it likewise contains provisions directly at odds with the purpose and effect of the Rule. 
Under FIFRA, EPA must register a pesticide (with rare exceptions) before it may be sold or used in the United States. 7 U.S.C. § 136a(a). To register or re-register a pesticide, EPA must determine that its use "will not generally cause unreasonable adverse effects on the environment." Id. § 136a(c)(5)(D); see id. § 136(bb) (definition of "unreasonable adverse effects"). FIFRA defines "unreasonable adverse effects" as "any unreasonable risk to man or the environment, taking into account the economic, social, and environmental costs and benefits of the use of any pesticide." Id. § 136(bb).
EPA requires a company seeking establishment or retention of a pesticide registration to submit data and information to enable EPA to make its unreasonable adverse effects determination. In addition to a standard set of data, EPA can issue data call-in notices requiring additional testing and information. Often, EPA requires the registrant to conduct particular laboratory tests to assess the pesticide's toxicity. As Nancy Beck noted during the drafting of the rule, pesticide regulations require manufacturers to submit to EPA "a huge amount of data," and that the studies come to EPA as CBI. See Maria Hegstad, "Absent ORD Chief, Trump's Toxics Pick Expands Reach Across EPA Science," Inside EPA (May 10, 2018), https://insideepa.com/weekly-focus/absent-ord-chief-trumps-toxics-pick-expands-reach-across-epa-science. The raw data underlying the industry laboratory studies is rarely made available to the public, and the registrants would almost certainly oppose such disclosure on CBI grounds. Nor are such studies typically peer reviewed.
In addition, after a pesticide has been registered, the registrant must provide EPA all factual information regarding the pesticide's unreasonable adverse effects. 7 U.S.C. § 136d(a)(2). Such information comes in a variety of forms  -  from academic studies, poisoning incident reports, or studies conducted for other regulatory authorities at the state, federal, or international level. Often, the raw data are unavailable.
The Proposed Rule thus conflicts with FIFRA's pesticide registration requirements as it eliminates from consideration important studies used to show the unreasonable adverse effects of the pesticide toxins. As Beck herself acknowledged of an early version of the rule, the directive would "`jeopardize our entire pesticide registration/re-registration process.'" Maria Hegstad, "Absent ORD Chief, Trump's Toxics Pick Expands Reach Across EPA Science," Inside EPA (May 10, 2018). Accordingly, the Proposed Rule cannot stand.
Comment [6112-1623]: Commenter (6137) states that EPA failed to follow the specific procedures required by the FIFRA when issuing the Proposed Rule. Specifically, section 25w of FIFRA requires, among other things, that EPA provide a copy of any proposed rule or regulation to the Secretary of Agriculture for review and comment 60 days before a proposed rule is published in the Federal Register. 7 U.S.C. § 136w(a)(2)(A). EPA must then provide the Secretary a copy of the rule EPA intends to publish as a final rule no later than 30 days before publication. Id. § 136w(a)(2)(B). EPA must also consult with its own SAP in an effort to receive comments from it regarding the impact the proposed rule will have on health and the environment. Id. § 136w(d). Such consultation must occur under the same timelines sets forth for consulting with the Secretary of Agriculture. Id. § 136w(d)(1). Lastly, before a final rule can take effect, EPA is required to submit the rule to Congress and "the rule or regulation shall not become effective until the passage of 60 calendar days after the rule or regulation is so transmitted." Id. § 136w(a)(4).
Thus, FIFRA sets forth a clear process in which EPA must consult with a number of entities, including both the USDA and a SAP when issuing rules impacting pesticides. See, e.g., Nat'l Coal. Against the Misuse of Pesticides v. EPA, 867 F.2d 636 (D.C. Cir. 1989) (noting that "FIFRA generally requires the Administrator [of EPA] to consult with the Secretary of Agriculture . . . and a seven member Scientific Advisory Panel (on environmental health questions) prior to making public any notice of intent to cancel," in context of pesticide registration cancellation). Yet it is indisputable that EPA utterly failed to take these necessary steps. For this reason, too, EPA's Proposal cannot stand.
Comment [6118-1731]: Commenter (6143) asserts that EPA has issued its proposed rule on Strengthening Transparency in Regulatory Science pursuant to its authority under several environmental statutes that that it implements, including the CAA; CWA; SDWA; RCLA; CERCLA; EPCRA; FIFRA; and TSCA. This proposed action is fully consistent with the authorities and objectives laid out in those statutes. EPA must ensure that any final action it takes on this proposal continues to respect the balance reflected in those statutes between sound, transparent science, and legitimate privacy interests.
Comment [6152-651]: Commenter (6122) contends that EPA cites multiple statutory provisions in an attempt to support the proposed rule that it believes authorize this type of rulemaking, "including provisions providing general authority to promulgate regulations necessary to carry out the Agency's functions under these statutes and provisions specifically addressing the Agency's conducting of and reliance on scientific activity to inform those functions . . ." None of the specific statutory provisions provided by EPA provide specific authority for EPA to fundamentally rework the foundational processes it uses to assess and include/reject science. Indeed, if such authority existed, one would have expected EPA to have discovered this authority many decades earlier. Instead, EPA is attempting to use general grants of authority to create a completely novel approach to assessing science. However, as the Supreme Court has stated, "Congress...does not alter the fundamental details of a regulatory scheme in vague terms or ancillary provisions -- it does not, one might say, hide elephants in mouseholes."And if this is true, neither can the EPA alter the fundamental details of all of its regulatory schemes based on such vague provisions of law. We review each of the provisions listed by EPA, none of which discuss transparency, dose response, pivotal regulatory science, benefit-cost calculation, or changing "agency culture," the stated goals of this rulemaking.
Section 25(a)(1) of the FIFRA authorizes the Administrator "to prescribe regulations to carry out the provisions of this subchapter." This general authorization section of FIFRA differs from that of the other statutes listed because it actually does specify some considerations the Administrator must give when promulgating regulations. For example, it states, "Such regulations shall take into account the difference in concept and usage between various classes of pesticides, including public health pesticides, and differences in environmental risk and the appropriate data for evaluating such risk between agricultural, nonagricultural, and public health pesticides." This provision clearly intends to provide detail and guidance on rulemaking. The fact that the section does not mention transparency indicates that it does not grant EPA authority to undertake the proposed rule.
Section 136r(a) of the FIFRA directs the Administrator to conduct research to carry out the purposes of the statute. The section states, "The Administrator shall also take care to ensure that such research does not duplicate research being undertaken by any other Federal agency." The section does not state that the EPA can ignore sound science if it does not meet its nebulous definition of transparency.
Comment [6155-664]: Commenter (6125) states that the proposal lists FIFRA sec. 25 as an authority for the rulemaking. 83 FR 18769. The agency, however, has failed to follow required procedures for issuing a valid regulation under FIFRA. FIFRA requires the agency to seek comments from the Secretary of Agriculture on all draft proposed regulations prior to signing the regulation for publication in the Federal Register; and, if the USDA provides comments, the agency must respond in writing to the comments as part of the proposed rulemaking package. See FIFRA sec. 25(a)(2); 7 U.S.C. 136w(a)(2). The statute contains a similar provision regarding consultation with the SAP --a body of independent expert scientific peer reviewers chartered under the Federal Advisory Committee Act (FACA) on all draft proposed regulations. See FIFRA sec. 25(d); 7 U.S.C. 136w(d). FIFRA further requires
EPA to publish a notice in the Federal Register simultaneously with the transmission of the proposed rule to USDA. FIFRA sec. 25(a)(2)(C); 7 U.S.C. 136w(a)(2)(C). In addition, the statute requires the agency to submit a copy of the proposed rule to the Agriculture Committees in the 
House and Senate. See FIFRA sec. 25(a)(3); 7 U.S.C. 136w(a)(3). The agency did not comply with any of these requirements. A very serious consequence of these procedural mistakes is to deprive the agency of a full understanding of how the proposed rulemaking might affect the regulation of pesticides and thereby affect agriculture, human health, and the environment. Had this proposal been referred to the SAP, EPN and other interested entities would have had the opportunity, according to SAP practices, for comment, a process which assures that the decision-makers obtain the widest possible input to these critically important decisions.
Comment [6155-2725]: Commenter (6125) notes that other provisions prescribe factors EPA must use in making decisions. FIFRA, 7 USC 136a-1(g)(1) requires that EPA "shall conduct a thorough examination of all data submitted under this section concerning an active ingredient ... and of all other available data found by the Administrator to be relevant." 
Like the express science provisions discussed above, both of these provisions are mandatory and prescribe specific factors EPA must assess. Neither offers any basis for failing to consider such factors or evidence regarding such factors because it is supported by science that is not "transparent." When Congress wished to limit available science to information that is publicly available, it has done so. Its failure to impose such limits on the use of data or science in the provisions discussed above is thus purposeful. As noted, the proposal does not even mention any of these relevant statutory requirements governing the use of science or data or information, much less explain how any of them could authorize EPA to make a decision without using information that is relevant to decision-making, solely for lack of transparency and regardless of reliability or relevance. None of these statutes provide EPA with any authority to pick and choose what reliable and useful science to use or ignore specific studies based on transparency. While the proposal misleadingly pays lip service to using the best available science, it would expressly "preclude" the use of reliable and available science that is not, in its characterization, "transparent," regardless of whether it is the best available science. The proposal does not even mention the governing statutory requirements, much less explain how its approach is consistent with them.
Comment [6163-1111]: Commenter (6133) provides that under the FIFRA, EPA cites 7 U.S.C. § 136r(a), which authorizes the Administrator to "undertake research." That section does not allow the restriction of what types of research EPA may consider in rulemakings or otherwise. Nor does it draw any distinction between dose-response data and other types of data. The absence of any such congressional distinction or restriction makes it all the more obvious that EPA invented and added the distinction and restrictions in the Proposal as a matter of its own policy preferences, contrary to the Act. This EPA may not do.
EPA also cites 7 U.S.C. § 136w, which is the general rulemaking authority that allows the Administrator to carry out the provisions of FIFRA. As explained above, that general grant of authority cannot support the rule, especially when the rule conflicts with the Act. Moreover, the citation fails to provide sufficient notice for the public to comment on the Proposal. FIFRA does not authorize the proposed rule. 
Comment [6163-1141]: Commenter (6133) states that FIFRA requires that all pesticides distributed or sold in the United States be registered by EPA. EPA cannot register pesticides that would cause "unreasonable adverse effects on the environment." 7 U.S.C. § 136a. Likewise it may "by regulation" limit the distribution, sale or use of a pesticide to prevent "unreasonable adverse effects on the environment," id. § 136a(a) and must cancel the registration of pesticides that cause such "unreasonable adverse effects." Id. § 136d. The term "unreasonable adverse effects on the environment" is defined to include unreasonable risks to human health, and dietary risks that violate the standard for pesticide residues under the Federal Food, Drug and Cosmetic Act (FFDCA). Id. § 136(bb). Given that registration decisions often depend heavily on dose-response data and models, EPA must clarify whether the Proposal will apply to registration and registration review decisions. If so, the Proposal conflicts with FIFRA's requirement that EPA determine whether pesticides proposed for registration would have unreasonable adverse effects on the environment. In light of that language, EPA cannot exclude relevant studies bearing on a pesticide's effect on human health or the environment simply because the underlying data cannot be made public. 
The potential applicability of the Proposal to exclude consideration of epidemiological studies of the health impacts of pesticides where the underlying data cannot be made public also highlights the logical inconsistency and arbitrary approach in the embodied proposed rule. On the one hand, the Proposal appears to be intended to prohibit consideration of such public health studies, but on the other hand seems to envision that industry-conducted studies and models claimed to include CBI would be allowed to be considered. This highlights the arbitrary and one-sided nature of the Proposal, and the clear underlying intent, which is to undermine public health protections for the benefit of industry. 
Regardless of whether the Proposal applies to registration decisions, it conflicts with FIFRA in other ways. FIFRA directs EPA, when promulgating rules, to "take into account the difference in concept and usage between various classes of pesticides, including public health pesticides, and differences in environmental risk and the appropriate data for evaluating such risk between agricultural, nonagricultural, and public health pesticides." Id. § 136w(a) (emphasis added); see also id. § 136w(c) (setting forth some examples of rules EPA may promulgate under FIFRA). EPA may not exclude "appropriate data" in these regulatory decisions simply because those data cannot be made public. Thus, the Proposal conflicts with FIFRA. 
Finally, EPA has violated FIFRA's procedural requirements. FIFRA requires EPA to provide the SAP and the Secretary of Agriculture with a copy of the Proposal at least 60 days before publication in the Federal Register. Id. § 136w(a)(2), (d). Any notification to the Secretary must be published in the Federal Register. Id. § 136w(a)(2)(D). There is no evidence in the Proposal that EPA followed these procedural requirements. (EPA also must provide the Panel and the Secretary a copy of the final rule 30 days before publication in the Federal Register. Id. § 136w(a)(2), (d).) Similarly, EPA must furnish a copy of the proposed and final regulation to the Committee on Agriculture of the House of Representatives and the Committee on Agriculture, Nutrition, and Forestry of the Senate. Id. § 136w(a)(3). Again, there is no evidence this occurred. 
Comment [6340-1461]: Commenter (6362) contends that EPA cites Section 25(a)(1) of the FIFRA, which does provide the Agency with broad authority to "prescribe regulations to carry out the provisions of this subchapter [FIFRA]." It should be noted, however, that the statutory language is a bit different from the other cited statutes and does not read as "as are necessary to carry out....". In addition, FIFRA Section 136r(a) does not relate to rulemaking and instead provides the Agency broad authority to undertake research necessary to carry out the purposes of FIFRA. As such, EPA may mistakenly have included Section 136r(a) to support the proposal as cited on 83 Fed. Reg. 18769.
Comment [6352-3512]: Commenter (6374) supports EPA's efforts to strengthen the integrity of its regulatory actions and recognizes the Agency's need to be able to independently validate the data that support many of its decisions. Under the FIFRA and the FFDCA, the data submitted by pesticide applicants and registrants in support of product approvals are subject to rigorous and well-defined EPA regulatory requirements that confirm their quality, validity, and integrity. To ensure that EPA's product approval decisions are based on reliable and verifiable science, the toxicological and exposure studies submitted by registrants in support of each pesticide registration must: 
 meet stringent data quality standards; 
 comply with EPA's good laboratory practice quality criteria; 
 use generally accepted methods, sufficient numbers of measurements (to achieve statistical reliability) and sufficient controls; and 
 achieve reproducible results.
The commenter appreciates EPA's statement that when dealing with science developed outside of the Agency, "it is the charge of the regulators to ensure that key findings are valid and credible, as required by OMB's Guidelines." 83 Fed. Reg. at 18770. We also appreciate EPA's statement that it "must also ensure that its decision-making is marked by independence, objectivity, transparency, clarity, and reproducibility" relying upon regulatory science. Data submitted by registrants in support of pesticide products regulated under FIFRA and FFDCA are subject to a well-developed and effective process that meets EPA's data quality goals. CropLife America (CLA) believes that all research EPA relies on to make pesticide regulatory decisions should be required to meet the same standards.
Notably, EPA already has full access to raw data underlying studies submitted by pesticide registrants under FIFRA and FFDCA. Consistent with the requirements of EPA's Good Laboratory Practice Standards (40 C.F.R. Part 160), all raw data underlying a study that supports a pesticide product registration or tolerance action must be retained and made available to EPA for inspection and copying. 40 C.F.R. §§ 160.15(a); 160.190(a). It is equally important to ensure the quality of other scientific information used by EPA to prepare the risk assessments that form the foundation for its pesticide registration decisions. CLA has consistently suggested that EPA require all scientific information used in pesticide risk assessments, including studies from academic and other non-registrant sources, meet the same high data quality standards that the Agency applies to registrant data submissions under FIFRA and FFDCA.
Comment [6352-3516]: Commenter (6374) encourages the Agency to develop procedures to ensure that data from non-registrants that are used to make pesticide regulatory decisions meet the same data access and quality standards required of registrants. At the same time, the commenter understands that EPA's April 30, 2018 proposal is specifically directed at those data underlying final regulations determined to be "significant regulatory actions" by the OMB, consistent with definitions, formal exclusions, and practices dating to 1993. As described below, this threshold provision appropriately excludes data submitted in support of individual pesticide product registration or tolerance decisions from the scope of the Agency's proposal. In so doing, EPA also is ensuring that it complies with the statutory framework established by Congress to carefully balance the requirements of public notice and transparency (which are accomplished through several statutory provisions under FIFRA and FFDCA) with the need to incentivize pesticide product innovation through the protection of proprietary data rights (also accomplished through specific provisions of FIFRA and FFDCA). 
Under these circumstances, the commenter asks EPA to expressly clarify that the proposed rule (and any final rule that EPA promulgates) does not include data submitted under FIFRA or FFDCA to support individual pesticide registration or tolerance decisions, consistent with its approach to other types of permits. By contrast, the scope would encompass more broadly applicable rulemakings under FIFRA or FFDCA that do meet the definition of "significant regulatory action."
Comment [6352-3522]: Commenter (6374) provides that the scope of EPA's newly proposed transparency regulations is set forth at proposed 40 C.F.R. § 30.3, which provides that: 
([t]he provisions of this subpart apply to dose response data and models underlying pivotal regulatory science that are used to justify significant regulatory decisions. . . [T]he provisions of this subpart do not apply to any other type of agency action, including . . . permit proceedings.)
As separately described at proposed 40 C.F.R. § 30.2, the term "regulatory decisions" means "final regulations determined to be significant regulatory actions" by the OMB pursuant to Executive Order (EO) 12866. 
Pesticides are among the most highly regulated products in the United States, and FIFRA requires the generation and submission of extensive amounts of human health and environmental data to allow EPA to make registration decisions. The amount and types of data required to register a specific pesticide are functions of both the attributes of the compound and the uses the applicant seeks to register. Even after EPA's initial approval, each pesticide product remains subject to comprehensive EPA review and oversight, requiring substantial additional data on an ongoing basis to maintain each registration. FIFRA includes requirements for public notice and access to information (see, e.g., FIFRA §§ 3(c)(2)(A), 3(c)(4), 10(d)). 
Pesticide product registration actions are not, however, "significant regulatory actions" under EO 12866. Specifically, EO 12866 defines a "significant regulatory action" to mean any regulatory action that is likely to result in a rule meeting certain specified criteria (i.e., any substantive action by an agency that promulgates or is expected to lead to the promulgation of a final rule or regulation). 58 Fed. Reg. 51735 (Oct. 4, 1993). 
EPA's pesticide registration approvals under FIFRA do not involve the promulgation of any rule or regulation with broad application. Instead, EPA issues individual pesticide product registrations as specific licenses or permits, based on the Agency's evaluation of the data required for the specific license. Part 158 of EPA's regulations describes "the minimum data and information EPA typically requires to support an application for pesticide registration ...." The general types of data required include:
 Product chemistry data to evaluate the chemical characteristics of the product;
 Toxicology data to measure potential human health effects of exposure to the pesticide, including tests conducted on laboratory animals; 
 Ecological effects data to measure the product's potential impact on exposed biological systems, including wildlife, aquatic organisms, and non-target plant effects data; 
 Human exposure data to measure the amount of direct exposure to the product experienced by agricultural workers and others who use the product; 
 Environmental fate data to evaluate the product's behavior and persistence in environmental media, including soil and groundwater; and 
 Residue chemistry data to measure the amounts and chemical constituents of pesticide residues remaining on treated crops used for food or animal feed. 
See, e.g., 40 C.F.R. Part 158 Subparts D, F, G, K, N, and O. All pesticide data submissions must demonstrate compliance with EPA's good laboratory practice standards, as required by 40 C.F.R. § 160.12. 
EPA's individual pesticide registration decisions, like other permit decisions, are not notice-and-comment rulemaking actions and, as such, have never been subject to EO 12866 review requirements. 
For pesticides registered for use on food crops, EPA must also establish tolerances or tolerance exemptions as required by FFDCA. Unlike pesticide product registration decisions, these tolerance actions are issued by formal notice-and-comment regulation. FFDCA also includes requirements for public notice and access to information that are consistent with FIFRA (see, e.g., FFDCA §§ 408(d)(3), 408(e)(2), 408(i). 
By virtue of longstanding OMB policy, regulatory actions regarding specific pesticide tolerances, temporary tolerances, and tolerance exemptions (with the exception of actions that make an existing tolerance more stringent) are expressly exempt from EO 12866's definition of "regulatory action." OMB, Guidance for Implementing EO 12866 (Oct. 12, 1993), at App. C. Consistent with this guidance, EPA has exempted all tolerances and tolerance exemptions from EO 12866 for decades. 
The two categories of decisions above stand in contrast to more broadly applicable pesticide regulations determined by OMB to meet the definition of "significant regulatory action," and which therefore, like other non-exempt regulations, are subject to the requirements of EO 12866. See, e.g., "Revisions to EPA's Rule on Protections for Subjects in Human Research Involving Pesticides," 76 Fed. Reg. 5735 (Feb. 2, 2011) (stating that "[u]nder Executive Order 12866, this has been identified as a `significant regulatory action'"); "Regulations Under FIFRA for Plant-Incorporated Protectants," 66 Fed. Reg. 37772 (July 19, 2001) (stating that "[p]ursuant to Executive Order 12866, OMB has determined that this proposed rule is a `significant regulatory action' because this action might raise novel legal or policy issues arising out of legal mandates, the President's priorities, or the principles set forth in the Executive Order"). 
For these threshold reasons, the commenter understands that pesticide product registration data would not be subject to EPA's proposed transparency requirements. As further described below, this exclusion is mandated by law.
Comment [6352-3525]: Commenter (6374) asserts that the exclusion of pesticide product registration and tolerance data from the scope of EPA's proposal is mandated by the overriding statutory requirements that EPA is charged with administering. Specifically, EPA's disclosure of data submitted in support of pesticide registration actions must comply fully with FIFRA's special protections, which recognize the need to protect the confidentiality of certain information that relates to trade secrets or commercial or financial information and includes: 
 A prohibition against EPA's disclosure of any "information which in the Agency's judgment contains or relates to trade secrets or commercial or financial information obtained from a person and privileged and confidential";
 A requirement that EPA provide advance written notice to an applicant or registrant if it proposes to disclose information claimed as confidential; and 
 A prohibition against EPA's disclosure of any information that discloses: (i) "manufacturing or quality control processes," (ii) "the details of any methods for testing, detecting or measuring the quantity of any deliberately added inert ingredient of a pesticide," or (iii) "the identity or percentage quantity of any deliberately added inert ingredient of a pesticide."
Alongside these significant confidentiality protections, FIFRA also provides its own transparency mandate for pesticide registration data with two important features. First, it requires that EPA make available to the public information concerning the objectives, methodology, results, or significance of any test or experiment performed on or with a registered pesticide, and any information concerning the effects of such pesticide on any organism or the behavior of such pesticide in the environment. FIFRA § 10(d)(1).
Second, this broad disclosure mandate for pesticide registration data is subject to an essential limitation. Recognizing the proprietary nature of pesticide studies and the possibility that industry competitors might obtain and seek to improperly use copies of another submitter's unpublished data to satisfy its own pesticide regulatory requirements in other jurisdictions, FIFRA expressly prohibits EPA from disclosing studies submitted by an applicant or registrant to any employee or agent of a foreign or multinational pesticide company. FIFRA § 10(g)(1). Underscoring the importance of this prohibition, FIFRA requires that EPA obtain an affirmation (which must be submitted subject to criminal penalties under 18 U.S.C. § 1001) from any person seeking access to pesticide registration data to ensure that such person will not deliver or offer the data to any such business or entity. FIFRA §§ 10(g)(1), (3). Specifically, before EPA will provide a member of the public with access to a pesticide submission, the requestor must affirm that he or she: 
 is not a business or other entity engaged in the production, sale, or distribution of pesticides in countries other than the United States or in addition to the United States (nor an employee or agent of any such business or entity);
 does not seek access to the information for purposes of delivering it or offering it for sale to any such business or entity or to any of its employees or agents; and
 will not purposefully deliver the information (or negligently cause it to be delivered) to any such entity or to any of its employees or agents. 
EPA has confirmed that it would be unlawful for a person to execute the required affirmation while having the intent to publish or otherwise deliver a FIFRA data submitter's previously unpublished data submission to a foreign or multinational pesticide producer, explaining that "the information delivered likely could be used to satisfy a data requirement imposed by a foreign country as a condition of producing, distributing, selling, shipping, or using the product in that country." See, EPA, "Limitations on Disclosure of Information Under Pesticide Law" (available at: https://www.epa.gov/foia/limitations-disclosure-information-under-pesticide-law). 
FFDCA § 408(i) extends the same protections as FIFRA's to data submitted in support of EPA tolerance decisions. 
The Agency's April 30, 2018, proposed transparency requirements, like all regulations issued by EPA, must comply fully with FIFRA's and FFDCA's protections for pesticide data. EPA has successfully administered these data disclosure and confidentiality requirements for many years, ensuring that high-quality, verifiable data are available to the Agency from applicants and registrants while balancing the important need to make pesticide data available for public review and at the same time preventing disclosure to industry competitors. Accordingly, EPA's proposed transparency requirements should explicitly incorporate the protections specifically guaranteed to pesticide registration data under FIFRA and FFDCA.
Comment [6353-2329]: Commenter (6375) provides that EPA solicits comment on whether alternative or additional regulatory or other policy vehicles are appropriate to establish and implement these policies, and whether further regulatory or other policy vehicles at the programmatic or statutory level would be appropriate as alternative or additional steps the agency may take to further the policies articulated in Section II. (pg. 18,771).
The regulatory language that EPA proposes provides general requirements that apply to the Agency's use of science under the many statutes that it implements. The generality of the proposed regulations may be a function of the diverse purposes of those statutes Specifically, some of these statutes focus on the introduction of new substances into the marketplace (e.g., FIFRA and TSCA). The scientific analyses required for these different programs and the sources of the scientific information underlying those analyses differ. The Agency should consider supplementing the general language that it has proposed with statutory specific regulations. A statutory-specific rulemaking would provide greater certainty going forward.
Comment [6424-2639]: Commenter (6446) contends that the proposed rule also deviates from accepted practices that EPA routinely uses, and is required to use, to evaluate science it relies on for regulatory decisions. EPA has long recognized that proprietary CBI submitted to it cannot be shared with the public, yet it routinely relies on that information, including scientific studies and other scientific information, as the foundation for important regulatory decisions. (See, e.g., FIFRA at 7 U.S.C. 136h, subd. (b).) EPA also adheres to specific Congressional directives regarding the science it should consider for regulatory action, including directives to consider whether the scientific information is reasonable, clear, complete and whether it has been peer reviewed.
Comment [6894-3610]: Commenter (6915) provides that although Section 25 of the FIFRA directs that regulations "take into account . . . the appropriate data for evaluating [] risk," 7 U.S.C. 136w (emphasis added), it would be arbitrary and capricious to define "appropriate" to exclude from consideration relevant and valid scientific studies, as EPA proposes to do in this rulemaking. Requirements to review the "latest" and "appropriate" scientific data are not carte blanche to impose new, unscientific limits on that data.
Comment [8750-1815]: Commenter (9227) notes that the Proposal lists FIFRA section 25 as an authority for the rulemaking. The agency, however, has already failed to follow several required procedures for issuing a valid regulation under this section of FIFRA. FIFRA section 25 requires the agency to seek comments from the Secretary of Agriculture on all draft proposed regulations 60 days prior to signing a proposed regulation for publication, and 30 days prior to publication for a final rule. If the Secretary of Agriculture provides comments, the Administrator must also respond in writing as part of the proposed rulemaking package. FIFRA additionally requires EPA to publish a notice in the Federal Register simultaneously with the transmission of the proposed rule to USDA. And the statute requires the agency to submit a copy of the proposed rule for comment to the SAP, as well as a copy to the Agriculture Committees in the House and Senate any time the agency is required to consult with the Secretary of Agriculture. This means that EPA here should have provided both committees and the SAP with a copy of the proposed regulation at least 60 days prior to publication of the Proposal in the Federal Register. The agency did not comply with any of these requirements and does not indicate that it will in any final rule. The Proposal is therefore unlawful.
To be sure, in some instances the Administrator and Secretary may together agree to waive some of the consultation requirements among themselves, but there is no indication that Administrator Pruitt did that with this Proposal. And even if the Administrator and Secretary later agree to waive the consultation requirement section 25(a)(2)(A) and (B), that waiver would not alter EPA's obligation to provide the SAP and the House and Senate Committees with a copy of the regulation. Nor would it change the fact that the Administrator illegally issued the Proposal without consulting the Secretary of Agriculture. A very serious consequence of these procedural mistakes is to deprive the agency of a full understanding of how the proposed rulemaking might affect the regulation of pesticides and thereby affect agriculture, human health, and the environment. Therefore, the only lawful path forward here is for the Agency to withdraw the Proposal, consult with the entities required by FIFRA, and then subsequently re-notice the Proposal.
Comment [8750-1964]: Commenter (9227) provides that EPA arbitrarily failed to consider the far-reaching implications this Proposal could have on inter-agency coordination and consultation given that other agencies normally rely on research potentially excluded by the Proposal.
      Consultation Provisions in FIFRA
      136w(a)(2)(A)
      Authority of the Administrator: Procedure: Proposed regulations
      (A) Proposed Regulations: 
      At least 60 days prior to signing any proposed regulation for publication in the Federal Register, the Administrator shall provide the Secretary of Agriculture with a copy of such regulation. If the Secretary comments in writing to the Administrator regarding any such regulation within 30 days after receiving it, the Administrator shall publish in the Federal Register (with the proposed regulation) the comments of the Secretary and the response of the Administrator with regard to the Secretary's comments. If the Secretary does not comment in writing to the Administrator regarding the regulation within 30 days after receiving it, the Administrator may sign such regulation for publication in the Federal Register any time after such 30-day period notwithstanding the foregoing 60-day time requirement.
      ***
      136w(a)(2)(B)
      Authority of the Administrator: Final Regulations
      At least 30 days prior to signing any regulation in final form for publication in the Federal Register, the Administrator shall provide the Secretary of Agriculture with a copy of such regulation. If the Secretary comments in writing to the Administrator regarding any such final regulation within 15 days after receiving it, the Administrator shall publish in the Federal Register (with the final regulation) the comments of the Secretary, if requested by the Secretary, and the response of the Administrator concerning the Secretary's comments. If the Secretary does not comment in writing to the Administrator regarding the regulation within 15 days after receiving it, the Administrator may sign such regulation for publication in the Federal Register at any time after such 15-day period notwithstanding the foregoing 30-day time requirement. In taking any final action under this subsection, the Administrator shall include among those factors to be taken into account the effect of the regulation on production and prices of agricultural commodities, retail food prices, and otherwise on the agricultural economy, and the Administrator shall publish in the Federal Register an analysis of such effect
      ***
      136w(a)(3)
      Authority of the Administrator: Procedure: Congressional Committees
      At such time as the Administrator is required under paragraph (2) of this subsection to provide the Secretary of Agriculture with a copy of proposed regulations and a copy of the final form of regulations, the Administrator shall also furnish a copy of such regulations to the Committee on Agriculture of the House of Representatives and the Committee on Agriculture, Nutrition, and Forestry of the Senate.
      ***
      136w(a)(4)
      Authority of the Administrator
      Simultaneously with the promulgation of any rule or regulation under this subchapter, the Administrator shall transmit a copy thereof to the Secretary of the Senate and the Clerk of the House of Representatives. The rule or regulation shall not become effective until the passage of 60 calendar days after the rule or regulation is so transmitted.
      ***
      136w-3
      Identification of Pests; cooperation with USDA
      The Administrator, in coordination with the Secretary of Agriculture, shall identify those pests that must be brought under control. The Administrator shall also coordinate and cooperate with the Secretary of Agriculture's research and implementation programs to develop and improve the safe use and effectiveness of chemical, biological, and alternative methods to combat and control pests that reduce the quality and economical production and distribution of agricultural products to domestic and foreign consumers.
      ***
      136(r)(a)
      Research and Monitoring: Research
      The Administrator shall undertake research including research by grant or contract with other Federal agencies, universities, or others as may be necessary to carry out the purposes of this subchapter, and the Administrator shall conduct research into integrated pest management in coordination with the Secretary of Agriculture. The Administrator shall also take care to ensure that such research does not duplicate research being undertaken by any other Federal agency.
      ***
      136a-1(n)(2)-(3)
      Reregistration of registered pesticides: Authorization of funds to develop public health data
      (2) Consultation. In the case of a pesticide registered for use in public health programs for vector control or for other uses the Administrator determines to be human health protection uses, the Administrator shall, upon timely request by the registrant or any other interested person, or on the Administrator's own initiative may, consult with the Secretary [of HHS] prior to taking final action to suspend registration under section 3(c)(2)(B)(iv) or cancel a registration under section 4, 6(e), or 6(f). In consultation with the Secretary, the Administrator shall prescribe the form and content of requests under this section.
      (3) Benefits to support family. The Administrator, after consulting with the Secretary, shall make a determination whether the potential benefits of continued use of the pesticide for public health or health protection purposes are of such significance as to warrant a commitment by the Secretary to conduct or to arrange for the conduct of the studies required by the Administrator to support continued registration under section or reregistration under section 4
      ***
      7 USCS 136(ll) (2) 
      Definitions: Minor Use
      (2) the Administrator, in consultation with the Secretary of Agriculture, determines that, based on information provided by an applicant for registration or a registrant, the use does not provide sufficient economic incentive to support the initial registration or continuing registration of a pesticide for such use and--
      ***
      136i(a)(1)
      Use of restricted use pesticides; applicators
      Requires the Administrator to consult with Governor of each state to conduct a program for the certification of use of specific pesticides.
      ***
      136a(c)(1)(F)(ii)
      Registration of Pesticides: Procedure for registration
      The period of exclusive data use provided under clause (i) shall be extended 1 additional year for each 3 minor uses registered after the date of enactment of this clause [enacted Aug. 3, 1996] and within 7 years of the commencement of the exclusive use period, up to a total of 3 additional years for all minor uses registered by the Administrator if the Administrator, in consultation with the Secretary of Agriculture, determines that, based on information provided by an applicant for registration or a registrant, that--(I) there are insufficient efficacious alternative registered pesticides available for the use; (II) the alternatives to the minor use pesticide pose greater risks to the environment or human health; (III) the minor use pesticide plays or will play a significant part in managing pest resistance; or (IV) the minor use pesticide plays or will play a significant part in an integrated pest management program.
      ***
      136t(b)
      Delegation and Cooperation
      (b) Cooperation. The Administrator shall cooperate with the USDA, any other Federal agency, and any appropriate agency of any State or any political subdivision thereof, in carrying out the provisions of this Act and in securing uniformity of regulations.
      ***
      136o(e)
      Imports and Exports
      Secretary of the Treasury shall prescribe regulations for this section in consultation with the Administrator.
      ***
      136p
      Exemption of Federal and State Agencies
      The Administrator may, at the Administrator's discretion, exempt any Federal or State agency from any provision of this Act if the Administrator determines that emergency conditions exist which require such exemption. The Administrator, in determining whether or not such emergency conditions exist, shall consult with the Secretary of Agriculture and the Governor of any State concerned if they request such determination.
      ***
      136w-7
      USDA Minor Use Program
      (A) Grant authority. The Secretary, in consultation with the Administrator, shall establish a program to make grants for the development of data to support minor use pesticide registrations and re-registrations. The amount of any such grant shall not exceed 1/2 of the cost of the project for which the grant is made.
      ***
      136i-1(a)(1)
      Pesticide Recordkeeping
      The Secretary of Agriculture, in consultation with the Administrator of the EPA, shall require certified applicators of restricted use pesticides
      ***
      136i-2(c)
      Collection of Pesticide Use Information Coordination.
      The Secretary of Agriculture shall, as appropriate, coordinate with the Administrator of the EPA in the design of the surveys and make available to the Administrator the aggregate results of the surveys to assist the Administrator.
Comment [9078-3466]: Commenter (6194) asserts that given the complexity and potentially broad-reaching impact of the Proposal, the limited time allotted for review of the Proposal by the OMB's OIRA suggests that the process was intended to be pro forma as opposed to meaningful. Yet, despite this truncated review, the Proposal was changed significantly during this period. To the extent that any of those changes were made by OIRA, it would have exceeded the office's authority under Executive Order 12,866, which charges OIRA with conducting cost benefit analyses, not changing the substantive content of agencies' regulations. To the extent the changes were made by EPA, it is unclear on what version of the proposal OIRA conducted a cost benefit analysis and whether the version of the Proposal released to the public was reviewed by OIRA. Furthermore, as reported by Greenwire, former EPA Administrator Pruitt signed the Proposal before OIRA even completed its review; raising further doubt about the opportunity for any meaningful review by the OMB. These concerns are addressed in greater detail by the Union of Concerned Scientists in a blog post from May 7, 2018.
EPA is also supposed to provide OIRA, and the public, an "assessment" and the "underlying analysis" of the anticipated benefits and costs of the proposal. E.O. 12,866, § 6(a)(3)(B)-(C), (E)(i). Yet the Proposal contains no serious attempt at assessing either. With respect to benefits, the Proposal contains a single sentence, stating that it will improve "the scientific quality of the Agency's actions and facilitate expanded data sharing and exploration of key data sets." 83 Fed. Reg. at 18,772. This assertion is questionable at best and presented without any underlying analysis. Regarding costs, the Proposal does not attempt an independent assessment, indicating only that "[t]his action should be implemented in a cost-effective way" and referencing an analysis by the Mercatus Center. Id. However, a CBO analysis of a similar legislative proposal concluded that it could cost hundreds of millions of dollars a year; the Proposal does not address or distinguish this analysis. EPA did not provide OIRA, and has not provided the public, with sufficient information for an analysis of the Proposal's benefits and costs.
Similar questions exist as to whether EPA solicited sufficient, if any, interagency review of the Proposal. For example, the Proposal cites the rulemaking provision of the FIFRA as a source of authority; this provision requires EPA to seek comments from the Secretary of Agriculture and the FIFRA SAP on draft regulations. 7 U.S.C. § 136w. EPA did not seek such input prior to publication of the Proposal. Such interagency review would be particularly important with respect to other agencies whose work could be impacted by the Proposal (see discussion in Section V below).
EPA's own SAB, which Congress created in 1978 to provide scientific advice to EPA, 42 U.S.C. § 4365(a), challenged EPA's failure to solicit SAB's input on the Proposal -- pointing out that it was not even aware of the Proposal until it was published in the Federal Register and discussed in news articles. Given that EPA should have provided the Proposal to another federal agency to review, EPA was also required to make the Proposal available to the SAB, along with any relevant scientific and technical information on which the Proposal was based. 42 U.S.C. § 4365(c). A SAB Work Group concluded that the Proposal deals with a "myriad of scientific issues for which the Agency should seek expert advice from the SAB," explaining that:
 "There are . . . sensitive situations where public access may infringe on legitimate confidentiality and privacy interests, and where exceptions from complete public access may be appropriate. In addition, there are considerations associated with the cost and effort that would be involved in making large and complex existing datasets available."
 "[T]he precise design of the rule appears to have been developed without a public process for soliciting input from the scientific community. Nor does the preamble to the rule describe precisely how the proposal builds on previous efforts to promote transparency such as the Information Quality Act and EPA's Information Quality Guidelines."
 "The proposed rule does not include any assessment of the impact of data restrictions on existing or future regulatory programs. . . Furthermore, the rule could have the effect of removing legal, ethical, and peer-reviewed studies of health effects as sources to support the agency's regulatory efforts."
Even when EPA is not required to seek SAB's opinion, failing to utilize SAB's expertise is inconsistent with the purpose of SAB, which is to "to provide independent advice and peer review to EPA's Administrator on the scientific and technical aspects of environmental issues." The SAB's expertise and input is similarly valuable to the public, and should be available well before the end of the public comment period so as to inform discussion.
Comment [9217-3563]: Commenter (9225) contends that equally problematic, the proposed rule is not authorized by any authority delegated to EPA by Congress and is contrary to a number of statutes under EPA's authority. This includes, but is not limited to, the CAA; CWA; TSCA; LCSA; SDWA; FIFRA; and more. 
Comment [6857-2945]: Commenter (6879) provides that EPA rules and regulations have been critical to the protection of wildlife, natural habitats, and the ecosystem services they provide to communities and the economy. This includes protection of endangered and other wildlife from pesticides and other toxic substances under FIFRA.
These laws require EPA to use the "best available" science, or all "reasonably available" science and information. By excluding high-quality peer-reviewed and validly published science from the regulatory decision-making process, the proposed regulation directly contravenes the specific mandate of these statutes. And by deliberately excluding the best science available, the rule also contravenes the Congressional mandates for these statutes and undermines their public health and environmental protection purposes.
Comment [6163-1276]: Commenter (6133) states that in addition to addressing how and whether the agency will consider science, the Proposal also contains a seemingly unrelated provision regarding agency peer review. The Proposal, in § 30.7, reads: 
      What role does independent peer review in this section?
      EPA shall conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein. 
      Because transparency in regulatory science includes addressing issues associated with assumptions used in models, EPA shall ask peer reviewers to articulate the strengths and weaknesses of EPA's justification for the assumptions applied and the implications of those assumptions for the results. (83 Fed. Reg. at 18,774.) 
There is no statutory authority for EPA to "conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein." The federal statutes that EPA lists as putative authority for the Proposal provide no authority for proposed § 30.7. See FIFRA sections 25(a)(1), 136r(a), 7 U.S.C. 136r(a), 136w. The claimed authorities that EPA lists do not mention peer review or even allude to the concept. Id. Neither the Proposal nor accompanying docket materials identify any provision of any federal statute that authorizes EPA to promulgate proposed § 30.7. Our own research revealed no provision of any federal statute that authorizes EPA to promulgate proposed § 30.7. 
When Congress writes federal statutes, Congress knows how to create legal authority for peer review, who shall conduct that peer review, what role, if any, that EPA or other parties will play, and how that peer review may be conducted. None of the provision in the law authorize EPA's Proposal in § 30.7. Instead, the statutes require that EPA use peer-reviewed science, regardless of whether it would meet EPA's definition of the term in § 30.7. And in cases where the law requires EPA to conduct the review, the statutes often spell out specifically how that should happen.
The commenter provides provisions for peer review for several statutes. Only the section relevant to FIFRA is excerpted here.
      § 136w. Authority of Administrator 
      (e) Peer review. The Administrator shall, by written procedures, provide for peer review with respect to the design, protocols, and conduct of major scientific studies conducted under this subchapter by the EPA or by any other Federal agency, any State or political subdivision thereof, or any institution or individual under grant, contract, or cooperative agreement from or with the EPA. In such procedures, the Administrator shall also provide for peer review, using the advisory panel established under subsection (d) of this section or appropriate experts appointed by the Administrator from a current list of nominees maintained by such panel, with respect to the results of any such scientific studies relied upon by the Administrator with respect to actions the Administrator may take relating to the change in classification, suspension, or cancellation of a pesticide. Whenever the Administrator determines that circumstances do not permit the peer review of the results of any such scientific study prior to the Administrator's exercising authority under section 136d(c) of this title to immediately suspend the registration of any pesticide to prevent an imminent hazard, the Administrator shall promptly thereafter provide for the conduct of peer review as provided in this sentence. The evaluations and relevant documentation constituting the peer review that relate to the proposed scientific studies and the results of the completed scientific studies shall be included in the submission for comment forwarded by the Administrator to the advisory panel as provided in subsection (d). As used in this subsection, the term "peer review" shall mean an independent evaluation by scientific experts, either within or outside the EPA, in the appropriate disciplines. (7 U.S.C. § 136w (emphasis added)). 
      § 136w-8. Pesticide registration service fees 
      (a) Definition of costs in this section, the term "costs", when used with respect to review and decision-making pertaining to an application for which registration service fees are paid under this section, means-- (1) costs to the extent that -- 
      (A) officers and employees provide direct support for the review and decision-making for covered pesticide applications, associated tolerances, and corresponding risk and benefits information and analyses; (B) persons and organizations under contract with the Administrator engage in the review of the applications, and corresponding risk and benefits information and assessments; and (C) advisory committees and other accredited persons or organizations, on the request of the Administrator, engage in the peer review of risk or benefits information associated with covered pesticide applications; (2) costs of management of information, and the acquisition, maintenance, and repair of computer and telecommunication resources (including software), used to support review of pesticide applications, associated tolerances, and corresponding risk and benefits information and analyses; and (3) costs of collecting registration service fees under subsections (b) and (c) and reporting, auditing, and accounting under this section. (7 U.S.C. § 136w-8 (emphasis added)). 
Under longstanding federal case law, when Congress authorizes an approach in one section of a statute using specific language but does not do the same in other sections of a statute, courts presume that Congress acted purposefully and did not mean to address or authorize that approach in those other statutory sections. See, e.g., Dean v. United States, 556 U.S. 568 (2009) ("It is generally presumed that Congress acts intentionally when including particular language in one section of a statute but not in another." (citing Russello v. United States, 464 U.S. 16, 23 (1983)). Not only are there no implied grants of authority to an agency in the other statutory sections, but the congressional decisions to authorize the approach elsewhere in the statute give even greater force to the conclusion that the agency has not been given authority where Congress did not use the same or similar authorizing language.
The EPA approach proposed in § 30.7 is even more unlawful than would be the case, independently, under this case law. Proposed § 30.7 says "EPA shall conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein." 83 Fed. Reg. at 18,774/2 (emphasis added). EPA proposes to bind itself ("shall") to conduct a particular form of peer review on "all pivotal regulatory science" based on an unenforceable, non-binding OMB bulletin that has never been the subject of notice and comment rulemaking and that itself is not authorized by any federal law. To the contrary, treating the content of the unenforceable bulletin as a binding regulation would itself violate the federal statutes that EPA implements, because those statutes do not codify the bulletin, and EPA would be unlawfully codifying a mere policy preference. This EPA may not do.
This Proposal is unlawful, arbitrary, and capricious, and an abuse of EPA discretion. To reiterate, the Proposal identifies no statutory authority for EPA to conduct independent peer review for any, much less all, "pivotal regulatory science" consistent with the dense content (and exceptions) of the OMB bulletin. The Proposal identifies no statutory authority for EPA to bind itself, and future administrations, to conduct peer review only in this fashion, unless and until future notice-and-comment rulemaking is undertaken. The Proposal identifies no suggestion in statutory language or legislative history that Congress intended EPA to conduct binding peer review consistent with this OMB bulletin, notwithstanding that Congress has known about this bulletin since 2005. EPA has simply made up proposed § 30.7 -- with its the link to the OMB bulletin, and the putative authority for the Proposal -- out of whole cloth. This EPA may not do.
Comment [6112-1605]: Commenter (6137) asserts that not only does the Proposed Rule contravene the statutes upon which EPA relies as statutory authority, but its limitations on science likewise conflict with core provisions of other environmental statutes. For example, the Food Quality Protection Act (FQPA), which regulates pesticide residue in conjunction with FIFRA, sets safety standards based on the consideration of all available data. Limiting the data available to conduct studies necessary to evaluate the harmful effects of pesticides runs counter to this mandate.
Specifically, Congress overhauled our food safety laws when it unanimously passed the FQPA, amending both FFDCA and FIFRA. The overhaul responded to a seminal 1993 NAS report criticizing EPA for treating children like "little adults" by failing to address the unique susceptibility of children to pesticide exposures based on the foods they eat, their play, metabolism, and sensitive stages of their development. NAS, Pesticides in the Diets of Infants and Children (1993). The NAS recommended that EPA revamp and strengthen its pesticide regulations to account for children's vulnerabilities, consumption patterns, and exposures. Because it would take time to fill gaps in knowledge, safeguards, and methodologies, the NAS recommended that additional protection be afforded in the form of "uncertainty" or "safety factors." The NAS first described how EPA has regularly used uncertainty factors and then proposed an additional uncertainty factor for fetal developmental toxicity and where data are incomplete: "In the absence of data to the contrary, there should be a presumption of greater toxicity to infants and children. To validate this presumption, the sensitivity of mature and immature individuals should be studied systematically to expand the current limited data base on relative sensitivity." Id.
The FQPA strengthened the food safety standard in several ways. First, under the FQPA, the EPA Administrator "may establish or leave in effect a tolerance for a pesticide chemical residue in or on food only if the Administrator determines that the tolerance is safe. The Administrator shall modify or revoke a tolerance if the Administrator determines it is not safe." 21 U.S.C. § 346a(b)(2)(A)(i). In other words, the absence of sufficient information to find a pesticide safe means it cannot be allowed in or on our food. 
Second, safe "means the Administrator has determined there is a reasonable certainty that no harm will result from aggregate exposure to the pesticide chemical residue, including all anticipated dietary exposures and all other exposures for which there is reliable information." Id. § 346a(b)(2)(A)(ii). The FQPA, therefore, requires that EPA conduct an assessment based on aggregation of all exposures to a pesticide whether from eating foods, drinking water with residues of the pesticide, or uses of the pesticide in and around the home or other places where people can be exposed. 21 U.S.C. § 346a(b)(2)(A)(ii), (C)(i)(I) & (ii). The FQPA also requires EPA to assess and protect against unsafe risks posed by cumulative exposures to pesticides that share a "common mechanism of toxicity," as is the case with pesticides in the organophosphate, carbamate, and pyrethroid families. See 21 U.S.C. § 346a(b)(2)(C)-(D). 
Third, EPA must make specific safety determinations for infants and children. Id. § 346a(b)(2)(C)(ii)(I) & (II). It must consider available information concerning "the special susceptibility of infants and children," including "neurological differences between infants and children and adults, and effects of in utero exposure to pesticide chemicals." Id. § 346a(b)(2)(C)(i)(II). EPA must also base its tolerance decisions on available information about food "consumption patterns among infants and children." Id. § 346a(b)(2)(C)(i)(I) & (III).
Fourth, EPA must account for children's sensitivities, scientific uncertainty, and gaps in available data. The statute requires that "an additional tenfold margin of safety for the pesticide chemical residue and other sources of exposure shall be applied for infants and children to take into account potential pre -and post-natal toxicity and completeness of the data with respect to exposure and toxicity to infants and children." Id. § 346a(b)(2)(C). EPA can depart from this requirement and use a different margin of safety "only if, on the basis of reliable data, such margin will be safe for infants and children." Id.; see Nw. Coal. for Alts. to Pesticides v. EPA, 544 F.3d 1043, 1046 (9th Cir. 2008) (reversing EPA's shrinkage of the safety factor in the absence of supporting data). 
As an over-arching mandate, the FQPA directs EPA to make its tolerance determinations based on its assessment of the pesticide's risk. 21 U.S.C. § 346a(b)(2)(C)(i). And throughout its mandates to assess the full effects of a pesticide, the FQPA directs EPA to base its risk assessment on "available information" about consumption patterns, special susceptibility of infants and children, and cumulative effects. Id. The FQPA also expressly directs EPA to consider the validity, completeness, and reliability of the available data from studies, the nature of the toxic effect, available information about the relationship between study results and human risk, and available information about aggregate and cumulative effects. Id. § 346a(b)(2)(C)(ii). 
The Proposed Rule upends these mandates by requiring EPA to put on blinders and ignore a huge and important subset of the available data. It collides squarely with the congressional direction to consider all available data and information in order to protect our food and children in particular. It also conflicts with the congressional mandate to afford greater protection to children and our food when gaps in data prevent a full quantitative assessment and establishment of a dose-response. It thus undermines EPA's ability to carry out its obligations under the FQPA.
1.3.2.2 FIFRA (SNPRM Comments)
Comment [48-643]: Commenter (12727) contends that, as with the original proposal, the Supplemental Notice fails to comply with the pre-proposal review requirements set forth in the FIFRA. EPA continues to cite section 25 of FIFRA as a source of authority for this proposed action. Section 25, however, requires the agency to seek comments from the Secretary of Agriculture on all draft proposed regulations 60 days prior to signing a proposed rule. EPA is to publish this solicitation in the Federal Register, and respond to any written comments from the Secretary as part of the Federal Register proposal. FIFRA also requires that any time the EPA is required to consult with the Secretary of Agriculture, the agency must also submit a copy of the proposed rule for comment to the Agriculture Committees in the House and Senate at least 60 days prior to publication, and provide the SAP with an opportunity to provide comment on the health and environmental impacts of the proposed action.
There is no indication that EPA has satisfied any of these statutory obligations. Pre-proposal review by the Secretary, however, would at least have raised the question of what potential impacts the proposal might have on FIFRA programs -- an issue on which the Supplemental Notice is mute. Congress, by requiring these consultation steps, evidently viewed them as necessary and important. EPA's disregard of these requirements is thus consequential, arbitrary, and unlawful.
Comment [73-2080]: Commenter (11576) contends that the current rulemaking process has also violated the requirements of other statutes the EPA may or may not be relying on in attempting to issue the proposed rule. Under FIFRA, for instance, the EPA is required to provide the Secretary of Agriculture with a copy of any proposed rule for review and comment at least 60 days before a proposed rule is signed for publication in the Federal Register. The EPA's refusal to satisfy these requirements cannot be allowed to stand.
Comment [82-835]: Commenter (12464) asserts that the Supplemental Notice's claim that the Proposal is not "substantive" rests on the assertion that the rule "would not regulate the conduct or determine the rights of any entity outside the federal government" and that "it exclusively pertains to the internal practices of EPA." 85 Fed. Reg. at 15,398. Past court decisions, however, have found that EPA's refusal to consider certain scientific studies in its regulatory decision-making processes carry the force and effect of law.
In CropLife America v. U.S. EPA, for example, pesticide manufacturers and trade associations sought judicial review of an EPA directive that provided that the agency, contrary to its established practice, would no longer consider or rely on third-party human studies in evaluating the safety of pesticides under FIFRA. Petitioners argued, and the D.C. Circuit agreed, that this directive was a substantive rule with the force of law, that the rule violated FIFRA's requirement that EPA "consider all relevant reliable data," and that the rule was arbitrary and capricious in violation of the APA.
Specifically, in CropLife, the D.C. Circuit held that:
      The disputed directive constitutes a binding regulation that is directly aimed at and enforceable against petitioners. It provides that "the Agency will not consider or rely on any [third-party] human studies in its regulatory decision making." This clear and unequivocal language, which reflects an obvious change in established agency practice, creates a "binding norm" that is "finally determinative of the issues or rights to which it is addressed."
The Proposal would limit EPA's consideration of scientific studies whose data and models are not sufficiently available to the public for reanalysis. It is therefore directly analogous to the directive at issue in CropLife regarding "best available science" requirements in that it creates a binding norm.
Moreover, in CropLife, the D.C. Circuit determined that the directive was a substantive rule with binding effect as it not only impacted private parties, but also the agency's regulatory actions (as opposed to its internal practices). In particular, the court found that "there [was] little doubt" that the directive prohibiting EPA from considering third-party studies:
      "binds private parties [and] the agency itself with the `force of law,'" and thus constitutes a regulation rather than a policy statement. The directive clearly establishes a substantive rule declaring that third-party human studies are now deemed immaterial in EPA regulatory decision-making under [the Federal Food, Drug and Cosmetic Act] and FIFRA.
Comment [89-2187]: Commenter (11403) states that EPA's 2018 version of this rule cited a hodgepodge of statutory authority, none of it valid. EPA at least corrected the obvious typographical errors from the previous version. However, those edits still do not grant EPA the authority to restrict science from all regulatory decision-making processes within the agency. EPA is attempting to use general grants of authority to create a completely novel approach to assessing science. However, as the Supreme Court has stated, "Congress . . . does not alter the fundamental details of a regulatory scheme in vague terms or ancillary provisions -- it does not, one might say, hide elephants in mouseholes." EPA does not have the authority to alter the fundamental details of all of its regulatory schemes based on such vague provisions of law.
The commenter provides several statutory examples, however only FIFRA material is excerpted here.
      Section 25(a)(1) of the FIFRA authorizes the Administrator "to prescribe regulations to carry out the provisions of this subchapter." This general authorization section of FIFRA differs from that of the other statutes listed because it actually does specify some considerations the Administrator must give when promulgating regulations. For example, it states, "Such regulations shall take into account the difference in concept and usage between various classes of pesticides, including public health pesticides, and differences in environmental risk and the appropriate data for evaluating such risk between agricultural, nonagricultural, and public health pesticides." Unlike the other general grants of rulemaking authority, this provision clearly intends to offer detail and guidance on the nature of the rulemaking authority that the provision grants. As it does not mention restriction of scientific data, there is no indication that EPA has authority to undertake the proposed rule under this section.
      Section 136r(a) of the FIFRA directs the Administrator to conduct research to carry out the purposes of the statute. The section states, "The Administrator shall also take care to ensure that such research does not duplicate research being undertaken by any other Federal agency." The section does not state that EPA can ignore sound science if the raw data is not publicly available.
In short, none of the statutory provisions cited by EPA provide authority for the proposed rule. The proposed rule fails at step one under Chevron v. Natural Resources Defense Council since it is unreasonable to believe that any statutory general grant of authority or any mention of EPA's oversight of other science or research programs would enable EPA to conduct rulemaking in a way that is altogether inconsistent with the rule of law. None of the cited provisions provide the authority sought by EPA, and as such the proposed rule should be withdrawn.
Comment [133-1200]: Commenter (11894) states that this supplemental notice of proposed rulemaking, like the initial proposal, is procedurally deficient because it failed to comply with the rulemaking requirements in FIFRA §25. EPA states that its supplemental proposal pertains to regulatory actions taken under the authority of FIFRA and cites FIFRA as one authority for the overall rulemaking that its supplemental proposal modifies. Thus, both the initial and supplemental proposals are subject to the procedural requirements for rulemaking imposed by FIFRA §25. FIFRA §25 requires the agency to transmit a copy of proposed rules to the USDA and HHS and to the FIFRA SAP prior to publication of the proposal for public comment. The statute gives the SAP, USDA, and HHS 30 days within which to comment, and, if any does so, EPA must publish their comments and the agency's responses in the Federal Register that announces the opportunity for public comment. FIFRA also requires that EPA provide a copy of the proposed rulemaking to the Senate and House Agriculture Committees prior to publication of the proposed rule.
The preamble of EPA's supplemental proposal contains no information concerning the agency's compliance with these essential procedural requirements for rulemaking under FIFRA; we can only conclude that EPA has flagrantly disregarded its legal responsibilities. In these congressionally imposed requirements, Congress has deemed the input of USDA, HHS, and the SAP essential to ensuring that, in issuing regulations affecting the operation of EPA's regulatory programs affecting pesticides, EPA fully considers the impacts of all proposed rules on agriculture (USDA) and human health (HHS), and that the rules are grounded in sound science (SAP). Without such input, there is a high likelihood that the rules will produce unsound outcomes from essential scientific and public policy perspectives.
Comment [151-1345]: Commenter (12405) contends that all pesticides distributed or sold in the United States must first be registered by the EPA and reregistered every 15 years. To be registered, the FIFRA requires the applicant show that its proposed pesticide does not cause unreasonable risk to human health or the environment. The applicant typically provides studies that comply with the EPA's testing guidelines along with its application materials. The EPA reviews the data provided and performs some of its own work, including human health and ecological risk assessments, on a chemical. Additionally, under the FQPA which amended FIFRA, EPA must find a pesticide poses a "reasonable certainty of no harm" before it can be registered for use on food or feed.
For example, the herbicide paraquat is currently undergoing reregistration review. As part of that process, EPA is looking at studies relevant to the chemical's health concerns, including the connection with Parkinson's disease. Over the past few decades, studies consistently show a correlation between exposure to pesticides and Parkinson's disease, but that full breadth of data may not be reviewable by EPA under the current proposal. For example, a meta-review examined 40 studies and concluded, "epidemiologic studies suggest a relatively consistent association between exposure to pesticides and an increased risk of developing [Parkinson's disease], despite differences in study design, case ascertainment and definition, control selection, and pesticide exposure assessment." Many of these studies would be excluded from consideration under the proposed rule.
In addition, relevant studies have design characteristics that make them vulnerable to noncompliance and exclusion. Specifically, two studies of California's Central Valley found years of exposure to a combination of herbicides paraquat and maneb increased the risk of Parkinson's later in life. Another study found that Central Valley residents under age 60 who lived near fields where the pesticides paraquat and maneb were used between 1974 and 1999 had a Parkinson's rate many times higher than other residents in the region.
Parkinson's is rare enough such that in many communities, data that would need to be disclosed, such as behavioral factors (occupation, tobacco, or alcohol use, how long they have lived in the area), will render individuals easily identifiable. To protect patient privacy, scientists may not want to make even de-identified data public. Without these and similarly designed studies, the EPA is likely to miss relevant information in its review.
Comment [165-1508]: Commenter (12453) states that EPA's proposed framework is consistent with existing statutory transparency mandates for pesticide registration and tolerance data. FIFRA requires EPA to make available to the public information concerning the objectives, methodology, results, or significance of any test or experiment performed on or with a registered pesticide, and any information concerning the effects of such pesticide on any organism or the behavior of such pesticide in the environment -- subject to important restrictions necessary to protect the proprietary nature of pesticide studies. FIFRA §§ 10(d)(1), 10(g)(1).
The framework provided under FIFRA and the FFDCA is a successful example of balancing the importance of transparency with the need to ensure protection of data submitters' proprietary rights. FIFRA and the FFDCA ensure the general public is provided secure access to pesticide studies in a manner sufficient for independent validation, while at the same time protecting the proprietary nature of those studies by prohibiting, through the required affirmation, publication or any other type of delivery of the studies to foreign or multinational pesticide producers.
Comment [165-1510]: Commenter (12453) appreciates that there are circumstances in which there may be data that contain potentially relevant information that EPA should be able to consider, but that may not meet all the standards of transparency and quality that the majority of the data meet under FIFRA and the FFDCA, for instance. Two routine examples in the FIFRA and FFDCA context include (a) benefits data (often developed by agricultural experts) that must be submitted to, and evaluated by, EPA under FIFRA § 3 and §§ 6(b-e) and (b) information that must be submitted to and evaluated by EPA under FIFRA § 6(a)(2) adverse effects reporting requirements. In some circumstances, despite EPA's best efforts, it may not have the optimum degree of access. In such circumstances, EPA should have the option to assess both the degree of access to and the attributes of the data and, based on those two factors, to determine how much consideration to give to the particular data. In doing so, it should take into account the full set of data available to the Agency and ensure that the proper relative weight is given to specific items within that data set.
This approach is consistent, for instance, with the way in which EPA has addressed FIFRA's explicit provision authorizing EPA to consider data that appear in the "public literature." FIFRA § 3(c)(1)(F). In implementing this statutory provision, EPA has long taken into account the validity of the data and other considerations about the extent to which the Agency should rely on such data, consistent with Option 2. See, e.g., 40 CFR 152.94. In this statutory example, EPA may give more or less consideration to publications, depending on the degree of access and attributes of the data.
Comment [172-2092]: Commenter (12465) provides that while the supplemental proposal notes that "in the event the procedures outlined in the proposed rulemaking conflict with the statutes that EPA administers, or their implementing regulations, the statutes and regulations will control," therein lies significant ambiguity as to the applicability and scope of regulation designed to underlie the basis of EPA regulatory science decisions. The details on how the Agency would navigate these potential regulatory conflicts should be considered and advanced concurrently with the current proposal to effect maximum stakeholder input. For example, EPA needs to fully understand the potential vulnerability this transparency rule would create for regulatory decisions made under the TSCA and the FIFRA that rely on CBI as well as the extent to which models relied on by both programs comply with the rule as proposed. It is possible that a significant number of EPA decisions under FIFRA and TSCA could be successfully challenged if this rule were finalized as proposed.
Comment [187-1708]: Commenter (12720) states that the FIFRA, EPA cites 7 U.S.C. § 136r(a), which authorizes the Administrator to "undertake research." That section does not allow the restriction of what types of research EPA may consider in rulemakings or otherwise. Nor does it draw any distinction between dose-response data, "data and models underlying pivotal regulatory science and pivotal science," or any other types of data. The absence of any such congressional distinction or restriction makes it all the more obvious that EPA invented and added the distinction and restrictions in the Proposal and Supplemental Proposal as a matter of its own policy preferences, contrary to the Act. This EPA may not do. 
EPA also cites 7 U.S.C. § 136w, which is the general rulemaking authority that allows the Administrator to carry out the provisions of FIFRA. As explained above, that general grant of authority cannot support the rule, especially when the rule conflicts with the Act. Moreover, the citation fails to provide sufficient notice for the public to comment on the Proposal or Supplemental Proposal. FIFRA does not authorize the proposed rule.
1.3.3 Safe Drinking Water Act (SDWA) (NPRM and SNPRM Comments).
Comment: Sections 1.3.3.1 and 1.3.3.2 of this Chapter present comments received on the NPRM (Section 1.3.3.1) and SNPRM (Section 1.3.3.2) related to the EPA's legal authority for the rule under the SDWA. The EPA provides the following general response to these comments:
Response: EPA appreciates your comments. As discussed in Section I.C. of the final rule preamble and in this RTC document in Section 1.2, EPA has decided it is most appropriate to rely on its authority to promulgate housekeeping regulations, rather than any of the substantive environmental statutes cited in the 2018 proposal or the 2020 supplemental proposal as legal authority for the rule. EPA intends to promulgate future science transparency rules under its substantive environmental statutes. Additionally, the final rule states in § 30.3 that "[i]n the event the procedures outlined in this rule conflict with statutes that EPA administers, or their implementing regulations, the statutes and regulations will control."
1.3.3.1 SWDA (NPRM Comments)
Comment: [37-34]: Commenter (0046) writes that although unacknowledged in the proposal including its request for comments, SDWA's standard-setting section, §1412 (42 U.S.C. § 300g - 1), addresses the use of science in decision-making under that authority: 
      §1412(b)(3)(A) Use of science in decision-making.  --  In carrying out this section, and, to the degree that an Agency action is based on science, the Administrator shall use --  
      (i) the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices; and (ii) data collected by accepted methods or best available methods (if the reliability of the method and the nature of the decision justifies use of the data).
So long as the data used is otherwise collected, assessed, and presented "in accordance with sound and objective scientific practices," Congress did not give the Administrator discretion to ignore the "best available, peer-reviewed science and supporting studies" based on any factor relating to the public availability or unavailability of data, as this proposal would seek to compel.
Comment [2772-2712]: Commenter (2787) writes that the proposed rule is inconsistent with EPA's statutory obligations to ground its actions on scientific evidence. The SDWA require that EPA use the "best available science."
Comment [4827-2071]: Commenter (4838) writes that the proposed rule violates several environmental statutes because it hinders EPA's ability to rely on best available science, or the most up-to-date information, as they require. The CAA, CWA, SDWA, TSCA, and EPCRA all require certain decisions or regulatory criteria be based on the most up-to-date science. These criteria are described as "best available science," "latest scientific knowledge," and "best available public health information." The proposed rule would illegally limit EPA's ability to rely on best available science in violation of these statutes.
Comment [4837-3058]: Commenter (4848) notes that administrative procedure requires that EPA consider data submitted by the public in evaluating regulations. Let's be clear. Scientific studies submitted have always been of uneven quality. EPA has processes in place (including use of a scientific advisory board, testimony and written and oral public notice and comment) using both internal and external peer review to evaluate data. Depending on context, some studies are given greater weight than others. Some studies are disregarded entirely.
It is inappropriate, and likely unlawful under the APA and the express language of numerous environmental statutes including the SDWA and the TSCA, for EPA to categorically eliminate certain types of studies, and hence certain types of data, without considering context.
Comment [5865-328]: Commenter (6111) notes that EPA's historic position is consistent with the Statutes. The SDWA
Furthermore, because EPA is required under the Statutes to assess the public health benefits of its regulations, it must take into account all relevant science and cannot arbitrarily exclude certain studies demonstrating those benefits. Under the SDWA, EPA must determine whether a contaminant "may have an adverse effect on the health of persons" before deciding to regulate it.
Many of the fundamental public health studies on which EPA has based key rules and standards under the Statutes are studies for which the raw data were not or could not have been released. Attachment 1 to the commenters' comment letter contains a partial list of studies [note: included under a separate excerpt] that likely contain confidential data; these are all studies on which EPA has relied and cited as the basis for its actions under some of the Statutes. Until now, release of the underlying raw data was not an EPA criterion for determining the "best available" reports, studies, analyses, or models. Indeed, none of the Statutes invoked by EPA as support for the proposed rule limits EPA in this fashion; none of the Statutes requires EPA to make raw data publicly available.
EPA's proposed new approach, which conflicts with the agency's obligations and curtails its authority, is irrational at best and detrimental to public health and safety at worst.
Comment [6112-1552]: Commenter (6137) writes that despite EPA's contrary contentions, the SDWA, 42 U.S.C. § 300f et seq., does not provide any authority for the adoption of a policy that would "preclude" EPA from considering all relevant scientific evidence in carrying out its duty to protect the quality of drinking water in the United States. See 83 Fed. Reg. at 18,769, n.3. In the Proposal, EPA points to two specific provisions of the SDWA as authorizing the rule, neither of which provides the necessary authority.
First, EPA points to 42 U.S.C. § 300j-1, but its reliance on this section is misplaced. Rather than authorizing the Administrator's selective exclusion of science and research, this section simply describes the Agency's responsibility to gather information -- that is, "[to] conduct research, studies, and demonstrations relating to the causes, diagnosis, treatment, control, and prevention of physical and mental diseases and other impairments of man resulting directly or indirectly from contaminants in water, or to the provision of a dependably safe supply of drinking water." 42 U.S.C. § 300j-1(a)(1). This section also directs EPA to study certain serious threats to drinking water, including "polychlorinated biphenyl contamination," "disposal of waste (including residential waste)," "surface spills of contaminants," "virus contamination," "abandoned injection or extraction wells," "intensive application of pesticides and fertilizers in underground water recharge areas," "surface disposal of contaminants in underground water recharge areas," and "the nature, extent, sources of and means of control of contamination by chemicals or other substances suspected of being carcinogenic." Id. § 300j-1(a)(3) -  (9). It therefore provides no legitimate basis for a rule that aims to limit the data that EPA can consider in executing the purposes of the SDWA. 
Second, EPA cites the general grant of rulemaking authority in 42 U.S.C. § 300j-9(a)(1) as authorizing the Proposed Rule. However, this section likewise does not offer authorization. While this section empowers EPA "to prescribe such regulations as are necessary or appropriate to carry out [its] functions under this subchapter," the Proposed Rule is neither necessary nor appropriate to effectuate the SDWA. 
The SDWA requires EPA to protect the public by limiting contaminants in public water systems. Specifically, the Act directs EPA to establish a MCLG for each contaminant "at the level at which no known or anticipated adverse effects on the health of persons occur and which allows an adequate margin of safety." 42 U.S.C. § 300g-1(b)(4)(A). EPA must then set an enforceable Maximum Contaminant Level (MCL) as close to this goal as is feasible. Id. § 300g-1(b)(4).
To accomplish these goals, in 1996, Congress amended the SDWA to ensure that EPA's regulatory decisions were scientifically sound and adequately protective of public health. As amended, the SDWA directs EPA to base its determination about whether to regulate any particular contaminant "on the best available public health information." Id. § 300g1(b)(1)(B)(ii)(II) (emphasis added). In addition, the amended SDWA expressly requires that, "to the degree that an Agency action is based on science, [EPA] shall use . . . the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices[] and . . . data collected by accepted methods or best available methods (if the reliability of the method and the nature of the decision justifies use of the data)." Id. § 300g1(b)(3)(A) (emphasis added). "Best available" means precisely what it says  -  the best of all that is available, not the best of some subset of what is available. The only qualifiers the SDWA places on what is "best available" are that the science be "peer-reviewed," and that the "supporting studies" be "conducted in accordance with sound and objective scientific practices." Id. Disclosure of confidential data underlying the studies plays no role in determining whether the science is the best available and is in no way required by the rule (but rather is expressly rejected by the scientific community, see infra). Any rule proposing to disregard reliable scientific information relevant to the regulation of drinking water contaminants directly conflicts with the SDWA's sound science mandate.
Given that the Proposed Rule is manifestly contrary to the SDWA, which expressly requires use of the best science available, it is not authorized by the general rulemaking authority in § 300j-9(a)(1). Mourning, 411 U.S. at 369; Ragsdale, 535 U.S. at 86. EPA is thus left without an appropriate authorizing provision under the SDWA.
Comment [6112-1589]: Commenter (6137) writes that EPA's proposal to exclude reliable, accessible, and relevant science is antithetical to the requirements of the SDWA and thus is unlawful. The SDWA was established to protect the quality of the drinking water in the United States. To accomplish this, the SDWA requires EPA to limit contaminants in public water systems. As discussed supra, it does this by establishing a MCLG for each contaminant "at the level at which no known or anticipated adverse effects on the health of persons occur and which allows an adequate margin of safety." 42 U.S.C. § 300g-1(b)(4)(A). And in deciding whether to regulate any particular contaminant to protect public health, EPA must rely on "the best available public health information." Id. § 300g-1(b)(1)(B)(ii)(II) (emphasis added). And to the extent EPA relies on science, it must use "the best available, peer-reviewed science and supporting studies" available. Id. § 300g-1(b)(3)(A) (emphasis added). Thus, any decision to categorically ignore or otherwise fail to consider relevant scientific information when regulating drinking water would be unlawful under the SWDA.
Comment [6118-1726]: Commenter (6143) notes that EPA has issued its proposed rule on Strengthening Transparency in Regulatory Science pursuant to its authority under several environmental statutes that that it implements, including the CAA; CWA; SDWA; RCRA; CERCLA; EPCRA; FIFRA; and TSCA. This proposed action is fully consistent with the authorities and objectives laid out in those statutes. EPA must ensure that any final action it takes on this proposal continues to respect the balance reflected in those statutes between sound, transparent science, and legitimate privacy interests.
Comment [6119-1359]: Commenter (6144) writes that the policy, as drafted, is in apparent violation of laws like the TSCA and the SDWA because of the statutory requirement to use the "best available science" and allow the agency to effectively ignore certain information regardless of the quality of that work, that would help implement these science-based laws. According to judicial precedent, "best available science" is defined as "all existing scientific evidence relevant to the decision" and a body of evidence that "cannot ignore existing data." 
Comment [6147-2575]: Commenter (6172) supports the Agency's proposal to promulgate a regulation to strengthen the transparency of science utilized by EPA to support regulatory policies in regulations promulgated under the SDWA.
SDWA regulations and National Primary Drinking Water Regulations (NPDWR) result in a significant economic burden on small and rural communities and often force local communities to pay for federally mandated compliance that the communities do not believe is in their best interest for advancing public health policy. Under SDWA, this often results in a requirement to pay for treatment for nominal reductions of naturally occurring elements and compounds in drinking water supplies. Considering that rural and small communities are controlled and governed by people who are locally elected, all data, scientific studies and discretionary administrative decisions underlying federal actions that supersede local decisions should be made "publicly available in a manner sufficient for independent validation" and with clear explanation of discretionary decisions. NRWA fully supports a new regulation that requires use of peer-reviewed information, consistent data evaluation procedures, data transparency, and reproducible scientific assessments.
Comment [6147-2577]: Commenter (6172) notes that any regulations should improve the status quo and result in the following public disclosure of: 
 A clear principle for implementing the discretion provided to the Administrator for selecting new contaminants for regulations under SDWA Section 1412. The lack of a clear principle or interpretation has allowed EPA to make a determination not to regulate perchlorate on October 10, 2008 and a subsequent affirmative EPA decision on February 11, 2011, that perchlorate meets the Section 1412 criteria for regulation as a contaminant. Such ambiguity in decision-making does not provide for regulatory certainty or persuasive reasoning for the local governments.
Comment [6147-2579]: Commenter (6172) references levels of regulated substances that are unsafe in drinking water as referenced in the SDWA ("unreasonable risk to health," Section 1415 and "protective of public health," Section 1412). For example, the EPA established a standard of 10 parts per billion (ppb) for arsenic in drinking water. The public naturally infers that any level above the standard is a health risk. However, when pressed by Congress to confirm this health risk, EPA failed to do so. In 2002, EPA did not find that arsenic concentrations above their standard necessarily present an "unreasonable risk to health." In their reply to a Congressional inquiry, EPA stated that it is "determining what does not pose an unreasonable risk to health with respect to arsenic, rather than address the much more complex issue of what does constitute an unreasonable risk to health." EPA should identify these public health levels for all NPDWR.
An explanation of why EPA selectively releases only certain peer-review studies to support NPDWR. In 2014, NRWA questioned the Agency about why it was not considering recent published studies that contradict the Agency's current arsenic in drinking water risk assessment findings. The new studies specifically examined the low dose range data, the behavior of the risk function across the full exposure range and developed a new methodology to be able to examine the transition range between high risk and no risk. The first study (Lamm et al., RTP 2013) demonstrates that the finding of a dose-relationship among the low dose villages is a consequence of the use of the external reference group that among the low-dose study villages there is no positive dose-response. The second study (Lamm et al., Tox 2014) demonstrates that the data from the high-dose villages show a high cancer risk, but that risk disappears at 100 ug/L and below. The methodologies of the later papers are more refined and should be appreciated by the Agency. They demonstrate that extrapolation of the risk found at high doses to lower doses is inappropriate and significantly overpredicts the risk in the range of 10-100 ug/L.
Comment [6147-2582]: Commenter (6172) references all data used to craft SDWA policy including risk assessments. For example, a recent study (Mendez et al., J Expo Sci Environ Epidemiol 2017) analyzes the dose-response slope for certain public health endpoints for certain drinking water using data from the NCI and the U.S. Geological Survey. The conclusion of this study could influence the current Integrated Risk Information System (IRIS) Program that is developing an updated assessment of inorganic arsenic. However, the Mendez complete data set includes some data that is available to government agencies but not to the general public. Release of this data would allow for further assessment and demonstrate the repeatability of conclusions and comparisons to similar studies. 
The SDWA mandates that the Agency consider a detailed risk and cost assessment, as well as best available peer-reviewed science, when developing the NPDWR. Crafting a regulation that strengthens the transparency of science utilized by EPA will result in improved federal public policy and SDWA implementation. NRWA wants to ensure that the understanding of the public health risks of particular concentrations of substances (especially naturally occurring elements) in public drinking water are properly understood and applied in making public policy. 
Comment [6152-639]: Commenter (6122) writes that the proposed rule is not based on any valid statutory authority and is arbitrary and capricious.
EPA cites multiple statutory provisions in an attempt to support the proposed rule that it believes authorize this type of rulemaking, "including provisions providing general authority to promulgate regulations necessary to carry out the Agency's functions under these statutes and provisions specifically addressing the Agency's conducting of and reliance on scientific activity to inform those functions . . ." None of the specific statutory provisions provided by EPA provide specific authority for EPA to fundamentally rework the foundational processes it uses to assess and include/reject science. Indeed, if such authority existed, one would have expected EPA to have discovered this authority many decades earlier. Instead, EPA is attempting to use general grants of authority to create a completely novel approach to assessing science. However, as the Supreme Court has stated, "Congress...does not alter the fundamental details of a regulatory scheme in vague terms or ancillary provisions -- it does not, one might say, hide elephants in mouseholes." And if this is true, neither can the EPA alter the fundamental details of all of its regulatory schemes based on such vague provisions of law. We review each of the provisions listed by EPA, none of which discuss transparency, dose response, pivotal regulatory science, benefit-cost calculation, or changing "agency culture," the stated goals of this rulemaking.
Section 1442 of the SDWA generally authorizes the Administrator to "conduct research, studies, and demonstrations relating to the causes, diagnosis, . . . resulting directly or indirectly from contaminants in water, or to the provision of a dependably safe supply of drinking water. . ." This section does not authorize EPA to exclude science based on a perceived lack of transparency or support any other aspect of the proposed rule.
Section 1450(a)(1) of the SDWA states, "The Administrator is authorized to prescribe such regulations as are necessary or appropriate to carry out his functions under this subchapter." This catch-all provision provides the authority to prescribe regulations, but EPA has not provided a single rationale explaining how the proposed rule is "necessary" to the carrying out functions under the SDWA.
Comment [6155-633]: Commenter (6125) notes that the SDWA mandates national drinking water regulations setting required purity levels for water from public water supply systems. 42 U.S.C. §300g-1. Before regulating, the Administrator must conclude that the contaminant at issue "may have" an adverse effect on the health of persons. Id at (b)(1)(A)(i). In regulating, the Administrator must consider "the best available public health information." id at (b)(1)(B)(ii)(II). The section adds that in setting regulations, the Administrator "shall use ...the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices" and in addition, "data collected by accepted methods or best available methods." 42 USC section 300g-1 (3)(A). See City of Waukesha v. EPA, 320 F. 3d at 247 (D.C. Cir 2003) (agency peer review satisfies requirement to use best, peer-reviewed science and supporting studies); See ​City of Portland v. EPA, 507 F 3d 706, 716 (D.C. Cir 2002) (same). 
Finally, to ensure that these requirements are met, and science is fully integrated into the regulatory decision, Congress specifically directed EPA to solicit the views of the SAB before proposing any drinking water regulations. 300-g (1) (e). Here again, EPA's opaque "transparency" rule includes no showing that the proposal is consistent with or authorized by the language of the SDWA.
This proposal appears to be in conflict with the directions Congress gave for the use of and communication about scientific information in enacting P.L. 104-182, the 1996 Amendments to the SDWA. The previous law had mandated expedited schedules for EPA to promulgate regulatory standards for literally dozens of drinking water contaminants, notwithstanding "significant gaps in the scientific information available for many of these contaminants." (Sen. Rpt. 104-169, at 28). "[T]o assure that future standards are based on better science," the 1996 Amendments "add[ed] to the scientific foundation of future standards by imposing [new] requirements on EPA." Id. The Report of the Senate Committee on Environment and Public Works is authoritative on these provisions, as the language adopted in the Committee bill (S.1316) on the use of science was adopted verbatim in P.L. 104-182. The most fundamental of these added requirements  -  unacknowledged in the proposal  -  addresses the use of science in decision-making under SDWA's standard-setting section, §1412 (42 U.S.C. § 300g - 1):
§1412(b)(3)(A) Use of science in decision-making.  --  In carrying out this section, and, to the degree that an Agency action is based on science, the Administrator shall use --  
(i) the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices; and
(ii) data collected by accepted methods or best available methods (if the reliability of the method and the nature of the decision justifies use of the data).
In imposing these requirements, Congress did not give the Administrator the discretion claimed in this proposal to invent novel, EPA-specific definitions of "sound and objective scientific practices," or to ignore the "best available, peer-reviewed science and supporting studies" based on any factor relating to the public availability or unavailability of data. The report further emphasizes that these ​are requirements by specifying that the "Administrator has a duty to seek and rely upon the best available science and information to support.... [m]any of the most important activities including selecting contaminants for regulation, setting standards, designing analytical methods, and structuring waivers, variances and exemptions." (S. Rep. 104-169, at 28) (emphasis added). 
Particularly in light of this history, the requirements in this provision must be considered definitive as to the agency's permitted approach to the use of science in covered decision-making. Thus, the declaration in the preamble to the proposal that "[w]here available and appropriate, EPA will use peer-reviewed information, standardized test methods, consistent data evaluation procedures, and good laboratory practices to ensure transparent, understandable, and reproducible scientific assessments" but is "precluded" from using such information where it is not feasible to make it available (83 FR at 18770 and n.3 ), cannot be reconciled with the scope of the permissible use of science under SDWA §1412(b)(3)(A) (i).
First, under SDWA, the peer-reviewed science used must qualitatively be the "best available"; no discretion is given to the Administrator to further limit the science used on the ambiguous grounds of "appropriate[ness]," much less the non-statutory basis of "transparency" put forward in the proposal. Most important, the core objective of the proposal  -  "to ensure that the regulatory science underlying its actions is publicly available in a manner sufficient for independent validation" (proposed § 30.1 "What is the purpose of this subpart?") is one that the Administrator does not have authority to pursue under this SDWA provision. 
Tellingly, the authority under SDWA cited for the proposal, SDWA §1442, simply authorizes research, technical assistance, information and research facilities, and training. That section cannot be read to authorize in any way the changes in the use of "regulatory science underlying [agency] actions" in this proposal, especially in the face of the terms and stated intentions of §1412(b)(3)(A)(i). (The other SDWA provision cited in the proposal as "authority", §1450(a)(1), is simply a general rulemaking authority and cannot be used to bootstrap the lack of direct authority in SDWA for this proposal). 
The principal cases interpreting this provision have defined and maintained the strict requirements of this authority. EPA observes the "best available science" requirement where it uses newer, peer-reviewed data rather than older data developed by an EPA scientist for the regulatory determination (re filtration avoidance, City of Portland v. EPA, 507 F.3d 706, 716 (D.C. Cir. 2007)), and where the agency explained and provided "substantial scientific support" for the model and data it used (City of Waukesha v. EPA, 320 F.3d 228, 250-51, 256-57 (D.C. Cir. 2003), but not where it rejected the evidence that was the "best available" at the time of the rulemaking (Chlorine Chemistry Council v. EPA, 206 F.3d 1286, 1290-91 (D.C. Cir. 2000)). City of Waukesha v. EPA furthermore indicates that agency peer review satisfies the requirement to use best, peer-reviewed science and supporting studies, as noted above. 320 F. 3d at 247. 
Comment [6155-2631]: Commenter (6125) writes that it is clear from even a brief look that these provisions require EPA to consider ​all ​the best available science, without authorizing, or even mentioning any of the per se restrictions based on "transparency" that EPA has proposed to use as a universal overriding rule. For these reasons, EPA must justify its proposal as a proper exercise of its authority under ​each ​of the statutory provisions whose operation it would affect - provisions that the agency does not even cite. It is the job of the proposer, not the commenters, to survey these provisions and provide the needed justification.
The SDWA 1412(b)(3)(A) addresses EPA's "use of science in decision-making" by providing that risk assessments "shall" use "the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices."
This provision uses mandatory language directing EPA to use reliable and relevant scientific information -- and further specify how to do so. They all direct that decisions regarding protection of public health be based on the full range of relevant scientific information, in some cases specifying how to identify what information to use. None of them give EPA discretion to ignore such information for any of the reasons set forth in the proposal.
Comment [6163-1105]: Commenter (6133) notes that EPA cites 42 U.S.C. § 300j - 1 of the SDWA as authority for the rule. Subsection (a) of that section allows EPA to conduct some types of research on drinking water contamination and requires it to conduct other studies. But it says nothing about which types of studies EPA may consider in rulemakings and does not distinguish between dose-response studies and other types of studies. The absence of any such congressional distinction or restriction makes it all the more obvious that EPA invented and added the distinction and restrictions in the Proposal as a matter of its own policy preferences, contrary to the Act. This EPA may not do. The remainder of the subsections have nothing to do with data or research. At any rate, EPA does not state specifically which of the subsections in 42 U.S.C. § 300j - 1 it believes authorizes this proposed rule. Thus, the citation fails to provide sufficient notice for the public to comment on the proposed rule. 
EPA also cites 42 U.S.C. § 300j - 9(a)(1), but that says only that the "Administrator is authorized to prescribe such regulations as are necessary or appropriate to carry out his functions under this subchapter." As explained above, that general grant of authority cannot support the rule, especially when the rule conflicts with the Act. The SDWA does not authorize the proposed rule.
Comment [6163-1136]: Commenter (6133) notes that the SDWA protects the nation's public drinking water supplies. The Act generally applies to "each public water system in each State," 42 U.S.C. § 300g, and requires EPA to set standards for drinking water contaminants that may have an adverse effect on human health and are known or anticipated to occur in such systems, id. § 300g-1(b)(1)(A).
For a given contaminant, the SDWA requires that EPA first establish a MCLG, which is "the level at which no known or anticipated adverse effects on the health of persons occur and which allows an adequate margin of safety." Id. § 300g-1(b)(4)(A). EPA must then set a MCL "as close to the [MCLG] as is feasible." Id. § 300g-1(b)(4)(B).
EPA also must, every five years, "publish a list of contaminants" that "are not subject to any proposed or promulgated national primary drinking water regulation, which are known or anticipated to occur in public water systems, and which may require regulation . . . ." Id. § 300g1(b)(1)(B)(i). The SDWA requires EPA to prioritize that list based on vulnerable subpopulations that are at risk and other factors. Id. § 300g-1(b)(1)(C). EPA must then decide whether to regulate at least five contaminants on the list based on the "best available public health information." Id. § 300g-1(b)(1)(B)(ii). 
In making these determinations, the SDWA requires EPA to use "the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices," and "data collected by accepted methods or best available methods (if the reliability of the method and the nature of the decision justifies use of the data)." Id. § 300g1(b)(3)(A); see also id. § 300g-1(b)(12), (13) (similar); id. § 300j-19 (referring to best available science standard for risk assessment of algal toxins). 
The Proposal would conflict with the SDWA by prohibiting EPA from using the "best available" science and "data collected by acceptable or best available methods" solely because that data could not be made public. Indeed, courts interpreting these requirements have already rejected this proposed limitation on dose-response studies, making clear that they can indeed be the "best available" science regardless of whether the underlying data are publicly available. In City of Waukesha v. EPA, the court approved EPA's use of "studies of Hiroshima and Nagasaki atomic bomb survivors" in setting limits for radium and uranium in drinking water. 320 F.3d 228, 248, 252 (D.C. Cir. 2003). But of course, these and similar studies would likely be excluded under the Proposal because the underlying data are not available. The court also upheld the agency's use of the LNT model used by EPA for both radium and uranium, id. at 249 - 50, 252, which is precisely the model that EPA now implies -- without citing any evidence -- is not scientifically justified.
Additionally, in carrying out its obligations to establish drinking water standards, the Act directs the agency to discuss "peer-reviewed studies known to the Administrator that support, are directly relevant to, or fail to support any estimate of public health effects and the methodology used to reconcile inconsistencies in the scientific data." 42 U.S.C. § 300g-1(b)(3)(b)(v). Moreover, the agency must identify the "[q]uantifiable and non-quantifiable benefits for which there is a factual basis in the rulemaking record" in establishing a drinking water standard. Thus, under the express provisions of the SDWA, the agency cannot simply ignore peer-reviewed studies or other information in the record that the Proposed Rule would disallow from consideration, simply because the underlying data may be unavailable. Id. § 300g-1(b)(3)(c)(i). 
If Congress had intended for the data targeted by the Proposal to be excluded, it could have said so. Instead, Congress directed EPA to use "the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices," and "data collected by accepted methods or best available methods." 42 U.S.C. § 300g1(b)(3)(A). EPA cannot ignore these commands to achieve its political goal of rolling back public health protections.
Comment [6333-2425]: Commenter (6355) strongly agrees that "the best available science must serve as the foundation of EPA's regulatory actions". While the proposed rule does not define what is meant by `best available science', and scientists and policy-makers may have different ideas on how the concept is defined and interpreted, the scientific community agrees that the `best available science' is the product of a process that includes clearly stated objectives and rationale, appropriate study design and data collection methods, logical analysis and interpretation of data, and rigorous documentation and explanation of findings, all of which are evaluated by independent peer-review. 
The `best available science' provides an indispensable foundation for policy and regulatory decisions; yet the proposed rule restricts the science that can be used for regulatory decision-making to those data that conform to a list of constraints (i.e. , independence, objectivity, transparency, clarity, and reproducibility). The `best available science', as described above, is not bound by these criteria. Thus, the proposed rule would exclude relevant scientific evidence from the policy-making process. Ignoring relevant data would be in violation of those laws that mandate the use of `best available science' (e.g., SDWA). 
Comment [6340-1455]: Commenter (6362) writes that the provisions cited by EPA under the CAA, the CWA, the SDWA, the CERCLA, and the EPCRA in support of its authority to develop and implement its proposed rule all provide broad regulatory authority to promulgate regulations "as are necessary to carry out [the Administrator's] functions" under the statute. 
Comment [6353-2321]: Commenter (6375) notes that EPA solicits comment on whether additional or alternative sources of authority are appropriate bases for this proposed regulation. (pg. 18,771). EPA's statement concerning its legal authority could be enhanced by reference to statutory provisions that require it to rely on "accurate," "useful," or "best" data. For example, section 108(a)(2) requires that the Administrator issue air quality criteria that "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health and welfare . . .." 42 U.S.C. § 7408(a)(2). The SDWA requires the Administrator to assess risk using "the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices." 42 U.S.C. § 300g1(b)(3)(A). Increasing the transparency of regulatory science and ensuring that it is reproducible helps in evaluating its accuracy and usefulness.
Comment [6425-2355]: Commenter (6447) writes that the proposal is not authorized by statute and is contrary to Law.
EPA purports to rely on general statutory authority for the proposal, but no statutes authorize EPA to ignore science. On the contrary, as multiple U.S. Senators have noted, EPA's proposal would violate several statutes, including the TSCA, the SDWA, and the APA, which require the Agency to use the best available science, and all information presented to the agency during a rulemaking. The current proposal would do the opposite, requiring EPA to ignore some of the best available science, and to ignore certain information submitted during rulemaking. In particular, the proposal would make it difficult or impossible to use most epidemiological studies, where the source data are almost always comprised of sensitive personal information. Epidemiological studies are fundamental to EPA policymaking, and a critical source of information about the effects of various risk factors on human health. The Agency does not the option of suppressing this information, and courts have held that EPA "cannot ignore existing data."
In short, the proposed action would be illegal, and would not survive a legal challenge.
Comment [6884-3033]: Commenter (6905) notes that like EPA's mission statement, several of the federal statutes that provide authority for the agency's work call for use of the best science without regard for data transparency. For example, the SDWA calls on the Administrator to use "the best available, peer reviewed science and supporting studies conducted in accordance with sound and objective scientific practices." SDWA specifically calls on the Administrator to make information on public health effects available including peer reviewed studies. But it qualifies the requirement with the phrase "to the extent practicable" and does not require that all the data underlying the studies be publicly available. It appears that the proposed rule could reject some of the best available science and may result in a finding that the rule contradicts the federal statutes.
Comment [6894-3605]: Commenter (6915) writes that the SDWA requires that findings which support a determination to regulate a contaminant "be based on the best available public health information," and that, in developing the NPDWR, "to the degree that an Agency action is based on science, the Administrator shall use the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices." §§ 1412(b)(1)(B)(ii)(II), 1412(b)(3)(A)(i), 42 U.S.C. §§ 300g-1(b)(1)(B)(ii)(II), 300g1(b)(3)(A)(i) (emphases added).
Comment [6901-4442]: Commenter (6922) notes that in the SDWA: "Use of science in decision-making. In carrying out this section, and, to the degree that an Agency action is based on science, the Administrator shall use:
      1) The best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices; and
2) Data collected by accepted methods or best available methods (if the reliability of the method and the nature of the decision justifies use of the data)."
Comment: [7876-4129]: Commenter (8279) objects to the proposal to limit the use of scientific evidence in rulemaking proceedings. 
The proposed regulation will undercut the ability of EPA (including future EPA administrators) to protect the public pursuant to the SDWA and TSCA. The breadth of the proposed regulation is shown in part by reviewing the chemicals covered by the two acts. See, e.g., The non-confidential list of 67951 TSCA chemicals, https://www.epa.gov/tsca-inventory/how-access-tsca-inventory, the list of hazardous waste, https://www.epa.gov/hw/defining-hazardous-waste-listed-characteristic-and-mixed-radiological-wastes, SDWA Regulations, https://www.epa.gov/ground-water-and-drinking-water/national-primary-drinking-water-regulations, PDF copy of list, https://www.epa.gov/sites/production/files/2016-06/documents/npwdr_complete_table.pdf
Comment [8750-1433]: Commenter (9227) notes that EPA's statutory authorities generally require the agency to consider all available data when undertaking significant rulemakings.
As just noted, EPA's statutory authorities mandate a variety of requirements for what scientific information EPA must consider in rulemaking. These statutes are discussed in detail, at section I. B.3. of the commenter's comment letter. To take one example that appears in numerous statutes, including TSCA, CAA, SDWA, and the ESA, Congress has often required agencies to act on the "best available science." For an agency to comply with this obligation, the agency must at least consider all available scientific information. "Best" means "of the most excellent, effective, or desirable type or quality." "Available" means "able to be used or obtained." And "science" means "the intellectual and practical activity encompassing the systematic study of the structure and behavior of the physical and natural world through observation and experiment." Assessing which science is "best" requires consideration of the overall quality of the science, and the public availability of underlying data is, at best, one of many aspects that should inform that assessment of overall quality.
An agency "cannot ignore available. . . information." Numerous courts have indicated that a plaintiff or petitioner can establish a violation of the "best available science" requirement by "point[ing] to any scientific evidence that the agency failed to consider." "The best available data requirement. . . prohibits [an agency] from disregarding available scientific evidence that is in some way better than the evidence [it] relies on.'" "An agency does. . . have an obligation to deal with newly acquired evidence in some reasonable fashion." EPA's proposal will result in EPA precluding itself from considering certain studies that are "available," thus violating the requirement that EPA rely on the best available science.
In addition, the requirement that agencies use "best available" science or information often means that the agency must act even if the available science or information is imperfect.
"Even if the available scientific and commercial data were quite inconclusive, [the agency] may -- indeed must -- still rely on it" when the agency has a duty to act. "[W]here the information is not readily available, we cannot insist on perfection." Just as the Courts have recognized that they cannot expect perfection, agencies cannot choose to ignore certain studies or sources of information based solely on whether the data is publicly available -- especially where the validity of those studies has been established using techniques that do not rely on public availability of underlying data.
EPA cannot reasonably elevate the interest in public availability of all underlying information above all other factors in assessing the "best available science." Textually, EPA's approach is unlawful.
Comment [8750-1437]: Commenter (9227) writes that the proposed rule violates EPA's statutory authorities.
Not only is there no authority for EPA's pan-statutory Proposal, but the Proposal would also violate explicit statutory commands. Though EPA admits that "[t]he best available science must serve as the foundation of EPA's regulatory actions," proposed section 30.5 would prohibit EPA from considering high quality and critically important scientific studies -- precisely that "best available science" -- when undertaking regulatory actions. Specifically, section 30.5 would prevent EPA from considering any scientific study for which the underlying "dose response data and models" are not "publicly available in a manner sufficient for independent validation." This is reflected "the best available public health information" in a SDWA rulemaking, 42 U.S.C. § 300g-1(b)(1)(B)(ii)(II). Accordingly, this proposed prohibition would contravene an array of statutes governing EPA's consideration of science when promulgating rules, such as requirements to consider the "best available science" when setting environmental protection standards. See, e.g., SDWA, 42 U.S.C. § 300g-1(b)(3)(A) (EPA must use "[t]he best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices" and "[d]ata collected by accepted methods or best available methods"). And, by excluding science that meets these statutory criteria from supporting regulations to protect public health and welfare, the Proposal would frustrate Congress's policy in these statutes and frustrate EPA from achieving its fundamental mission.
Comment [8750-1465]: Commenter (9227) writes that EPA's Proposal contravenes the SDWA.
The SDWA requires EPA to issue national drinking water regulations setting required purity levels for water from public water supply systems. Before regulating, the Administrator must conclude that the contaminant at issue "may have" an adverse effect on the health of persons. In regulating, the Administrator must consider "the best available public health information." The section adds that in setting regulations, the Administrator "shall use ...the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices" and in addition "data collected by accepted methods or best available methods." When Congress promulgated these statutory requirements in 1996, the Senate Committee on Environment and Public Works explained that the "Administrator has a duty to seek and rely upon the best available science and information to support.... [m]any of the most important activities including selecting contaminants for regulation, setting standards, designing analytical methods, and structuring waivers, variances and exemptions."
By restricting EPA to considering only those scientific studies for which underlying data, models, and other information is publicly available, EPA's proposal prevents EPA from complying with the SDWA directive that it consider the "best available" public health information and science when setting SDWA standards. Specifically, as explained above, the public will not necessarily have access to the underlying information used to produce the "best available, peer-reviewed science and supporting studies." Nowhere does the SDWA authorize EPA to ignore such studies based on the public unavailability of underlying information. Thus, regardless of the merits of the core objective of EPA's proposal -- "to ensure that the regulatory science underlying its actions is publicly available in a manner sufficient for independent validation" (proposed § 30.1 "What is the purpose of this subpart?"), EPA's attempt to elevate this objective above the agency's statutory obligation to consider the "best available" science when promulgating SDWA standards is unlawful.
Comment [8750-1817]: Commenter (9227) writes that EPA's proposal fails to meet the procedural requirements of the SDWA, 42 U.S.C. § 300f Et Seq.
EPA cites the SDWA as an authority for the Proposal but has failed to comply with the procedural requirements of the statute. The SDWA provides authority to promulgate regulations at 42 U.S.C. 300g-1(d). Though EPA does not cite this particular section, it is the only provision of the SDWA that provides EPA with rulemaking authority. The SDWA requires the Administrator to consult with the Secretary of HHS and the National Drinking Water Advisory Council (NDWAC) in proposing and promulgating regulations under this section. EPA has not met these requirements here, and as such cannot claim to be using SDWA authority to promulgate this rule.
Comment [8750-1965]: Commenter (9227) writes that EPA arbitrarily failed to consider the far-reaching implications this Proposal could have on inter-agency coordination and consultation given that other agencies normally rely on research potentially excluded by the Proposal.
Consultation Provisions in the SDWA
300g-1 (b)(1)(D)
Standards: Listing of Contaminants for Consideration, Urgent Threats to Public Health
The Administrator may promulgate an interim NPDWR for a contaminant without making a determination for the contaminant under paragraph (4)(C), or completing the analysis under paragraph (3)(C), to address an urgent threat to public health as determined by the Administrator after consultation with and written response to any comments provided by the Secretary of HHS, acting through the director of the CDC or the director of the NIH.
***
300g-1(d)
Regulations:
Regulations; public hearings; administrative consultations. Regulations under this section shall be prescribed in accordance with section 553 of title 5, United States Code (relating to rulemaking), except that the Administrator shall provide opportunity for public hearing prior to promulgation of such regulations. In proposing and promulgating regulations under this section, the Administrator shall consult with the Secretary and the NDWAC.
***
300j-12(i)(2)
Funds: Indian Tribes: Use of Funds
(2) Use of funds. Funds reserved pursuant to paragraph (1) shall be used to address the most significant threats to public health associated with public water systems that serve Indian Tribes, as determined by the Administrator in consultation with the Director of the Indian Health Service and Indian Tribes.
***
300j-13(a)(5)
Source Water Quality Assessment
Demonstration project. The Administrator shall, as soon as practicable, conduct a demonstration project, in consultation with other Federal agencies, to demonstrate the most effective and protective means of assessing and protecting source waters serving large metropolitan areas and located on Federal lands.
***
300j-5(b)
NDWAC 
(b) Functions. The Council shall advise, consult with, and make recommendations to, the Administrator on matters relating to activities, functions, and policies of the Agency under this title [42 USCS §§ 300f et seq.].
***
300j-3d
Water Supply Cost Savings
(a) Drinking water technology clearinghouse. The Administrator, in consultation with the Secretary of Agriculture, shall -- 
(1) develop a technology clearinghouse for information on the cost-effectiveness of innovative and alternative drinking water delivery systems, including wells and well systems; and
(2) disseminate such information to the public and to communities and not-for-profit organizations seeking Federal funding for drinking water delivery systems serving 500 or fewer persons.
***
300i-3(a)
Contaminant Prevention, Detection and Response
In general. The Administrator, in consultation with the CDC, after consultation with appropriate departments and agencies of the Federal Government and with State and local governments, shall review (or enter into contracts or cooperative agreements to provide for a review of) current and future methods to prevent, detect and respond to the intentional introduction of chemical, biological or radiological contaminants into community water systems and source water for community water systems, including each of the following:
***
300j19(b)(2)(A)
Algal Toxin Risk Assessment and Management
(b) Information coordination. In carrying out this section the Administrator shall--
(2) as appropriate, consult with--
      (A) other Federal agencies that-- 
      (i) examine or analyze cyanobacteria or algal toxins; or
      (ii) address public health concerns related to harmful algal blooms.
Comment [9066-2646]: Commenter (6188) writes that the proposed rule contradicts the requirements of federal environmental law, including cited authority for the rulemaking.
The Proposed Rule's requirements are at odds with statutory mandates contained in the very same environmental laws it points to as providing a basis for the rule. They also directly controvert authority it cites as support for the rulemaking.
The requirements of the Proposed Rule conflict with EPA's statutory obligations under federal environmental laws. CAA § 109 mandates that EPA set air quality standards based on "air quality criteria," which must "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare." 42 U.S.C. §§ 7409, 7408(a)(2). The TSCA requires EPA to use the "best available science" when evaluating the testing and regulation of chemicals. 15 U.S.C. § 2625(h). Decisions are to be made using the "weight of the scientific evidence," and EPA is required to consider all information related to a chemical substance, including hazard and exposure information, "that is reasonably available to the Administrator." 15 U.S.C. §§ 2625(i), (k). Similarly, the SDWA mandates use of the "best available public health information" when EPA determines whether to regulate a contaminant, and reliance on the "best available, peer-reviewed science and supporting studies" when making regulatory decisions. 42 U.S.C. §§ 300g-1(b)(1)(B)(ii), 300g-1(b)(3).
Exclusion of relevant studies from consideration based not on the quality of the science but on the public availability of underlying data directly contradicts these mandates. EPA itself has defined "best available science" without reference to the public availability of data:
Use of best available science involves the use of supporting studies conducted in accordance with sound and objective scientific practices, including, when available, peer reviewed science and supporting studies and data collected by accepted methods or best available methods (if the reliability of the method and the nature of the decision justifies use of the data).
Simply put, the "best available science" is science that follows standards accepted by the scientific community, not science that EPA arbitrarily selects based upon the public availability of data -- a criterion irrelevant to scientists' standards for determining whether research represents the best science.
Specifically, scientists have observed that the best quality science oftentimes relies upon data that is not publicly available due to significant privacy considerations. It is for that very reason that courts have recognized EPA's need to rely upon studies based on publicly undisclosed underlying data when considering the best science. American Trucking Ass'ns, 283 F.3d at 372 (explaining that curtailing EPA's ability to rely on published studies would exclude "plainly relevant scientific information" from regulatory decision-making processes); see also Coalition of Battery Recyclers Ass'n, 604 F.3d at 623 (finding that EPA is entitled to rely on published study results as "raw data is often unavailable due to proprietary interests of a study's scientific investigators or confidentiality agreements with study participants"). Forcing EPA to ignore high quality science controverts EPA's statutory obligations and will impair EPA's ability to protect public health and the environment.
Comment [9078-3549]: Commenter (6194) writes that the proposal is inconsistent with EPA's statutory obligations and conflicts with other applicable federal requirements.
The Proposal would prevent EPA from relying on the best available information in discharging its duties. EPA is charged with developing regulations that provide societal benefits by reducing harm to human health and the environment from the presence of chemicals in air, soil, drinking water, food, and consumer products. The Proposal implicates research that is central to making such determinations; creating an obstacle to the consideration of such science is contrary to both primary statutory directives and requirements to conduct cost benefit analyses. Likewise, EPA fails to address how it could implement a regulation that is inconsistent with federal requirements and standards governing the use of science across agencies. 
A. Statutory Requirements to Develop Health Based Standards and Conduct Cost-Benefit Analyses EPA acknowledges that it must use the "best available science" in all of its regulatory actions. 
This directive is embodied in multiple statutes implemented by EPA and is reflected in other environmental statutes, further illustrating Congress' conceptualization of "best" available science. Broadly speaking, there are three basic types of statutory requirements that the EPA and other agencies consider scientific information. First, there are requirements that the agency consider the "best available science." Second, there are requirements that the agency consider particular factors, including scientific factors. Third, there are requirements that the agency balance economic costs or technological feasibility against public health, environmental, or other benefits that must be scientifically assessed. The precise terminology regarding the required use of science may, as illustrated below, vary across statutes, but what these provisions have in common are requirements for EPA and other agencies to use scientific information that is "best," not simply adequate, and "available." 
Table 1: Examples of Statutory Directives regarding Use of Science by EPA and other Agencies
 SDWA [42 U.S.C § 300g1(b)(3)(A)]: EPA must use "[t]he best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices," and "[d]ata collected by accepted methods or best available methods (if the reliability of the method and the nature of the excision justifies use of the data)." 
 [42 U.S.C. § 300g1(b)(1)(b)(ii)]: EPA must use "best available public health information" in determining whether to regulate contaminants.
This statutory provisions link the concept of the "best available science" to the availability of underlying raw data, nor can such a limitation be read into the statutes. Where Congress wanted to direct EPA's decision regarding what science to consider or how -- it did so. When Congress wanted to link "best available science" to data being publicly available -- it did so. Thus, if Congress had wanted to put a blanket limitation on EPA's consideration of science for which underlying data is not publicly available, it clearly knew how to do so -- and it chose not to.
As EPA has previously explained, the agency would be unable to fulfill statutory mandates if it could not consider studies that follow federal laws and ethical standards regarding the privacy of patient and individual data: 
If EPA and other governmental agencies could not rely on published studies without conducting an independent analysis of the enormous volume of raw data underlying them, then much plainly relevant scientific information would become unavailable to EPA for use in setting standards to protect public health and the environment. . . . [S]uch data are often the property of scientific investigators and are often not readily available because of the proprietary interests of the investigators or because of arrangements made to maintain confidentiality regarding personal health status and lifestyle information of individuals included in such data. Without provisions of confidentiality, the possibility of conducting such studies could be severely compromised.
The Proposal would also cut EPA off from information needed to complete the cost-benefit analysis of regulations required under certain statutes and government-wide requirements.
Not only would the Proposal impede EPA's fulfillment of its statutory duties, but it would also be contrary to judicial precedent, which holds that "best available science" includes "all existing scientific evidence relevant to the [agency] decision [in question]." Ecology Ctr., Inc. v. U.S. Forest Serv., 451 F.3d 1183, 1194 n.4 (10th Cir. 2006) (quoting Heartwood, Inc. v. U.S. Forest Serv., 380 F.3d 428, 436 (8th Cir. 2004)). While agencies need not produce new evidence, they must seek out information and "`cannot ignore existing data.'" Id. (emphasis added). Even if an agency is anticipating the arrival of better evidence, it must act on the basis of the evidence it currently has. Many of the fundamental public health studies on which EPA has based key rules and standards under the statutes are studies for which the raw data was not or could not have been released. Nonetheless these studies have been examined and validated, as will future studies. Arbitrarily cutting off consideration of a category of science is inconsistent with EPA's statutory obligations.
Comment [9217-3562]: Commenter (9225) writes that the proposed rule is not authorized by any authority delegated to EPA by Congress and is contrary to a number of statutes under EPA's authority. This includes, but is not limited to, the CAA; CWA; TSCA; LCSA; SDWA; FIFRA; and more. 
Comment [7869-3372]: Commenter (8272) writes that the proposed rule is not authorized by any authority delegated to EPA by Congress and is contrary to a number of statutes under EPA's authority. This includes, but is not limited to, the CAA; CWA; TSCA; LCSA; SDWA; and the FIFRA. The rule violates numerous public health and environmental provisions contained in these laws, as well as requirements to use the best available science or to consider all available information.
Comment [6168-4099]: Commenter (6178) notes that the NPRM proposes to establish a new EPA regulation that will `ensure the data and models underlying the science pivotal to what the Agency deems `significant regulatory actions' is publicly available in a manner sufficient for validation and analysis', hereinafter referred to as the `Transparency Rule' or `Rule'. Specifically, they wish to raise the following concerns with the Transparency Rule proposed by the NPRM: 
      1. The Agency cites specific sections of statutes it administers as the basis of the Agency's authority to adopt the Transparency Rule; specifically SDWA sections 1442. The Agency rightly cites acts of Congress in an attempt to support the NPRM as such authority is necessary for the Agency to adopt regulations pursuant to the above listed Acts. 
      2. However, Congress, in each of the Acts' sections cited, never instructs the Agency to consider transparency to the public of supporting data and/or models of the scientific research when deciding to rely on such research as support for regulations passed pursuant to any of the Acts. Consequently, the Transparency Rule requires the EPA to rely on factors which Congress has not intended it to consider when promulgating regulations pursuant to the Acts, rendering the Transparency Rule arbitrary and capricious under the APA. 
      3. Further, not only does the Transparency Rule lack sufficient Congressional intent to avoid being deemed arbitrary and capricious, implementing the Rule would actually contradict Congressional instruction to conduct scientific research with a focus on `promotion' and `coordination' among itself and other institutions that produce scientific research material to the objectives of the Acts Congress requires the EPA to enforce. This provides additional cause for finding the Transparency Rule arbitrary and capricious under the APA.
Comment [6141-2554]: Commenter (6166) writes that the proposed rule is not authorized by any authority delegated to EPA by Congress and is contrary to a number of statutes under EPA's authority. This includes, but is not limited to, the CAA; CWA; TSCA; LCSA; SDWA; FIFRA; and more. Substantively, the rule violates numerous public health and environmental provisions contained in these laws, as well as requirements to use the best available science or to consider all available information.
Comment [8750-1661]: Commenter (9227) writes that EPA arbitrarily failed to consider the implications of this proposal on interagency coordination.
Additionally, EPA arbitrarily failed to consider the far-reaching implications this Proposal could have on inter-agency coordination and consultation given that other agencies normally rely on research potentially excluded by the Proposal. In the numerous environmental statutes that EPA cites, there are dozens of provisions that require EPA to coordinate or consult with other Federal entities -- especially when implementing research programs and issuing information or guidelines. The Proposal would almost certainly frustrate and impair this coordination and consultation, either by forcing EPA to ignore the science provided by other agencies or by severely restricting the science that EPA itself would be able to share with other agencies in these statutorily required processes. The Proposal arbitrarily ignores these potential impacts.
In addition to the many examples of statutorily required consultation that are identified in Appendix B of the commenter's comment letter, other federal agencies routinely incorporate and rely upon EPA science assessments in their own efforts to carry out their mandates to protect human health and safety. As with statutorily required consultations, the Proposal utterly fails to acknowledge or consider what impacts restricting EPA's own use of dose-response studies would have on the work of these other agencies. Indeed, there is no evidence that these other agencies were even permitted to comment on the Proposal as part of the usual process of interagency review.
Some selected examples of other federal agency programs that rely on EPA science include:
 FDA also regulates contaminants in bottled water under the FFDCA Section 410 of the Act requires that FDA regulations for bottled water be issued in coordination with the effective date of NPDWR issued under the SDWA and be no less protective of public health than those standards. If the Proposal impedes EPA's work to establish drinking water standards, this may affect FDA's own ability to justify protective bottled water standards.
Comment [6428-1351]: Commenter (6450) writes that the proposed rule is not merely a disastrous policy choice, given its threats to public health and the integrity of EPA decision-making, but it is plainly unlawful. EPA clearly lacks the authority to issue the proposal. EPA cites to various provisions in federal law to justify its authority but, in virtually every case, the agency refers to sections that authorize or mandate EPA to undertake research, not to impose arbitrary limitations on the scientific data that informs public health decisions. EPA also cites to statutory provisions authorizing it to promulgate rules "necessary" to achieving the goals of relevant statutes but restricting the use of sound science is neither necessary nor consistent with the statutory goals of protecting clean air, clean water, and a safe environment. Indeed, the proposed rule directly contradicts the specific mandate of numerous federal laws, such as the SDWA and the TSCA, which require EPA to use the "best available" science or all "reasonably available" science and information in its decision-making.
Comment [6163-1276]: Commenter (6133) writes that the proposal's peer review provision lacks any statutory basis, is vague and contrary to existing requirements for peer review.
In addition to addressing how and whether the agency will consider science, the Proposal also contains a seemingly unrelated provision regarding agency peer review. The Proposal, in § 30.7, reads: 
      What role does independent peer review in this section?
      EPA shall conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein. 
 
Because transparency in regulatory science includes addressing issues associated with assumptions used in models, EPA shall ask peer reviewers to articulate the strengths and weaknesses of EPA's justification for the assumptions applied and the implications of those assumptions for the results. 
 
83 Fed. Reg. at 18,774. 
There is no statutory authority for EPA to "conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein." The federal statutes that EPA lists as putative authority for the Proposal provide no authority for proposed § 30.7. The SDWA sections 1442, 1450(a)(1), 42 U.S.C. 300j - 1, 300j -  9(a)(1). The claimed authorities that EPA lists do not mention peer review or even allude to the concept. Id. Neither the Proposal nor accompanying docket materials identify any provision of any federal statute that authorizes EPA to promulgate proposed § 30.7. Our own research revealed no provision of any federal statute that authorizes EPA to promulgate proposed § 30.7. 
When Congress writes federal statutes, Congress knows how to create legal authority for peer review, who shall conduct that peer review, what role, if any, that EPA or other parties will play, and how that peer review may be conducted. None of the provision in the law authorize EPA's Proposal in § 30.7. Instead, the statutes require that EPA use peer-reviewed science, regardless of whether it would meet EPA's definition of the term in § 30.7. And in cases where the law requires EPA to conduct the review, the statutes often spell out specifically how that should happen. 
Statute Provisions on Peer Review 
SDWA
§ 300g-1. National drinking water regulations 
 
(b) Standards 
 
(3) Risk assessment, management, and communication
(A) Use of science in decision-making in carrying out this section, and, to the degree that an Agency action is based on science, the Administrator shall use--
(i) the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices; and(ii) data collected by accepted methods or best available methods (if the reliability of the method and the nature of the decision justifies use of the data). 
 
(B) Public information
In carrying out this section, the Administrator shall ensure that the presentation of information on public health effects is comprehensive, informative, and understandable. The Administrator shall, in a document made available to the public in support of a regulation promulgated under this section, specify, to the extent practicable--
(i) each population addressed by any estimate of public health effects;
(ii) the expected risk or central estimate of risk for the specific populations;
(iii) each appropriate upper-bound or lower-bound estimate of risk;
(iv) each significant uncertainty identified in the process of the assessment of public health effects and studies that would assist in resolving the uncertainty; and
(v) peer-reviewed studies known to the Administrator that support, are directly relevant to, or fail to support any estimate of public health effects and the methodology used to reconcile inconsistencies in the scientific data.
[...]
 
(12) Certain contaminants
(B) Sulfate
(i) Additional study Prior to promulgating a national primary drinking water regulation for sulfate, the Administrator and the Director of the CDC shall jointly conduct an additional study to establish a reliable dose-response relationship for the adverse human health effects that may result from exposure to sulfate in drinking water, including the health effects that may be experienced by groups within the general population (including infants and travelers) that are potentially at greater risk of adverse health effects as the result of such exposure. The study shall be conducted in consultation with interested States, shall be based on the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices, and shall be completed not later than 30 months after August 6, 1996. 
 
42 U.S.C. § 300g-1 (emphasis added). 
 
§ 300j-2. Grants for State programs 
 
(d) New York City (NYC) watershed protection program
(1) In general The Administrator is authorized to provide financial assistance to the State of New York for demonstration projects implemented as part of the watershed program for the protection and enhancement of the quality of source waters of the NYC water supply system, including projects that demonstrate, assess, or provide for comprehensive monitoring and surveillance and projects necessary to comply with the criteria for avoiding filtration contained in 40 C.F.R. 141.71. Demonstration projects which shall be eligible for financial assistance shall be certified to the Administrator by the State of New York as satisfying the purposes of this subsection. In certifying projects to the Administrator, the State of New York shall give priority to monitoring projects that have undergone peer review.
 
42 U.S.C. § 300j-2 (emphasis added). 
1.3.3.2 SWDA (SNPRM Comments)
Comment [23-84]: Commenter (10973) writes that additional review of alignment to statutory requirements is needed.
On 85 FR 15398, the proposal indicates that "this internal agency procedure is intended to be consistent with the statues that EPA administers and EPA plants to implement this procedural rulemaking in accordance with all applicable statutory and regulatory requirements." This statement is appreciated, and we encourage EPA to provide additional analysis to confirm alignment with these requirements.
As noted in our 2018 comments, the SDWA requires that regulatory decisions be based upon "(i) the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices; and (ii) data collected by accepted methods or best available methods (if the reliability of the method and the nature of the decision justifies use of the data)." (Sec. 1412 (b)(3)(A)). SDWA does not distinguish peer-reviewed science that has data and models that are publicly available from those where data and models may not be publicly available. Other environmental statues have similar provisions which should also be carefully considered.
The current proposal in general is more flexible than the 2018 version in providing additional guidance on situations where "tiered access" can be established and/or EPA can waive the requirements for publicly available information, which will help to reduce potential conflicts between this Agency policy and statutory requirements. In finalizing this rule, a more formal assessment of potential conflicts between this policy and statutory language should be completed to assure that this policy does not hamper important regulatory decisions in SDWA or elsewhere. Additionally, by using EPA's alternate approach for prioritizing publicly available data and models but not eliminating those that are not, potential conflicts can be reduced further.
Comment [48-2185]: Commenter (12727) notes that just as EPA is required to consider all available science when making its regulatory decisions, governing environmental and public health statutes likewise require EPA to consider all available science when releasing ISI, which often forms the basis for the agency's regulatory decisions. For example the SDWA generally requires EPA to use "the best available, peer-reviewed science," 
A rule that prohibits EPA from considering (or that downgrades) valid, high quality scientific studies when generating ISI such as the products identified above would contravene the above- noted statutory directives regarding EPA's obligation to consider all available science.
Comment [81-2187]: Commenter (12715) writes that the proposed rule conflicts with fundamental statutory requirements to use the best available science and citation to "housekeeping" authority cannot cure that defect.
Whether EPA is arbitrarily barring consideration of scientific studies or information with nonpublic underlying data or data that are not able to be independently validated, or arbitrarily ascribing less weight to such studies or information, neither approach constitutes "housekeeping," and neither comports with statutory requirements that EPA use the best available science. To reiterate from the 2018 Comments, EPA's obligation with respect to the use of scientific information is clear: it must base its decisions on such criteria as the latest scientific knowledge, the best available, peer-reviewed science, and/or generally accepted scientific principles or laboratory tests. To cite just a few examples, in performing its duties, EPA must rely on: "the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices," SDWA of 1996, 42 U.S.C. § 300g-1(b)(3)(A)(i). EPA has repeatedly emphasized the importance of this fundamental precept, including in its 2018-2022 strategic plan, which states that one of the agency's priorities is to "identify, assess, conduct, and apply the best available science to address current and future environmental hazards."
These statutory requirements apply to all aspects of EPA's decision-making, including its scientific evaluations, adoption of regulatory standards, and development of policies. They do not permit what EPA proposes here: to either exclude or give less weight to scientific studies or information based on criteria -- availability of the underlying data and ability to independently validate underlying data -- that are not determinative of whether the studies or information constitute the best available science. Agencies may not adopt or implement regulations that conflict with the statutes under which they are promulgated, and an agency's interpretation of those statutes must always at least be reasonable. See Chevron, U.S.A., Inc. v. Nat. Res. Def. Council, Inc., 467 U.S. 837, 842-44 (1984). Furthermore, neither 5 U.S.C. § 301 nor any other purported housekeeping authority allows EPA to negate these statutes' fundamental, substantive, and common-sense commands. And even assuming EPA has some inherent housekeeping authority, it cannot rely on a general grant of rulemaking authority to promulgate regulations or develop policies that are otherwise inconsistent with more specific statutory directives. Glob. Van Lines, Inc. v. Interstate Commerce Comm'n, 714 F.2d 1290, 1293-97 (5th Cir. 1983).
Finally, EPA acknowledges that the statutes it administers would control in the event of a conflict between the proposed rule and those statutes, see 85 Fed. Reg. at 15,398, but that caveat cannot save the proposed rule. First, the core principle of the proposed rule is facially invalid, i.e., there is no circumstance in which it would be acceptable for EPA to exclude or give less weight to relevant, probative scientific studies, models, or other information that have been validated through peer review, on the sole basis that the underlying data are not publicly available or are not able to be independently validated. EPA cannot save an invalid rule by including what amounts to a waiver procedure, because the "essence of waiver is the assumed validity of the general rule." Alltel Corp. v. Fed. Commc'n Comm'n, 838 F.2d 551, 561 (D.C. Cir. 1988). Second, the agency's essential duty in rulemaking is to exercise its subject matter expertise to enact rules that implement and further the purposes of the statutes Congress has assigned it to administer. Chevron, 467 U.S. at 842-44. This does not include proposing rules without first determining the source of legal authority but instead asking commenters to supply that information. Third, the agency must propose its rules with specificity and be clear as to the programs, regulations, and information to which they will apply. 5 U.S.C. § 553(b)(3); Home Box Office, Inc., supra, 567 F.2d at 35-36.
Comment [82-2084]: Commenter (12464) writes that the cited provisions do not authorize EPA to promulgate the proposal.
"[I]t is `axiomatic' that `administrative agencies may act only pursuant to authority delegated to them by Congress.'" None of the cited provisions, however, provide EPA with authority to ignore relevant scientific information. The Proposal, however, is not related to EPA research or research funding, and will actively hinder EPA's ability to achieve the statutes' underlying goals by limiting EPA's ability to rely on the best available science.
EPA has long accepted that its decisions must be guided by the best available science. For example, in 2002, EPA issued Information Quality Guidelines in which it took the position that the standard set forth in the SDWA -- to use "the best available, peer-reviewed science" -- should apply to all of the agency's risk assessments. EPA even reiterated its commitment to this standard in the Initial Proposal. 83 Fed. Reg. at 18,769. However, the Proposal would directly undermine this longstanding commitment to use the best available science. As we pointed out in a comment letter in response to the Initial Proposal, the Proposal would prevent EPA from relying on crucial studies that underlie a number of past, present, and future regulations. This radical departure from EPA's standard practice cannot be justified by any of the statutes EPA identifies because it is contrary to the goals of those statutes.
Comment [89-2194]: Commenter (11403) writes that the proposed rule is not based on any valid statutory authority and is arbitrary and capricious.
a. The environmental statutory provisions cited by EPA do not authorize the proposed rule
EPA's 2018 version of this rule cited a hodgepodge of statutory authority, none of it valid. EPA at least corrected the obvious typographical errors from the previous version. However, those edits still do not grant EPA the authority to restrict science from all regulatory decision-making processes within the agency. EPA is attempting to use general grants of authority to create a completely novel approach to assessing science. However, as the Supreme Court has stated, "Congress . . . does not alter the fundamental details of a regulatory scheme in vague terms or ancillary provisions -- it does not, one might say, hide elephants in mouseholes." EPA does not have the authority to alter the fundamental details of all of its regulatory schemes based on such vague provisions of law. Section 1442 of the SDWA generally authorizes the Administrator to "conduct research, studies, and demonstrations relating to the causes, diagnosis, . . . and other impairments of man resulting directly or indirectly from contaminants in water, or to the provision of dependably safe supply of drinking water . . ." This section does not authorize EPA to exclude science based on a perceived lack of transparency or support any other aspect of the proposed rule.
Section 1450(a)(1) of the SDWA states, "The Administrator is authorized to prescribe such regulations as are necessary or appropriate to carry out his functions under this subchapter." This catch-all provision provides the authority to prescribe regulations, but EPA has not provided a single rationale explaining how the proposed rule is "necessary" to carrying out the functions under the SDWA.
In short, none of the statutory provisions cited by EPA provide authority for the proposed rule. The proposed rule fails at step one under Chevron v. Natural Resources Defense Council since it is unreasonable to believe that any statutory general grant of authority or any mention of EPA's oversight of other science or research programs would enable EPA to conduct rulemaking in a way that is altogether inconsistent with the rule of law. None of the cited provisions provide the authority sought by EPA, and as such the proposed rule should be withdrawn.
Comment [93-2087]: Commenter (11395) notes that the SDWA stipulates that, in risk assessment, management and communication actions that are based on science, "the Administrator shall use the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices" and defines "best available science," as "science that is reliable and unbiased ... conducted in accordance with sound and objective science practices, including, when available, peer reviewed science and supporting studies and data collected by accepted methods or best available methods." (40 CFR 300g-1) Again, a rule that precludes that use of high-quality peer reviewed studies in significant actions would contradict the SDWA language.
Comment [97-2150]: Commenter (11479) writes that the proposed rule, as modified in the supplemental notice, would clearly interfere with the agency's use of the best available science. For example, the policy violates laws like the TSCA and the SDWA because of the statutory requirement to use the "best available science." That requirement does not make exception for the agency to effectively ignore certain scientific information of high quality simply because the unanalyzed data cannot be made public for reasons of privacy, confidentiality, or intellectual property. According to judicial precedent, "best available science" is defined as "all existing scientific evidence relevant to the decision" and a body of evidence that "cannot ignore existing data." 
In fact, virtually all of EPA's statutory mandates require the use of the best available scientific evidence. Congress did not state in any of these laws that the agency shall only consider scientific evidence derived from studies where the underlying unanalyzed data is publicly available, nor that the evidence considered should be solely at the Administrator's discretion.
Comment [125-2200]: Commenter (11495) writes that EPA's Supplemental Notice is at odds with provisions of multiple environmental statutes which require the Agency to consider certain high-quality scientific information. For example, the SDWA requires that the issuance of national drinking water regulations be based on the "best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices." 
Each of these provisions directs EPA to incorporate a certain quality of scientific information into its rulemaking decisions made pursuant to the statute. In particular, these requirements are designed to ensure the scientific validity of the information upon which EPA bases its actions. Mandating a high-quality scientific basis for regulatory actions is necessary to guarantee the best possible protections for public health and welfare and to minimize partisan influence in scientific decision-making.
The Supplemental Notice, however, would impermissibly prohibit the agency from considering of relevant and high-quality scientific studies whenever the data underlying the study are not publicly available -- in direct contravention of the statutory directives that mandate, for example, consideration of the "best available science." None of the provisions requires, or even suggests, that EPA only consider information that is publicly available. It strains the plain meaning of words like "best," "latest," and even "appropriate" to suggest that they permit the unscientific requirement that all data be publicly available.
EPA "cannot ignore available . . . information." Courts have held that "best available data" requirements "prohibit[] [an agency] from disregarding available scientific evidence that is in some way better than the evidence [it] relies on." Furthermore, a plaintiff may establish a violation of a "best available science" requirement by "point[ing] to any scientific evidence that the agency failed to consider." The Supplemental Notice would undoubtedly preclude EPA from considering certain studies and scientific information that is "better" or the "best available," simply because the study's underlying data is not publicly available, thereby failing to satisfy Congressional directives and undermining the quality of regulatory decisions.
Likewise, EPA cannot avoid the statutory directives to consider high-quality scientific information simply by assigning a lesser weight to such information when the underlying data are not publicly available, as the Agency alternatively proposes. The Supplemental Notice's alternative proposal is designed to have largely the same pernicious effects as flatly prohibiting consideration of studies lacking publicly available data. Just as with the primary proposal, EPA's alternative proposal would limit the agency's consideration of the best available science in making important regulatory decisions, and is, therefore, arbitrary, and capricious.
The effect of the Proposed Rule is inconsistent with EPA's view that the Supplemental Notice is "intended to be consistent with the statutes that EPA administers and EPA plans to implement this procedural rulemaking in accordance with all applicable statutory and regulatory requirements." EPA attempts to resolve this inconsistency by stating that "in the event the procedures outlined in the proposed rulemaking conflict with the statutes that EPA administers, or their implementing regulations, the statutes and regulations will control." However, as discussed below, EPA lacks the statutory authority to issue a rule this broad, even when it claims that it will avoid statutory conflicts. 
Comment [156-1380]: Commenter (12418) writes that making regulatory determinations based on whether a study has publicly available data and models instead of on the quality of the study conflicts with existing statutory mandates.
Giving more consideration to studies where "underlying data and models are available" does not fulfill EPA's requirement to protect public health from harmful substances in air, water, and other media using the best available science. The laws that EPA administers to protect the public from harmful substances in the environment provide guidance on how scientific evidence is incorporated into rulemaking. Under the statutory text of the SDWA, EPA must identify whether drinking water contaminants present a threat to public health using "the best available, peer-reviewed science" and the decision to regulate a contaminant must be based on the "best available public health information." In the context of developing MCL, this process involves an extensive literature review of the available evidence, identification of the highest quality studies for sensitive health effects, and selection of a critical study that will serve as the basis of regulatory values. 
In the Supplemental Notice of Proposed Rulemaking (SNPR), EPA has failed to identify how relying more heavily on data and models that are publicly available fulfills these statutory mandates to use the best available science in decision making. EPA presents no evidence that studies in which the underlying data and models are publicly available are more valid or credible in a manner sufficient to justify greater consideration than studies where the underlying data are not available. Lacking in empirical support, the proposed rule conflicts with legal requirements for EPA's use of science in regulatory decision-making.
Comment [159-2090]: Commenter (12430) notes that in the SNPRM, U.S. EPA proposes to consider only (or, in the alternative, more heavily weight) studies and models with publicly available data. However, restricting agency consideration of scientific studies and models would run afoul of U.S. EPA's obligations under many of its operating statutes to rely on the best available science. For example:
 The SDWA requires that findings which support a determination to regulate a contaminant "be based on the best available public health information" and that, in developing the NPDWR, "to the degree that an Agency action is based on science, the Administrator shall use the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices."
Comment: [187-1704]: Commenter (12720) notes that EPA cites 42 U.S.C. § 300j - 1 of the SDWA as authority for the rule. Subsection (a) of that section allows EPA to conduct some types of research on drinking water contamination and requires it to conduct other studies. But it says nothing about which types of studies EPA may consider in rulemakings and does not distinguish between dose-response studies and other types of studies. The absence of any such congressional distinction or restriction makes it all the more obvious that EPA invented and added the distinction and restrictions in the Proposal and Supplemental Proposal as a matter of its own policy preferences, contrary to the Act. This EPA may not do. The remainder of the subsections have nothing to do with data or research. At any rate, EPA does not state specifically which of the subsections in 42 U.S.C. § 300j - 1 it believes authorizes this proposed rule. Thus, the citation fails to provide sufficient notice for the public to comment on the proposed rule.
EPA also cites 42 U.S.C. § 300j - 9(a)(1), but that says only that the "Administrator is authorized to prescribe such regulations as are necessary or appropriate to carry out his functions under this subchapter." As explained above, that general grant of authority cannot support the rule, especially when the rule conflicts with the Act. The SDWA does not authorize the proposed rule.
Comment [198-2210]: Commenter (13549) writes that as a baseline, whether working on "significant regulatory actions," ISI, or anything requiring the consideration of science, EPA must base its decisions on the best available science. Anything less is nonsensical, arbitrary, and in violation of the environmental laws EPA is charged with implementing. The SNPRM does not maintain this baseline, but instead provides for ways EPA would exclude and lessen consideration of the best available science. Therefore, the SNPRM, along with the proposed rule, must be withdrawn. See, e.g., 42 U.S.C. § 300g-1(b)(1)(B)(ii), (3)(A) (SDWA provisions requiring regulation of contaminants based on best available science); 15 U.S.C. § 2625(h) (TSCA provision requiring the use of best available science); see also 83 Fed. Reg. at 18,769 ("The best available science must serve as the foundation of EPA's regulatory actions.").
1.3.4 Toxic Substances and Control Act (TSCA) (NPRM and SNPRM Comments)
Comment: Sections 1.3.4.1 and 1.3.4.2 of this Chapter present comments received on the NPRM (Section 1.3.4.1) and SNPRM (Section 1.3.4.2) related to the EPA's legal authority for the rule under the TSCA. The EPA provides the following general response to these comments:
Response: EPA appreciates your comments. As discussed in the Section I.C of the final rule preamble and in this RTC document in Section 1.2, EPA has decided it is most appropriate to rely on its authority to promulgate housekeeping regulations, rather than any of the substantive environmental statutes cited in the 2018 proposal or the 2020 supplemental proposal as legal authority for the rule. EPA intends to promulgate future science transparency rules under its substantive environmental statutes. Additionally, the final rule states in § 30.3 that "[i]n the event the procedures outlined in this rule conflict with statutes that EPA administers, or their implementing regulations, the statutes and regulations will control."
1.3.4.1 TSCA (NPRM Comments)
Comment [37-33]: Commenter (0046) writes that while TSCA Section 26 is not identified in the proposal, it includes provisions that raise questions about EPA's authority for and potential application of the proposal. Section 26(h), "Scientific Standards", states that "to the extent that the Administrator makes a decision based on science, the Administrator shall use scientific information, technical procedures, measures, methods, protocols, methodologies, or models, employed in a manner consistent with the best available science." EPA must consider each of these factors "as applicable." The availability of sufficient underlying data to "validate" or "reproduce" study results is ​not among the relevant factors that EPA must consider.
Similarly, section 26(i) addresses WEIGHT OF SCIENTIFIC EVIDENCE directing that:" -- The Administrator shall make decisions under sections 4, 5, and 6 based on the weight of the scientific evidence." This requires the Administrator to evaluate the totality of available scientific evidence and make a judgment about its "weight"  -  not excluding evidence based solely on the availability of data sufficient for its validation. 
These subsections indicate, at a minimum, that this proposal to require that "dose response data and models underlying pivotal regulatory science are publicly available in a manner sufficient for independent validation...[w]hen promulgating significant regulatory actions" may not be consistent with the scientific standards and methodology for decision-making Congress prescribed for such actions under TSCA Section 26.
Comment: [996-3804]: Commenter (1011) believes that the proposed rule would create unnecessary difficulties for EPA and its ability to comply with statutes that rely upon the assessment of the best scientific data currently available. This includes the LSCA, which updates the TSCA and requires the Administrator to use "scientific information, technical procedures, measures, methods, protocols, methodologies, or models, employed in a manner consistent with the best available science." EPA should reconsider this proposed rule to ensure that any new guidelines comply with applicable laws and encourage the use of all relevant scientific information, even if the underlying data cannot be made public.
Comment [2322-2720]: Commenter (2337) notes that EPA's core mission, reflected in numerous environmental statutes, is to protect public health and the environment. These same statutes require that EPA, in fulfilling this mission, support and rely on the best available science. While the specific terms of each statute vary, their direction is similar: EPA is charged with conducting and supporting research into the impacts of pollutants on human health and the environment, using available scientifically accepted data to determine when pollutants pose a danger to human health or the environment, and establishing standards with a sufficient margin of safety to protect public health. Indeed, this requirement that EPA use the latest or best available science is repeated in numerous environmental statutes. The TSCA directs the Administrator, when making decisions based on science, to act "in a manner consistent with the best available science" and consider a number of factors, including the reasonableness of the methods used to develop the information, the completeness of the data, the extent of uncertainty, and independent verification or peer review. Notably, public availability or independent verification are not disqualifying requirements. 
Comment [2533-4926]: Commenter (2548) writes that they have spent the past eighteen years of their career as a faculty member at highly ranked research universities (The University of North Carolina at Chapel Hill and Duke University) and have devoted the majority of this time to studying the potential health effects of exposures in the natural and built environments. They have served as President of the largest professional organization of biostatisticians in North America, as Chair of the Biometrics Section of the American Statistical Association, and as a standing member of numerous review groups at the NIH. They are currently a member of the National Academies' Committee on Applied and Theoretical Statistics and have recently chaired selection committees for two of the top prizes in statistical science worldwide.
Their major concern regarding the proposed rule is that it violates the reformed TSCA, and indeed more importantly the best principles of science, in eliminating from consideration a class of studies that offer both large-scale population coverage and careful exposure assessment. These studies often provide our best information regarding potential health effects of environmental contaminants. In environmental health research, many research questions (e.g., health effects of smoking, exposure to high levels of other air pollutants) cannot be evaluated in randomized controlled trials of humans for ethical reasons. In such cases, large observational cohort studies typically provide the best "real world" data on associations between environmental contaminant levels and health outcomes like heart attack, stroke, asthma, and premature death.
Comment [2772-2263]: Commenter (2787) writes that the proposed rule is inconsistent with EPA's statutory obligations to ground its actions on scientific evidence. The TSCA and the SDWA require that EPA use the "best available science."
Comment [2887-4671]: Commenter (2902) writes that many questions remain as to whether the proposed rule violates existing laws, such as the TSCA. The rule, as written, appears poorly designed and without the mission of the EPA in mind. 
Comment [4582-4139]: Commenter (4593) writes that if EPA had adopted this data transparency limitation in past risk assessments, EPA would not have been able to take many of its historic actions to protect children, families, and the environment such as reducing or eliminating the exposure of children to lead in paint, gasoline and drinking water and promulgating protective air quality standards for PM and other air pollutants.
The proposed policy would affect assessments that will soon be carried out under TSCA Section 6. TSCA gives EPA the authority to regulate the manufacture, processing, distribution in commerce, use and disposal of chemicals. The problem formulation documents which set forth EPA's approach for assessing the first 10 chemicals under the amended TSCA are open for public comment now. How these chemicals are assessed will be the model for future assessments. The proposed policy would in fact make it impossible for EPA to consider the full array of well conducted and peer reviewed scientific studies of the health and environmental effects of pollution. It would bias the body of information in favor of industry supplied studies since they would have the means to provide the underlying data. Assessment of all relevant scientific information is essential in making sound judgments about protecting public health and the environment and is a legal requirement in all major environmental legislation.
TSCA also contains provisions to require chemical manufacturers to test the chemicals that they manufacture or process. To require industry to test chemicals under Section 4, EPA must make a set of legal findings.
It is the data inadequacy finding that we are interested in today for it is the nexus between TSCA and the proposed transparency policy. To make this finding, EPA conducts a thorough literature search and usually issues a rule to require studies that have not been published be submitted to the Agency. The bulk of the information considered, however, is typically studies published in peer-reviewed scientific journals. Despite being accepted by the scientific community, these studies do not meet the proposed transparency requirements of the proposed rule, since it requires that all raw underlying data and the models used to analyze data supporting the study are available for public review. Thus, if the transparency rule were in effect, EPA would have to judge studies from peer reviewed journals as inadequate. Ignoring this large category of information would cost industry hundreds of millions of dollars to repeat perfectly good, scientifically acceptable studies, which the public would ultimately pay for through higher prices. And it would significantly delay or, in some cases, preclude assessment and regulation of risks to human health and the environment.
Comment [4827-2074]: Commenter (4838) writes that the proposed rule violates several environmental statutes because it hinders EPA's ability to rely on best available science, or the most up-to-date information, as they require. The CAA, CWA, SDWA, TSCA, and EPCRA all require certain decisions or regulatory criteria be based on the most up-to-date science. These criteria are described as "best available science," "latest scientific knowledge," and "best available public health information." The proposed rule would illegally limit EPA's ability to rely on best available science in violation of these statutes.
Comment [4837-3059]: Commenter (4848) notes that administrative procedure requires that EPA consider data submitted by the public in evaluating regulations. Scientific studies submitted have always been of uneven quality. EPA has processes in place (including use of a scientific advisory board, testimony and written and oral public notice and comment) using both internal and external peer review to evaluate data. Depending on context, some studies are given greater weight than others. Some studies are disregarded entirely.
It is inappropriate, and likely unlawful under the APA and the express language of numerous environmental statutes including the SDWA and the TSCA, for EPA to categorically eliminate certain types of studies, and hence certain types of data, without considering context. 
Comment [4867-2026]: Commenter (4878) notes that the TSCA is EPA's primary authority for regulating non-pesticide chemicals. Under TSCA, EPA can secure information on new and existing chemicals and regulate chemicals it determines pose an unreasonable risk to public health or the environment. All studies used would be subject to the proposed rule.
In late 2016, the EPA moved to ban toxic chemical trichloroethylene (TCE) due to health risks, including a risk of Parkinson's disease, though this action is still pending. The original recommendation was based on hundreds of studies, many of which would not be considered under the proposed rule. 
For example, one study sent questionnaires to 134 people who had formerly worked on a site with heavy and long-term exposure to TCE. Fourteen had signs of Parkinson's disease, and an additional thirteen showed mild features of the condition, far more than expected given the population. Another asked twin pairs about exposure to solvents including TCE and showed a significant association between TCE exposure and Parkinson's disease risk. 
In these relatively small studies, a distinctive characteristic  --  people who all worked together, and twins  --  respectively, combined with the most basic additional medical information could render the participants identifiable. Both of the TCE studies are highly cited and the findings have been replicated. To exclude this evidence that TCE exposure is a risk factor for Parkinson's disease is illogical and does not serve the best health interests of the American public.
Furthermore, the proposed rule does not comply with the letter of TSCA. TSCA and other statutes administered by EPA requires the agency use the "best available science" and none require the agency to access to raw data. TSCA additionally requires that EPA consider all information that is reasonably available to the administrator. As drafted, the proposed rule violates these statutes because it would force the agency to ignore some of the best information available.
Comment [5865-330]: Commenter (6111) writes that EPA's historic position is consistent with the Statutes. The TSCA requires that regulation of chemical substances be "consistent with the best available science" and that EPA make decisions "based on the weight of the scientific evidence."
Furthermore, because EPA is required under the Statutes to assess the public health benefits of its regulations, it must take into account all relevant science and cannot arbitrarily exclude certain studies demonstrating those benefits. 
Many of the fundamental public health studies on which EPA has based key rules and standards under the Statutes are studies for which the raw data were not or could not have been released. Attachment 1 to this letter contains a partial list of studies [note: included under a separate excerpt] that likely contain confidential data; these are all studies on which EPA has relied and cited as the basis for its actions under some of the Statutes. Until now, release of the underlying raw data was not an EPA criterion for determining the "best available" reports, studies, analyses, or models. Indeed, none of the Statutes invoked by EPA as support for the proposed rule limits EPA in this fashion; none of the Statutes requires EPA to make raw data publicly available.
EPA's proposed new approach, which conflicts with the agency's obligations and curtails its authority, is irrational at best and detrimental to public health and safety at worst.
Comment [6093-532]: Commenter (6103) writes that the rule is counter to mandates in the reformed TSCA and other authorizing laws to use the best available science. Many of the statutes implemented by the EPA require the use of "best available science." This rule would actually undermine the use of best available science as called for in the statutes. As a result, the policy as drafted would be in violation of laws like the TSCA and the SDWA because the proposal would allow for the agency to ignore some of the best information that could help implement these laws.
Comment [6112-1557]: Commenter (6137) writes that EPA's reliance on the TSCA, 15 U.S.C. § 2609, as authority for the Proposed Rule, fares no better. Indeed, much like many of the other statutory provisions upon which EPA relies, this provision governs EPA's authority to conduct and support research and does not address EPA's authority to use scientific data or research in support of regulatory decisions. 
Specifically, section 2609 grants EPA authority to "conduct such research, development, and monitoring as is necessary to carry out the purposes of this chapter. [EPA] may enter into contracts and may make grants for research, development, and monitoring under this subsection." 15 U.S.C. § 2609(a). Section 2609 grants EPA additional related authorities, including authority to: 
 Create and operate information systems to store data relevant to chemical substances, id. § 2609(b); 
 Develop "screening techniques for carcinogenic, mutagenic, teratogenic, and ecological effects of chemical[s]," id. § 2609(c); 
 Establish a research program to develop chemical "monitoring techniques and instruments," id. § 2609(d);
 Conduct "basic research" on chemical screening and monitoring, id. § 2609(e), and train federal scientists on chemical screening and monitoring, id. § 2609(f); and 
 Develop systems for information sharing among "Federal, state, and local authorities," id. § 2609(g). 
Notably absent from the list of authorities under section 2609 is EPA's authority to determine what science it can consider when making regulatory decisions. Instead, the provision solely focuses on EPA's ability to conduct research or to fund research, independent of whether that research will or may be used by EPA to make regulatory decisions. Thus, section 2609 does not provide any basis for the authority claimed in the Proposed Rule.
Moreover, § 2625 of TSCA governs how EPA uses science when exercising its main regulatory powers under the statute and establishes detailed criteria that EPA must use when "the Administrator makes a decision based on science" when carrying out its regulatory powers. 15 U.S.C. § 2625(h). Thus, this provision, and not those cited by EPA, would theoretically govern a rule related to the use of science. See Bennett v. Islamic Republic of Iran, 618 F.3d 19, 25 (D.C. Cir. 2010) ("`where Congress includes particular language in one section of a statute but omits it in another section of the same Act, it is generally presumed that Congress acts intentionally and purposefully in the disparate inclusion or exclusion.'" (quoting Russello v. United States, 464 U.S. 16, 23 (1983)). However, and as discussed more fully infra, Section IV.A, the Proposed Rule contravenes the requirements of § 2625 that EPA consider all "reasonably available information" when making regulatory decisions, and thus is not authorized by this provision either.
Comment [6112-1600]: Commenter (6137) writes that EPA's proposed refusal to consider or use science relevant to decisions that will affect public health directly contravenes the newly enacted revisions to TSCA. Numerous provisions of TSCA make clear that EPA may not prohibit the consideration of non-public data in regulatory decision-making under TSCA. Indeed, when viewed as a whole, TSCA establishes a comprehensive scheme for how EPA is to evaluate and use science in making regulatory decisions that forecloses the Proposed Rule. 
First, TSCA requires EPA to consider all "reasonably available information" when making any regulatory decisions under sections 2603, 2604, and 2605. 15 U.S.C. § 2625(k) (emphasis added) (EPA "shall take into consideration information relating to a chemical . . . that is reasonably available to [the Agency]"); also, id. § 2605(c)(2)(A). Thus, the statute mandates that if a study is reasonably available to EPA, EPA must consider it when making a significant regulatory decision under these provisions of the statute. Whether the data underlying a scientific study is publicly available has no bearing on whether the study itself is reasonably available to EPA. Because the Proposed Rule purports to apply to all significant regulatory decisions made by EPA under TSCA  -  including those made under these provisions  -  it is unlawful.
Second, when making any regulatory decision under sections 2603, 2604, and 2605, TSCA requires EPA to make an individualized evaluation of any information reasonably available to the Agency, and thus, prohibits the blanket ban erected in the Proposed Rule. Section 2625(h) establishes five statutory factors that EPA must consider when "mak[ing] a decision based on science." 15 U.S.C. § 2625(h)(1)-(5). One of these statutory factors expressly addresses situations in which non-public scientific data is before the Agency and requires the Agency to "consider . . . the extent of independent verification or peer review of the information or of the procedures, measures, methods, protocols, methodologies, or models." Id. § 2625(h)(5). In addition, EPA must consider whether the methodologies used to collect the data are "reasonable," id. § 2625(h)(1), and the "degree of clarity and completeness" with which the methods used were documented, id. § 2625(h)(3). In sum, Section 2625(h) requires EPA to review each scientific study on a case-by-case basis to determine whether and how to use it. This case-by-case evaluation requires EPA to consider the public or non-public nature of the underlying data as one of many factors and prohibits EPA from implementing a blanket ban on the use of non-public data in significant regulatory decisions under TSCA.
Third, the Proposed Rule is at odds with the requirement that EPA act "consistent with the best available science." Id. § 2625(h) (emphasis added). Although Congress did not define the term in TSCA, it is clear from other statutes that an agency cannot lawfully act consistent with the best available science when it categorically bars consideration of any science based on non-public data. For example, the ESA requires federal agencies to consider the "best scientific and commercial data available," see 16 U.S.C. § 1536(a)(2), and courts have held that this provision requires an agency to consider "all relevant data . . . even when it is imperfect, weak, and not necessarily dispositive." League of Wilderness Defenders/Blue Mountains Biodiversity Project v. Connaughton, 752 F.3d 755, 763-64 (9th Cir. 2014) (emphasis added); see also Bldg. Indus. Ass'n of Superior Cal. v. Norton, 247 F.3d 1241, 1246 (D.C. Cir. 2001) (an agency "must utilize the `best scientific . . . data available,' not the best scientific data possible") (emphasis in original). EPA itself acknowledged that this sort of restriction on science is contrary to the requirements of TSCA; when analyzing the same restrictions proposed in the HONEST Act, EPA recognized:
Provisions under the newly amended TSCA . . . would be significantly impacted by the HONEST Act. First, a number of provisions in section 26 could not be upheld under the HONEST Act. Section 26(h) requires the Agency to "use scientific information, technical procedures, measures, methods, protocols, methodologies, or models, employed in a manner consistent with best available science." . . . [T]he HONEST Act would not allow EPA to use the best available science. Section 26(i) requires the Agency to use the "weight of scientific evidence" in making decisions under TSCA, and EPA believes this would not be possible given that the provisions of the HONEST Act would prohibit the use of some data. Finally, section 26(k) requires the Agency to "take into consideration information relating to a chemical substance or mixture, including hazard and exposure information, under the conditions of use, that is reasonable available." EPA would be in violation of the HONEST Act when upholding these provisions under TSCA, namely instead of using the best available science and all reasonable available data for chemical regulations, EPA would be restricted to selecting information based on availability. This approach would introduce research bias that would compromise the quality of the Agency's work. 
EPA, EPA Analysis of HONEST Act to CBO at 3-4 (2016), https://www.scribd.com/document/344731162/EPA-analysis-of-Honest-Act-to-CBO (emphasis added). 
Indeed, where Congress has sought to qualify a best available science requirement by implementing a total bar on particular types of science, it has done so expressly. For example, in the Consumer Productive Safety Improvement Act of 2008, Congress directed a panel studying phthalates to use "the most recent, best-available, peer-reviewed, scientific studies." 15 U.S.C. § 2057c(b)(2)(B) (emphasis added). The absence of any such prohibition in TSCA is further proof that the Proposed Rule is prohibited by the statute's best-available-science requirement.
Fourth, the Proposed Rule is inconsistent with the TSCA requirement that EPA make regulatory decisions using a "weight of the scientific evidence" approach. Id. § 2625(i). As EPA has itself recognized, this approach requires the Agency to individually evaluate the strengths and weakness of any study reasonably available to the Agency. 40 C.F.R. § 702.33 (defining "weight of scientific evidence" as "comprehensively, objectively, transparently, and consistently, identify[ing] and evaluat[ing] each stream of evidence, including strengths, limitations, and relevance of each study and "integrat[ing] evidence as necessary and appropriate based upon strengths, limitations, and relevance" for purposes of risk evaluations under 15 U.S.C. § 2605 (emphasis added)). Thus, the Proposed Rule's outright ban on consideration of scientific studies that rely on non-public data is prohibited by the weight of the scientific evidence approach required under TSCA.
In sum, these provisions  -  considered together and in light of other provisions of the statute  -  establish a comprehensive scheme for how EPA is to consider scientific data, and this scheme prohibits the Proposed Rule's ban on the consideration of non-public data. Together, they require EPA to consider all reasonably available scientific information; evaluate each piece of information, including the methods by which it was acquired and analyzed; use each piece of information in a manner consistent with the best available science; and give each piece of information its due weight.
In addition, in deciding whether or not "there may be a reasonable basis to conclude that a chemical substance or mixture presents a significant risk of serious or widespread harm to human beings," EPA must consider "any . . . information available to the Administrator." 15 U.S.C. § 2603(f). And other provisions of TSCA expressly address certain types of non-public data and authorize EPA to consider it in making regulatory decisions. See 15 U.S.C. §§ 2604(b), 2613. 
In light of the numerous provisions of TSCA addressing the consideration of scientific data, it is evident that if Congress had intended to allow EPA to bar the consideration of nonpublic data, it surely would have said so expressly. Given the comprehensiveness of these provisions providing otherwise, there is simply no room for a blanket ban on science that relies on non-public data. The Proposed Rule is therefore unlawful.
Comment [6112-1625]: Commenter (6137) writes that EPA failed to follow the procedural requirements under TSCA.
Pursuant to section 2609 of TSCA, prior to promulgating any rules, EPA must "consult[] and cooperate[] with the Secretary of Health and Human Services and with other heads of appropriate departments and agencies." 15 U.S.C. § 2609(a); see also id. § 2609(b)(2)(A), (c), (d), (e), (g). Therefore, to the extent EPA relies on TSCA as authority for the Proposed Rule, it must consult and cooperate with the Secretary of HHS. Yet nowhere in the Proposed Rule does EPA indicate that it has done so. Failure to comply with this requirement would render any final rule unlawful. See 5 U.S.C. § 706(2)(D) (a reviewing court "shall . . . hold unlawful and set aside agency action . . . found to be . . . without observance of procedure required by law").
Comment [6118-1732]: Commenter (6143) notes that EPA has issued its proposed rule on Strengthening Transparency in Regulatory Science pursuant to its authority under several environmental statutes that that it implements, including the CAA; CWA; SDWA; RCRA; CERCLA; EPCRA; FIFRA; and TSCA. This proposed action is fully consistent with the authorities and objectives laid out in those statutes. EPA must ensure that any final action it takes on this proposal continues to respect the balance reflected in those statutes between sound, transparent science, and legitimate privacy interests.
Comment [6118-1733]: Commenter (6143) notes that in its proposal, the Agency recognizes that "[t]he best available science must serve as the foundation of EPA's regulatory actions." This is true not only as a matter of good governance, but as a statutory requirement. In particular, TSCA requires EPA to "use scientific information, technical procedures, measures, methods, protocols, methodologies, or models . . . in a manner consistent with the best available science" when making regulatory decisions that are based on science. Further, Congress directed EPA to consider the following five specific factors in doing so:
 the extent to which the scientific information, technical procedures, measures, methods, protocols, methodologies, or models employed to generate the information are reasonable for and consistent with the intended use of the information;
 the extent to which the information is relevant for the Administrator's use in making a decision about a chemical substance or mixture;
 the degree of clarity and completeness with which the data, assumptions, methods, quality assurance, and analyses employed to generate the information are documented;
 the extent to which the variability and uncertainty in the information, or in the procedures, measures, methods, protocols, methodologies, or models, are evaluated and characterized; and
 the extent of independent verification or peer review of the information or of the procedures, measures, methods, protocols, methodologies, or models.
Comment [6118-1734]: Commenter (6143) writes that EPA's proposed action will help the Agency to carry out TSCA's requirements regarding the use of the "best available science." By ensuring greater transparency into the data, assumptions, and methodologies used in the studies on which EPA relies, this proposed rule will facilitate EPA's analysis of the factors specified in TSCA §26(h) -- particularly with respect to suitability, documentation, and consideration of uncertainty and variability.
The concern that some opponents of this proposed action have expressed -- that it will inhibit or preclude use of certain studies that they believe to be relevant and useful, but for which data sufficient to permit independent verification are not publicly available -- should be offset by recognizing that it is an explicit intent of the Agency to rely on the best available science. Science, studies, and conclusions from data that are not available for independent verification do not meet this requirement. They are not suitable for use as a basis for rulemaking insofar as they cannot be independently inspected and verified.
As the Agency recognizes in its proposal, this greater transparency serves a dual function in EPA's efforts to implement its statutory duties. Greater transparency can help to "[strengthen] the integrity of EPA's regulatory actions and its obligation to ensure the Agency is not arbitrary in its conclusions." By supporting application of the "best available science" analysis, the proposed rule will help EPA reach regulatory decisions grounded in sound, quality science that will withstand judicial review -- which, in the TSCA context, means decisions that are supported by "substantial evidence." At the same time, greater transparency will "enhanc[e] the public's ability to understand and meaningfully participate in the regulatory process." Public input is a fundamental part of the administrative process, and TSCA provides for public comment at multiple steps of the decision-making process under several of its provisions. The proposed rule will improve the public's access to data necessary to make informed comments on the science underpinning EPA's actions.
Comment [6119-1358]: Commenter (6144) writes that the policy, as drafted, is in apparent violation of laws like the TSCA and the SDWA because of the statutory requirement to use the "best available science" and allow the agency to effectively ignore certain information regardless of the quality of that work, that would help implement these science-based laws. According to judicial precedent, "best available science" is defined as "all existing scientific evidence relevant to the decision" and a body of evidence that "cannot ignore existing data." In addition to the "best available science" requirements, TSCA also requires EPA to take into consideration information "that is reasonably available to the Administrator." EPA's own regulations define that term as "information that EPA possesses or can reasonably generate, obtain, and synthesize . . . , whether or not that information is confidential business information, [sic] that is protected from public disclosure...." Not only would this proposed rule conflict with TSCA's mandate to consider the best available science, it would challenge the method by which EPA can review the weight of the scientific evidence which in rulemaking for TSCA is defined by a systematic review method. These reviews consider the entire body of scientific evidence, so the exclusion of certain studies based on whether the raw data and models are publicly available would render these reviews, as mandated, impossible.
Comment [6127-2244]: Commenter (6152) writes that the focus on reproducibility and limiting studies that contain personal health data would undermine worker health studies. Under the 2014 updates to the TSCA, EPA is required to assess worker exposures when the agency is evaluating the safety of chemicals. However, this proposal would essentially prohibit the use of epidemiology studies in that process because they are not considered reproducible in the same way as animal or toxicological tests.
Historically, epidemiology studies have been vital to identifying and protecting against harmful workplace exposures. An example from our union's history is the body of epidemiology research initiated through a labor and management partnership that identified the high incidence of bladder cancer among workers in the rubber industry. Subsequently those workers and employers worked together to reduce and eliminate the harmful chemical exposures causing occupational cancer.
Comment [6133-2162]: Commenter (6158) writes that the proposed rule likely violates federal laws that require the EPA to use the best scientific information available to it. EPA has a duty to use the best scientific information available under several laws. For example, the TSCA requires EPA to operate "in a manner consistent with the best available science" and make regulatory decisions based on the weight of the scientific evidence.
Comment [6143-2594]: Commenter (6168) writes that EPA's focus on transparency in dose-response science is inconsistent with its recent turn to secrecy.
Some brief examples include: 
 As Environmental Data and Governance Initiative (EDGI) has documented, EPA has made less accessible or removed extensive information about climate change that had been available on EPA's website. 
 EPA has failed to provide the information necessary for independent evaluation of its decisions under TSCA, e.g. EPA's recent decision to green light a new fragrance chemical despite a range of health concerns.
 EPA has pushed to delay releasing reports documenting harms of chemicals, e.g. formaldehyde and Per- and Polyfluoroalkyl Substances (PFAS). 
 EPA has reduced opportunities for public participation by creating short comment periods and trying to circumvent public comment entirely. 
 EPA's procedure for implementing TSCA is designed to limit the range of information included in regulatory decisions by limiting the sorts of uses/exposures that will be analyzed.
 EPA has drastically shifted influence away from the broader public and toward business interests, for example in private meetings and in appointments to SABs. 
Especially under Scott Pruitt -- who was Administrator when this proposed rule was written and published -- EPA has been secretive, for example denying FOIA until required to release documents by courts. Pruitt's secrecy was famous: not telling reporters where he was speaking, not letting the press into events, hiding, and even changing his calendars, etc. 
The litany of ways the EPA under the current administration has been secretive and has sought to limit information and evidence of chemical harms suggests that the idea of "transparency" grows not out of commitment to openness and public participation. Rather, it is being used here strategically, in bad faith, to limit the evidence used in regulatory decision-making and to undermine moves toward protective regulations. 
Comment [6152-652]: Commenter (6122) writes that the proposed rule is not based on any valid statutory authority and is arbitrary and capricious.
EPA cites multiple statutory provisions in an attempt to support the proposed rule that it believes authorize this type of rulemaking, "including provisions providing general authority to promulgate regulations necessary to carry out the Agency's functions under these statutes and provisions specifically addressing the Agency's conducting of and reliance on scientific activity to inform those functions . . ." None of the specific statutory provisions provided by EPA provide specific authority for EPA to fundamentally rework the foundational processes it uses to assess and include/reject science. Indeed, if such authority existed, one would have expected EPA to have discovered this authority many decades earlier. Instead, EPA is attempting to use general grants of authority to create a completely novel approach to assessing science. However, as the Supreme Court has stated, "Congress...does not alter the fundamental details of a regulatory scheme in vague terms or ancillary provisions -- it does not, one might say, hide elephants in mouseholes." And if this is true, neither can the EPA alter the fundamental details of all of its regulatory schemes based on such vague provisions of law. We review each of the provisions listed by EPA, none of which discuss transparency, dose response, pivotal regulatory science, benefit-cost calculation, or changing "agency culture," the stated goals of this rulemaking.
Section 10 of the TSCA grants the Administrator authority on research, development, collection, dissemination, and utilization of data. There is no mention of any type of research or data that should be excluded in order to carry out the purposes of the statute.
In short, none of the statutory provisions cited by EPA provide authority for the proposed rule. The proposal fails at step one under Chevron v. Natural Resources Defense Council and should be withdrawn as beyond the authority of the EPA.
Comment [6155-638]: Commenter (6125) notes that TSCA, in section 26(h), provides that in regulating "[t]he Administrator shall make decisions...based on the weight of the scientific evidence" and shall "take into consideration information...that is reasonably available to the Administrator." id at (j). The law grants EPA broad authority to act against any "unreasonable risk of injury to health or the environment" created by the use of a chemical substance or mixture. Congress directed EPA to make this unreasonable risk determination without considering costs, though costs become relevant later at the stage of regulation. This directive to consider health impacts alone makes science of even more relevance to an unreasonable risk determination than it might be under a different statutory approach, such as cost benefit balancing. 
Congress addressed in detail the types of science that EPA should use. Specifically, Congress directed that to the extent the Administrator based a regulatory decision on science, the agency use information, technical procedures, measures, methods, protocols, methodologies, or models, employed in a manner consistent with the best available science, and [to] consider as applicable -- 
 the extent to which the scientific information, technical procedures, measures, methods, protocols, methodologies, or models employed to generate the information are reasonable for and consistent with the intended use of the information; 
 the extent to which the information is relevant for the Administrator's use in making a decision about a chemical substance or mixture; 
 the degree of clarity and completeness with which the data, assumptions, methods, quality assurance, and analyses employed to generate the information are documented; 
 the extent to which the variability and uncertainty in the information, or in the procedures, measures, methods, protocols, methodologies, or models, are evaluated and characterized; and 
 the extent of independent verification or peer review of the information or of the procedures, measures, methods, protocols, methodologies, or models. 15 U.S.C. §2025 (h)
As noted above, Congress added that "[t]he Administrator shall make [regulatory] decisions...based on the weight of the scientific evidence," id at (i), and shall "take into consideration information...that is reasonably available to the Administrator." id at (j). Nowhere is the Administrator barred from considering studies for which underlying data may be unavailable. Quite the contrary. The Administrator is to evaluate studies based on a range of factors  --  including peer review, internal consistency of study methodology and how well uncertainties are characterized. 
The proposed rule is inconsistent with and would lead to decisions that violate these provisions, since it substitutes an a priori requirement of data availability to the public in place of the statutory factors. The availability of sufficient underlying data for the public to `validate' or `reproduce' study results is not even among the factors the agency is to consider, much less a determinative factor nullifying all those enumerated. The proposal is likewise at odds with the requirement that decisions be based on the "weight of the scientific evidence" since it bars evidence from the weighting process. In sum, EPA has not identified a lawful basis for applying this rule to regulatory decisions under TSCA. The general regulatory and research authorities cited in the proposal do not provide such a basis.
Small wonder, then, that the proposal, having failed to identify any substantial legal basis for limiting science under the various affected programs "solicits public comment on whether additional or alternative sources of authority are appropriate bases for this proposed regulation." 83 FR at 18771. But it is not the job of commenters to identify the legal sources of authority for the proposal, and the proposal fails to do so. That failure alone precludes adopting a final rule based on the proposal.
Comment [6155-2634]: Commenter (6125) writes that it is clear from even a brief look that these provisions require EPA to consider ​all ​the best available science, without authorizing, or even mentioning any of the per se restrictions based on "transparency" that EPA has proposed to use as a universal overriding rule. For these reasons, EPA must justify its proposal as a proper exercise of its authority under ​each ​of the statutory provisions whose operation it would affect - provisions that the agency does not even cite. It is the job of the proposer, not the commenters, to survey these provisions and provide the needed justification.
In several of the listed statutes Congress has explicitly told EPA how to use science in decision-making. 
 TSCA 15 USC Section 2605 (h) provides that "the Administrator ​shall use scientific information, technical procedures, measures, methods, protocols, methodologies, or models, employed in a manner consistent with the best available science." 
These provisions all use mandatory language directing EPA to use reliable and relevant scientific information -- and further specify how to do so. They all direct that decisions regarding protection of public health be based on the full range of relevant scientific information, in some cases specifying how to identify what information to use. None of them give EPA discretion to ignore such information for any of the reasons set forth in the proposal.
Comment [6163-1112]: Commenter (6133) notes that EPA cites 15 U.S.C. § 2609 under the TSCA as support for this rule. But that section states only that the "Administrator shall, in consultation and cooperation with the Secretary of Health and Human Services and with other heads of appropriate departments and agencies, conduct such research, development, and monitoring as is necessary to carry out the purposes of this chapter." It does not allow EPA to limit the type of data considered in regulatory decisions, nor does it draw a distinction between dose-response data and other types of data. TSCA does not support the proposed rule. The absence of any such congressional distinction or restriction makes it all the more obvious that EPA invented and added the distinction and restrictions in the Proposal as a matter of its own policy preferences, contrary to the Act. This EPA may not do. 
Comment [6163-1142]: Commenter (6133) writes that under TSCA, EPA has broad authority to protect the public from harm from chemical substances and mixtures. TSCA authorizes EPA to issue regulations designed to gather information on, require testing of, and control exposure to chemical substances and mixtures. EPA must restrict or ban any chemical substance that presents an unreasonable risk of injury to human health or the environment. See, e.g., 15 U.S.C. §§ 2603, 2604, 2605. 
TSCA contains specific provisions regarding EPA's use and consideration of science in rulemakings. "In carrying out sections 2603, 2604, and 2605," EPA must "use scientific information, technical procedures, measures, methods, protocols, methodologies, or models, employed in a manner consistent with the best available science." Id. § 2625(h). EPA must further consider the following: 
 the extent to which the scientific information, technical procedures, measures, methods, protocols, methodologies, or models employed to generate the information are reasonable for and consistent with the intended use of the information; 
 the extent to which the information is relevant for the Administrator's use in making a decision about a chemical substance or mixture; 
 the degree of clarity and completeness with which the data, assumptions, methods, quality assurance, and analyses employed to generate the information are documented; 
 the extent to which the variability and uncertainty in the information, or in the procedures, measures, methods, protocols, methodologies, or models, are evaluated and characterized; and 
 the extent of independent verification or peer review of the information or of the procedures, measures, methods, protocols, methodologies, or models. 
Id. After consideration of these matters, EPA must make decisions "based on the weight of the scientific evidence." Id.
In short, EPA must examine the reliability of a study on a case-by-case basis by weighing several indicators of scientific validity. Noticeably absent from Congress's enumerated factors in § 2625(h) is whether the underlying data can be made available to the public. While § 2625(h)(4) provides that EPA should take into account "the extent of independent verification or peer review" of scientific information, this language indicates that peer review of a study could provide sufficient assurance of its reliability even without additional verification.
TSCA further directs EPA to make available to the public, among other things, "a list of the studies considered by the Administrator in carrying out each such risk evaluation, along with the results of those studies" and "each designation of a chemical substance . . . along with an identification of the information, analysis, and basis used to make the designations." Id. § 2625(i). Again, the statute, despite calling out specific information to be made publicly available, does not state that the underlying data for these studies must be made publicly available. Thus, the rule is flatly inconsistent with TSCA.
Finally, even if it were not already clear from the above provisions that EPA cannot bar consideration of studies as provided in the Proposal, TSCA also states that EPA "shall take into consideration information relating to a chemical substance or mixture, including hazard and exposure information, under the conditions of use, that is reasonably available to the Administrator." Id. § 2625(k). EPA has defined "reasonably available" to mean "information that EPA possesses or can reasonably generate, obtain and synthesize for use . . . for prioritization and risk evaluation. Information that meets such terms is reasonably available information whether or not the information is CBI that is protected from public disclosure under 15 U.S.C. 2613." 40 C.F.R. § 702.3. Thus, if the studies covered by the rule are "reasonably available" to EPA, the agency must consider them, regardless of whether the raw data can be made public. EPA cannot create a double standard where studies withheld from the public as CBI must be considered but studies for which the underlying data cannot be made publicly available cannot be considered. The Proposal is unlawful under TSCA and cannot be promulgated. 
Comment [6340-1462]: Commenter (6362) writes that EPA's reference to section 10 under the TSCA also does not appear on-point. American Chemistry Council (ACC) believes EPA's authority to implement this rule is derived from TSCA Section 26(h), which speaks directly to scientific information and standards to which the Agency must adhere in the administration of its work under TSCA Sections 4, 5, and 6.
Comment [6343-2705]: Commenter (6365) writes that they are committed to meeting their obligations to operate in a responsible manner to avoid adversely impacting the environment and public health and safety. Both lead itself and the production of lead are extensively regulated by the EPA, including under the TSCA, CAA, CWA, and RCRA. As a result of these programs and other actions taken by industry, "[t]he United States have made tremendous progress in reducing lead concentrations in the outdoor air," with the national average concentrations dropping "more than 90 percent since 1980," and "EPA expects that most of the areas previously designated as nonattainment will have attained or are on track to attain by the end of 2016." 
In order to ensure that the regulatory standards that the ABR's members are subject to are supported by the best possible science, the ABR supports EPA's objective of strengthening transparency in regulatory science by ensuring that the data and models underlying regulatory science is publicly available in a manner sufficient for validation and analysis. Furthermore, EPA should establish clear criteria for data and model availability so that stakeholders can effectively determine whether the data and models underlying each regulatory proposal are available for review.
Comment [6353-2330]: Commenter (6375) notes that EPA solicits comment on whether alternative or additional regulatory or other policy vehicles are appropriate to establish and implement these policies, and whether further regulatory or other policy vehicles at the programmatic or statutory level would be appropriate as alternative or additional steps the agency may take to further the policies articulated in Section II. (pg. 18,771).
The regulatory language that EPA proposes provides general requirements that apply to the Agency's use of science under the many statutes that it implements. The generality of the proposed regulations may be a function of the diverse purposes of those statutes Specifically, some of these statutes focus on the introduction of new substances into the marketplace (e.g., FIFRA and TSCA). The scientific analyses required for these different programs and the sources of the scientific information underlying those analyses differ. The Agency should consider supplementing the general language that it has proposed with statutory specific regulations. A statutory-specific rulemaking would provide greater certainty going forward.
Comment [6424-2640]: Commenter (6446) writes that the proposed rule also deviates from accepted practices that EPA routinely uses, and is required to use, to evaluate science it relies on for regulatory decisions. EPA has long recognized that proprietary CBI submitted to it cannot be shared with the public, yet it routinely relies on that information, including scientific studies and other scientific information, as the foundation for important regulatory decisions. (See, e.g., CBI protection provisions in the TSCA at 15 U.S.C. § 2613, subd. (c). EPA also adheres to specific Congressional directives regarding the science it should consider for regulatory action, including directives to consider whether the scientific information is reasonable, clear, complete and whether it has been peer reviewed. (See, e.g., the TSCA at 15 U.S.C. § 2625, subds. (h) and (i), requiring EPA to consider several factors to evaluate science relating to toxic chemicals and requiring that decisions be based on "the weight of the scientific evidence."
Comment [6425-2641]: Commenter (6447) writes that the proposal is not authorized by statute and is contrary to Law.
EPA purports to rely on general statutory authority for the proposal, but no statutes authorize EPA to ignore science. On the contrary, as multiple U.S. Senators have noted, EPA's proposal would violate several statutes, including the TSCA, the SDWA, and the APA, which require the Agency to use the best available science, and all information presented to the agency during a rulemaking. The current proposal would do the opposite, requiring EPA to ignore some of the best available science, and to ignore certain information submitted during rulemaking. In particular, the proposal would make it difficult or impossible to use most epidemiological studies, where the source data are almost always comprised of sensitive personal information. Epidemiological studies are fundamental to EPA policymaking, and a critical source of information about the effects of various risk factors on human health. The Agency does not the option of suppressing this information, and courts have held that EPA "cannot ignore existing data."
In short, the proposed action would be illegal, and would not survive a legal challenge.
Comment [6833-3415]: Commenter (6855) writes that the requirements should apply specifically to risk evaluations performed under Section 6(b) of TSCA; to assessments (including certain retrospective assessments), models, criteria documents, and regulatory impact analyses that provide the basis for EPA's policies, procedures, guidance, and proposed and final regulatory decisions; and to individual party adjudications, enforcement activities, and permit proceedings.
Comment [6852-2770]: Commenter (6874) writes that a recent analysis tallied the costs to the U.S. economy of a set of endocrine-disrupting chemicals (EDCs) for which robust exposure, toxicity, and epidemiologic data exist. Teresa Attina and colleagues calculated that these EDCs cause annual losses of $340 billion, or 2% of the 2010 U.S. GDP, with the majority of that due to IQ point loss and intellectual disability from in-utero exposure to polybrominated diphenyl ethers (PBDEs). The total also includes costs associated with 86,000 cases of endometriosis attributable to the phthalate DEHP and 37,000 cases of uterine fibroids attributable to DDT exposures; uterine fibroids are both more common and more severe in African American women. The authors note that although the LSCA's update of the TSCA gives EPA the authority to limit production of chemical hazards and protect vulnerable populations, it does not provide the kind of funding necessary to adequately assess the safety of the many chemicals that are in use despite a lack of sufficient safety data. Given this combination of insufficient federal funding for testing and the substantial potential benefits of regulations that reduce EDC exposure, as well as reformed TSCA's requirement to make decisions based on the "weight of the scientific evidence," EPA should be using the best available science to guide regulation rather than restricting the pool of science that can inform its regulatory work.
Comment [6884-3029]: Commenter (6905) writes that states understand EPA's desire to be transparent with the public and we work hard to achieve this where possible. However, states are concerned that requiring all data used in scientific studies that underpin environmental and public health protection regulations be publicly available will lead to issues. For example, under the TSCA, EPA regularly relies on industry testing and data to make decisions on whether a chemical presents an unreasonable risk to human health or the environment. But some of this data is protected from public access under CBI claims. 
Comment [6884-3032]: Commenter (6905) notes that like EPA's mission statement, several of the federal statutes that provide authority for the agency's work call for use of the best science without regard for data transparency. For example, TSCA as amended under the LSCA requires the EPA Administrator to use "scientific information, technical procedures, measures, methods, protocols, methodologies, or models, employed in a manner consistent with the best available science." The statute states that this best available science should be clear and complete, but it does not require it to be publicly available. It appears that the proposed rule could reject some of the best available science and may result in a finding that the rule contradicts the federal statutes.
Comment [6885-2986]: Commenter (6906) writes that in effect, EPA's proposal could be interpreted in a manner that is overly restrictive. EPA might also interpret this proposal in a manner that screens science studies using narrower criteria than congress indicated for risk evaluations under TSCA, where EPA is required to use the "best available science." The concept of "best available science" also applies to EPA decision-making generally and in other statutory programs, as explained in E.O. 13563. TSCA § 26(h) (as recently amended by the LSCA) requires that "in carrying out sections 4,5, and 6, to the extent that the Administrator makes a decision based on science, the Administrator shall use scientific information, technical procedures, measures, methods, protocols methodologies, or models, employed in a manner consistent with the best available science" (italics added). The phrase "best available science" incorporates evaluation of similar supporting studies, objectivity, consistency with intended use of information, relevance, variability, and independent verification. Under this standard, EPA is not only required to evaluate the clarity of data, but also assumptions, methods, protocols, methodologies and/or models. In effect, the standard requires the broader set of considerations used in the scientific community when evaluating studies. [...]
American Coatings Association [the commenter] (ACA) recognizes that proposed 40 CFR §30.5 (requiring disclosure of dose response data and models) and §30.7 (requiring peer review and comment to EPA) are designed to give the public enough information to evaluate whether EPA has used the "best available science." The proposal and existing "best available science" standard are not necessarily mutually exclusive, but ACA suggest that EPA amend its proposal, so it more clearly incorporates consideration of factors that constitute "best available science." 
These changes would allow EPA to continue considering data from the EU and Canada during TSCA risk evaluations. Suggested changes also promote industry participation in TSCA risk evaluations. 
Comment [6893-2964]: Commenter (6914) writes that they have been integrally involved with both the passage and the implementation of the original 1976 TSCA and the 2016 LSCA, which amended the 1976 TSCA law. The TSCA law requires EPA to rely on the "best available science" and specifies that the agency "cannot ignore existing data." This proposal is contradictory in that it allows the agency to ignore some of the best available information that could help implement these laws. The proposal would limit exactly the kind of information Congress intended the agency to collect related to workplace chemical exposures and work-related disease.
Comment [6894-3607]: Commenter (6915) writes that the TSCA requires the Administrator, in decisions based on science, to "use scientific information, technical procedures, measures, methods, protocols, methodologies, or models, employed in a manner consistent with the best available science," and, in carrying out certain sections of the Act, to "take into consideration information relating to a chemical substance or mixture . . . that is reasonably available to [him or her]." § 26(h), (k), 15 U.S.C. § 2625(h), (k) (emphases added).
Comment [6901-4445]: Commenter (6922) notes that in the TSCA: "In carrying out sections 2603, 2604, and 2605 of this title, to the extent that the Administrator makes a decision based on science, the Administrator shall use scientific information, technical procedures, measures, methods, protocols, methodologies, or models, employed in a manner consistent with the best available science[.]
Comment [6903-1997]: Commenter (6924) writes that the rule is counter to mandates in the reformed TSCA to use the best available science and systematic reviews for chemical evaluations.
Reformed TSCA requires the use of the "best available science" for decision-making under the Act. In contrast, this proposed rule mandates EPA to ignore well-conducted, relevant studies simply because all the data are not publicly available and/ or may not conform to the rule's invalid assumptions about GLP/Guideline and dose-response modeling; this is inconsistent with the TSCA statute and the Agency's legal mandate.
Further, EPA's rule delineating risk evaluation procedures under TSCA mandates the use of systematic review methods. Systematic reviews consider the entire body of scientific evidence and provide the most comprehensive, accurate and transparent evaluations of chemicals. In a systematic review, the quality and strength of all relevant individual studies is considered to reach an overall conclusion about the body of evidence, without requirements for data re-analysis or public access to data. This proposed rule would prevent EPA from completing systematic reviews as its regulation mandates for chemical evaluations because it would prohibit the Agency from considering studies where the data were not publicly available and downgrade studies based on criteria not related to actual study quality, such as GLP.
Comment [6912-3758]: Commenter (6933) writes that for all of these reasons, and many more beside (including the blatant inconsistency with requirements of the CAA, TSCA and other statutes to use best available science), they ask that you withdraw this proposal. 
Comment [8750-1434]: Commenter (9227) writes that EPA's statutory authorities generally require the agency to consider all available data when undertaking significant rulemakings.
As just noted, EPA's statutory authorities mandate a variety of requirements for what scientific information EPA must consider in rulemaking. These statutes are discussed in detail, infra at Section I.B.3. To take one example that appears in numerous statutes, including TSCA, CAA, SDWA, and the ESA, Congress has often required agencies to act on the "best available science." For an agency to comply with this obligation, the agency must at least consider all available scientific information. "Best" means "of the most excellent, effective, or desirable type or quality." "Available" means "able to be used or obtained." And "science" means "the intellectual and practical activity encompassing the systematic study of the structure and behavior of the physical and natural world through observation and experiment." Assessing which science is "best" requires consideration of the overall quality of the science, and the public availability of underlying data is, at best, one of many aspects that should inform that assessment of overall quality.
An agency "cannot ignore available. . . information." Numerous courts have indicated that a plaintiff or petitioner can establish a violation of the "best available science" requirement by "point[ing] to any scientific evidence that the agency failed to consider." "The best available data requirement. . . prohibits [an agency] from disregarding available scientific evidence that is in some way better than the evidence [it] relies on.'" "An agency does. . . have an obligation to deal with newly acquired evidence in some reasonable fashion." EPA's proposal will result in EPA precluding itself from considering certain studies that are "available," thus violating the requirement that EPA rely on the best available science.
In addition, the requirement that agencies use "best available" science or information often means that the agency must act even if the available science or information is imperfect.
"Even if the available scientific and commercial data were quite inconclusive, [the agency] may -- indeed must -- still rely on it" when the agency has a duty to act. "[W]here the information is not readily available, we cannot insist on perfection." Just as the Courts have recognized that they cannot expect perfection, agencies cannot choose to ignore certain studies or sources of information based solely on whether the data is publicly available -- especially where the validity of those studies has been established using techniques that do not rely on public availability of underlying data.
EPA cannot reasonably elevate the interest in public availability of all underlying information above all other factors in assessing the "best available science." Textually, EPA's approach is unlawful.
Comment [8750-1438]: Commenter (9227) writes that the proposed rule violates EPA's statutory authorities.
Not only is there no authority for EPA's pan-statutory Proposal, but the Proposal would also violate explicit statutory commands. Though EPA admits that "[t]he best available science must serve as the foundation of EPA's regulatory actions," proposed section 30.5 would prohibit EPA from considering high quality and critically important scientific studies -- precisely that "best available science" -- when undertaking regulatory actions. Specifically, section 30.5 would prevent EPA from considering any scientific study for which the underlying "dose response data and models" are not "publicly available in a manner sufficient for independent validation." This would be true even if that scientific study constituted "information available to the Administrator" in a TSCA § 4(f) rulemaking. Accordingly, this proposed prohibition would contravene an array of statutes governing EPA's consideration of science when promulgating rules, such as requirements to consider the "best available science" when setting environmental protection standards. See, e.g., TSCA, 15 U.S.C. § 2625(h) ("[T]he Administrator shall use scientific information, technical procedures, measures, methods, protocols, methodologies, or models, employed in a manner consistent with the best available science."). And, by excluding science that meets these statutory criteria from supporting regulations to protect public health and welfare, the Proposal would frustrate Congress's policy in these statutes and frustrate EPA from achieving its fundamental mission.
Comment [8750-1447]: Commenter (9227) notes that EPA has previously stated in several different forums that a scientific study can be valid even if the underlying dose response data and models are not publicly available. For example, EPA recently explained in its own Plan to Increase Access to Results of EPA-Funded Scientific Research that even though "some research data cannot be made fully available to the public but instead may need to be made available in more limited ways," the lack of full public availability "does not affect the validity of the scientific conclusions from peer-reviewed research publications." Under the plan, EPA must make publications resulting from EPA-funded research publicly accessible on NIH's PubMed Central (PMC). The plan aims to "maximize access, by the general public and without charge, to digitally formatted data resulting from EPA funded research, while protecting confidentiality and personal privacy, recognizing proprietary interests, business confidential information and intellectual property rights, and preserving the balance between the relative benefits and costs of long-term preservation and access." The plan recognizes important exceptions for when "the research data cannot be released due to one or more constraints, such as requirements to protect confidentiality, personal privacy, proprietary interest, or property rights." It specifically declares: "The validity of scientific conclusions drawn from research publications or their associated research data, or EPA's ability to consider those conclusions and data in its actions, does not depend on compliance with this Plan."
Likewise, EPA's Science Policy Council explains in A Summary of General Assessment Factors for Evaluating the Quality of Scientific and Technical Information that EPA's determination as to the quality and reliability of a particular scientific study does not depend on one single factor (e.g., the public availability of underlying data), but instead turns on the agency's consideration of five general factors. Congress implicitly endorsed this approach by including a directive for EPA to use these same five factors in evaluating science under the TSCA Amendments passed in 2016, and just last year this Administration included these same factors in a recent regulation implementing TSCA. The factors comprise: (1) soundness; (2) applicability and utility; (3) clarity and completeness; (4) uncertainty and variability; and (5) evaluation and review. Of these, the only ones with any possible direct relevance to EPA's proposed approach are the third and fifth factors, but neither supports the elevation of public availability of data above all other considerations or the exclusion of studies with non-public data. The third factor, "clarity and completeness" requires EPA to consider "[t]he degree of clarity and completeness with which the data, assumptions, methods, quality assurance, sponsoring organizations and analyses employed to generate the information are documented." The fifth factor, "evaluation and review," requires EPA to consider "[t]he extent of independent verification, validation and peer review of the information or of the procedures, measures, methods or models." Even clear and complete "documentation" of the data used does not require that the data be made publicly available. Nor does factor five require either that a study's findings must have been replicated using the same data, or that the data must be available to allow for such replication. Moreover, these are only portions of two of five key factors to consider.As discussed at Section I. B.3. b) ii of the commenter's comment letter, even this Administration, in the context of promulgating regulations under TSCA, has adopted a regulatory definition of "best available science" expressly incorporating a multi-factor analysis, and that definition recognizes that public unavailability of data does not render a study incapable of being "best available science."
Comment [8750-1464]: Commenter (9227) writes that EPA's Proposal contravenes the TSCA.
i. TSCA expressly requires that EPA consider reasonably available information and EPA's proposal would preclude EPA from considering some reasonably available information.
When Congress amended TSCA through passage of the LSCA, Congress provided a number of detailed instructions on how EPA should consider scientific information with respect to chemical substances; EPA's proposal contradicts Congress's carefully crafted scheme. In particular, Congress included a provision specifically requiring that EPA consider all "reasonably available information." When making decisions about testing or the risk evaluation or regulation of new or existing chemicals, "the Administrator shall take into consideration information relating to a chemical substance or mixture, including hazard and exposure information, under the conditions of use, that is reasonably available to the Administrator." 15 U.S.C. § 2625(k) (emphases added). But under EPA's proposed rule, EPA would often be precluded from considering such reasonably available information if all the underlying data and models were not publicly available. See 83 Fed. Reg. at 18,769 n.3 (stating that proposal "would preclude [EPA] from using [non-public] data in future regulatory actions"). EPA's proposal violates the plain language of TSCA § 26(k), as well as Congress's clear purpose of ensuring that EPA consider all reasonably available information relating to a chemical when making a decision about the chemical.
Under its plain language, "available" means "able to be used or obtained; at someone's disposal." Congress chose this standard to ensure that EPA would make decisions based on all reasonably available information. S. Rep. No. 114-67 at 9 (June 18, 2015) ("The section ... requires EPA to consider reasonably available information about potential hazards and exposures of a chemical substance under the conditions of use when making decisions under TSCA.... The Committee intends that EPA systematically search for and identify relevant information that is available to inform safety assessments and determinations."); Oversight of the EPA's Progress in Implementing Inspector General and Government Accountability Office Recommendations: Hearing before the Subcomm. on Superfund, Waste Management, and Regulatory Oversight of the S. Comm. on Environment and Public Works, 114th Cong. at 63 (June 14, 2016) ("[F]or the EPA to properly evaluate and regulate toxic substances, it is essential that they have the most up-to-date chemical and toxicity data available."). Congress also selected this standard to avoid paralysis by analysis -- Congress wanted EPA to act on available information and not to postpone action waiting for new or perfect information to become available. See, e.g., 162 Cong. Rec. S3511, S3517 (daily ed. June 7, 2016) (referring to "information reasonably available to EPA" as "ensur[ing] that such considerations do not require additional information to be collected or developed"). "Congress recognized the need to use available studies, reports and recommendations for purposes of chemical assessments rather than creating them from whole cloth." Id. at S3522. And Congress intended for EPA to consider studies even when they had not undergone all possible forms of vetting. "[I]n instances where there were other studies and reports unavailable at the time of the [NAS] recommendations, EPA should take advantage of those studies and reports in order to ensure that the science used for chemical assessments is the best available and most current science." Id. at S3522. Congress intended for EPA to consider all reasonably available information, and EPA's proposal would thwart that clear purpose.
Notably, this Administration has adopted two regulations under the amended TSCA defining reasonably available information. These regulations generally provide that:
Reasonably available information means information that EPA possesses or can reasonably generate, obtain, and synthesize for use in risk evaluations, considering the deadlines specified in TSCA [for action]. Information that meets the terms of the preceding sentence is reasonably available information whether or not the information is CBI, that is protected from public disclosure under TSCA section 14.
40 C.F.R. § 702.33; see also 40 C.F.R. § 702.3 (similar definition for prioritization decisions). This bears no resemblance to the limitations put forward in the Proposal. Indeed, EPA has defined "reasonably available information" to include information EPA withholds as CBI under TSCA § 14. 15 U.S.C. § 2613. If the proposed rule forecloses EPA from considering information that cannot be fully disclosed, as it appears to do, then EPA cannot comply with both these regulations and the proposed rule.
EPA's proposal also violates other provisions of TSCA that expressly require EPA to act on "available information." For example, in preparing risk evaluations for existing chemicals, EPA "shall integrate and assess available information on hazards and exposures for the conditions of use of the chemical substance, including information that is relevant to specific risks of injury to health or the environment and information on potentially exposed or susceptible subpopulations identified as relevant by the Administrator." Under the proposed rule, EPA would not be able to integrate and assess available information where all underlying data has not been disclosed. Similarly, when developing regulations for existing chemicals, EPA "shall consider and publish a statement based on reasonably available information with respect to" a number of factors, including the effects of the chemical on health and the environment. But under the proposed rule, EPA cannot consider all reasonably available information when assessing those health and environmental effects.
Indeed, TSCA § 4(f) imposes a duty upon EPA to initiate regulation in response to any available information that meets certain substantive standards. However, if all the underlying information were not available, EPA's proposed rule would then foreclose EPA from considering that information during the resulting rulemaking. Congress would not have created a scheme where EPA must act in response to certain information but then cannot consider that information in taking action. Specifically, under TSCA § 4(f):
Upon the receipt of -- (1) any information required to be submitted under this Act, or (2) any other information available to the Administrator -- which indicates to the Administrator that there may be a reasonable basis to conclude that a chemical substance or mixture presents a significant risk of serious or widespread harm to human beings, the Administrator shall, ... initiate applicable action under section 5, 6, or 7 to prevent or reduce to a sufficient extent such risk or publish in the Federal Register a finding, made without consideration of costs or other non-risk factors, that such risk is not unreasonable.
Thus if "any ... information available" to EPA provides a reasonable basis to conclude that a chemical "presents a significant risk of serious or widespread harm to human beings," then EPA must initiate action to regulate the chemical. But under EPA's proposed rule, EPA would then be required to ignore the information triggering this duty when crafting the final regulation unless the source of the information fully disclosed all underlying data. That result clearly contradicts Congress's intent, which was to create a duty for EPA to react to any available information meeting the substantive standard of TSCA § 4(f).
In sum, Congress repeatedly directed EPA to consider all reasonably available information when making decisions under TSCA. The proposed rule would illegally preclude EPA from considering available information. The two cannot be reconciled, and the rule is unlawful.
ii. TSCA requires an agency to act on the "best available science," meaning that EPA must consider all available science and assess the quality of the science based on a variety of factors.
EPA's proposed blanket prohibition against basing a rulemaking on science for which underlying data or models are not publicly available would be particularly hard to reconcile with the "best available science" standard as articulated in TSCA, which clearly contemplates a case-by-case analysis in which EPA weighs a variety of factors when identifying the best available science. The relevant provision of TSCA requires that:
(h) Scientific standards. In carrying out sections 4, 5, and 6, to the extent that the Administrator makes a decision based on science, the Administrator shall use scientific information, technical procedures, measures, methods, protocols, methodologies, or models, employed in a manner consistent with the best available science, and shall consider as applicable -- 
(1) the extent to which the scientific information, technical procedures, measures, methods, protocols, methodologies, or models employed to generate the information are reasonable for and consistent with the intended use of the information;
(2) the extent to which the information is relevant for the Administrator's use in making a decision about a chemical substance or mixture; (3) the degree of clarity and completeness with which the data, assumptions, methods, quality assurance, and analyses employed to generate the information are documented;
(4) the extent to which the variability and uncertainty in the information, or in the procedures, measures, methods, protocols, methodologies, or models, are evaluated and characterized; and
(5) the extent of independent verification or peer review of the information or of the procedures, measures, methods, protocols, methodologies, or models.
Thus, Congress provided EPA with factors to guide its consideration of the "best available science," and Congress did not make the public disclosure of all underlying data a requirement for material to be the "best available science." Quite the opposite; Congress included aspects of disclosure and independent review as parts of factors to be considered when weighing scientific information. But these are just aspects of five different factors to be weighed "as applicable," and Congress clearly contemplated that EPA would sometimes rely on science that does not meet the proposed rule's requirement of full disclosure of all underlying data.
First, Congress directed EPA to consider these factors when weighing particular information; Congress specifically did not develop (or direct EPA to develop) bright-line criteria for eliminating information from consideration entirely. Thus, each factor includes the phrase "degree of" or "extent to which," without identifying any threshold that would be disqualifying. This shows that Congress intended these factors to help EPA assess the weight information should be given based on its relative scientific reliability, not to create minimum thresholds of reliability below which information must be ignored by EPA altogether. For EPA to insert a screen on top of these factors -- excluding information where the underlying data and models are not publicly available as required by the proposed rule -- contradicts Congress's unambiguous intent about how EPA should approach its assessment of the best available science.
Second, Congress made the "degree of clarity and completeness" with which the underlying data is documented to be part of one factor for EPA to consider in evaluating whether a particular study is the "best available science." But EPA must also consider "the degree of clarity and completeness" with which "assumptions, methods, quality assurance, and analyses" are documented as well. Thus, Congress contemplated that EPA would still rely on some studies that did not document completely all the underlying data, much less disclose all of that information.
Third, Congress made "the extent of independent verification or peer review of the information or of the procedures, measures, methods, protocols, methodologies, or models" another factor to be weighed when considering whether information is the "best available." Notably, Congress's choice of the disjunctive "or" reflects that "peer review" can be an adequate alternative to "independent verification," and Congress did not require that either "independent verification or peer review" be accomplished through public availability of data as required in the proposed rule. Moreover, Congress contemplated scenarios where EPA would give more weight to evidence even if the "information" had not undergone "independent verification or peer review" based on the extent to which the "procedures, measures, methods, protocols, methodologies, or models" had done so.
Fourth and most importantly, EPA cannot rationally elevate the interest in public disclosure of all underlying data above all the other factors that Congress expressly required EPA to consider in evaluating science. Congress required EPA to consider these five factors "as applicable" when weighing information, and Congress did not make full public availability of underlying data one of the factors, much less a decisive or absolute one. 
This administration recently adopted a regulatory definition of "best available science" for purposes of TSCA which expressly incorporated consideration of these five factors and was otherwise inspired by use of the term in the SDWA. EPA defined the phrase:
Best available science means science that is reliable and unbiased. Use of best available science involves the use of supporting studies conducted in accordance with sound and objective science practices, including, when available, peer reviewed science and supporting studies and data collected by accepted methods or best available methods (if the reliability of the method and the nature of the decision justifies use of the data). Additionally, EPA will consider as applicable:
[TSCA § 26(h)(1)(5) factors]
According to EPA in selecting this definition, "the Agency is remaining consistent with the current approach already used Agency-wide, while also acknowledging the specific standards under TSCA." Notably, this definition does not require public disclosure of all underlying data for science to be the "best available science," yet many studies that meet this definition of "best available science" would be excluded under EPA's proposed rule.
EPA's Proposal cannot be reconciled with EPA's existing definition of best available science, with decades of court and agency precedent, or with text of the statute. When a statute requires the agency to make a decision based on the "best available science," it would be unlawful to follow EPA's proposed rule.
iii. EPA's proposed rule also contradicts TSCA's requirement that decisions be made based on the weight of the scientific evidence.
TSCA § 26(i) requires EPA to make decisions regarding testing and regulating new and existing chemicals "based on the weight of the scientific evidence." If EPA excludes certain information, as proposed, then EPA will not be able to weigh the evidence as a whole.
Indeed, this administration recently adopted a regulation defining "weight of scientific evidence" to mean "a systematic review method ... that uses a pre-established protocol to comprehensively, objectively, transparently, and consistently, identify and evaluate each stream of evidence, including strengths, limitations, and relevance of each study and to integrate evidence as necessary and appropriate based upon strengths, limitations, and relevance." Systematic reviews consider the entire body of scientific evidence, but EPA's proposed rule would prevent EPA from conducting true systematic review because it would prohibit the Agency from considering studies where the data were not publicly available and it would eliminate studies based on criteria other than their "strengths, limitations, and relevance." If the proposed rule forecloses EPA from considering information that cannot be fully disclosed, as it appears to do, then EPA cannot comply with this regulation and the proposed rule.
In sum, EPA's proposed rule is inconsistent with TSCA's plain text. EPA should not adopt the proposed rule because it cannot be reconciled with the agency's duties under TSCA.
iv. Section 10 of TSCA does not authorize this proposal.
Nothing in TSCA § 10 authorizes EPA to exclude scientific information during rulemakings on any basis. Section 10 authorizes EPA to research and develop information for purposes of carrying out TSCA. Section 10 also authorizes EPA to develop systems to collect and disseminate information about chemical substances. But TSCA § 10 is silent regarding rulemaking or EPA's use of scientific information in rulemaking. It does not authorize EPA to exclude scientific information on any basis; if anything, TSCA § 10 reflects a congressional judgment that EPA should be prepared to use any and all "toxicological and other scientific information which could be useful to the Administrator in carrying out the purposes of this [Act]." 
Comment [8750-1818]: Commenter (9227) writes that EPA unlawfully failed to consult with other agencies as required by TSCA.
When promulgating the Proposal, EPA unlawfully failed to consult with other entities as required by TSCA. For example, consider the sole statutory authority EPA cites under TSCA -- § 10. To the extent EPA acts under TSCA § 10, TSCA § 10 repeatedly directs EPA to consult, cooperate, and/or coordinate with the Secretary of HHS, and sometimes other agencies as well. EPA has not identified any specific provision of TSCA § 10 that authorizes the proposed rule, and as noted above, no provision does. But if EPA acts under TSCA § 10, then EPA needs to comply with the requirements of whichever provision EPA considers relevant. Most of the provisions of TSCA § 10 expressly require that EPA consult, coordinate, or cooperate with, at least, the Secretary of HHS (section 10(a), 10(b)(2)(A), 10(b)(2)(B), 10(c), 10(d), 10(e), 10(g)). For example, the provision that mentions "research and development results" states that EPA shall act "in consultation with the Secretary of Health and Human Services and other heads of appropriate departments and agencies." EPA does not appear to have complied with any of the procedural requirements of TSCA § 10.
Comment [8750-1819]: Commenter (9227) writes that EPA has failed to consult with the Science Advisory Committee on Chemicals (SACC).
As discussed above, this proposed rule has severe implications for the implementation of TSCA. The SACC's purpose is "to provide independent advice and expert consultation, at the request of the Administrator, with respect to the scientific and technical aspects of issues relating to the implementation of this subchapter." This rulemaking specifically involves "the scientific and technical aspects of issues relating to the implementation of [this Act]," yet there is no indication that the Administrator has consulted with the committee. Congress specifically created this Committee to consult on these types of issues, and thus EPA is abusing its discretion to not consult with this Committee about a proposal that will so radically affect the scientific and technical aspects of issues relating to the implementation of TSCA.
Comment [8750-2711]: Commenter (9227) writes that EPA arbitrarily failed to consider the far-reaching implications this Proposal could have on inter-agency coordination and consultation given that other agencies normally rely on research potentially excluded by the Proposal.
Consultation provisions under the TSCA
2609(a)
Research, Development, collection, dissemination, and utilization of data
(a) Authority. The Administrator shall, in consultation and cooperation with the Secretary of HHS and with other heads of appropriate departments and agencies, conduct such research, development, and monitoring as is necessary to carry out the purposes of this Act. The Administrator may enter into contracts and may make grants for research, development, and monitoring under this subsection. Contracts may be entered into under this subsection without regard to sections 3648 and 3709 of the Revised Statutes
***
2609(b)(1), (2)
Research, development, collection, dissemination, and utilization of information: Information Systems
Administrator shall Consult and cooperate with Secretary of HHS and other heads of appropriate departments and agencies, to establish an efficient system for retrieval of toxicological and other scientific information which could be useful
***
2609(c)
Research, development, collection, dissemination, and utilization of information: Screening Techniques
Administrator shall coordinate with Assistant Secretary for HHS to develop screening techniques
***
2609(d)
Research, development, collection, dissemination, and utilization of information: Monitoring
Administrator shall, in consultation and cooperation with the Secretary of HHS, establish and be responsible for research aimed at the development, in cooperation with local, State, and Federal agencies, of monitoring techniques and instruments which may be used in the detection of toxic chemical substances and mixtures and which are reliable, economical, and capable of being implemented under a wide variety of conditions
***
2609(e)
Research, development, collection, dissemination, and utilization of information: Basic Research
The Administrator shall, in consultation and cooperation with the Secretary of HHS, establish research programs to develop the fundamental scientific basis of the screening and monitoring techniques described in subsections (c) and (d), the bounds of the reliability of such techniques, and the opportunities for their improvement.
***
2609(g)
Research, development, collection, dissemination, and utilization of information: Exchange of research and development results
The Administrator shall, in consultation with the Secretary of HHS and other heads of appropriate departments and agencies, establish and coordinate a system for exchange among Federal, State, and local authorities of research and development results respecting toxic chemical substances and mixtures, including a system to facilitate and promote the development of standard information format and analysis and consistent testing procedures.
***
2608(d)
Coordination
"Coordination. In administering this Act [15 USCS §§ 2601 et seq.], the Administrator shall consult and coordinate with the Secretary of Health and Human Services and the heads of any other appropriate Federal executive department or agency, any relevant independent regulatory agency, and any other appropriate instrumentality of the Federal Government for the purpose of achieving the maximum enforcement of this Act . . ."
***
2608(e)
Exposure Information
If the Administrator obtains information related to exposures or releases of a chemical substance or mixture that may be prevented or reduced under another Federal law, including a law not administered by the Administrator, the Administrator shall make such information available to the relevant Federal agency or office of the EPA.
***
2604(f)(5)
Manufacturing and Processing Notices: Protection Against Unreasonable Risks
Consult with Assistant Secretary of Labor prior to adopting any restriction of chemical substance for workplace exposures
***
2604(h)(2)(B)(ii)
Manufacturing and Processing Notices: Exemptions
Consult with AG of the Federal Trade Commission about exempting persons from information requirements.
Comment [9066-1929]: Commenter (6188) writes that the TSCA already expressly dictates the information that EPA must make publicly available, including a list of the studies EPA considers in carrying out risk evaluations and the results of those studies. 15 U.S.C. § 2625(j). Nowhere does the statute require that the data underlying those studies be made publicly available or limit the use of data or studies in making evaluations or regulatory decisions. Once again, EPA has declined to cite the provision within the TSCA that deals most directly with data disclosure and has not explained how other provisions of the TSCA unrelated to data disclosure support a rule imposing additional data disclosure requirements.
Comment [9066-2647]: Commenter (6188) writes that the proposed rule contradicts the requirements of federal environmental law, including cited authority for the rulemaking.
The Proposed Rule's requirements are at odds with statutory mandates contained in the very same environmental laws it points to as providing a basis for the rule. They also directly controvert authority it cites as support for the rulemaking.
The requirements of the Proposed Rule conflict with EPA's statutory obligations under federal environmental laws. The TSCA requires EPA to use the "best available science" when evaluating the testing and regulation of chemicals. 15 U.S.C. § 2625(h). Decisions are to be made using the "weight of the scientific evidence," and EPA is required to consider all information related to a chemical substance, including hazard and exposure information, "that is reasonably available to the Administrator." 15 U.S.C. §§ 2625(i), (k). Exclusion of relevant studies from consideration based not on the quality of the science but on the public availability of underlying data directly contradicts these mandates. EPA itself has defined "best available science" without reference to the public availability of data:
Use of best available science involves the use of supporting studies conducted in accordance with sound and objective scientific practices, including, when available, peer reviewed science and supporting studies and data collected by accepted methods or best available methods (if the reliability of the method and the nature of the decision justifies use of the data).
Simply put, the "best available science" is science that follows standards accepted by the scientific community, not science that EPA arbitrarily selects based upon the public availability of data -- a criterion irrelevant to scientists' standards for determining whether research represents the best science.
Specifically, scientists have observed that the best quality science oftentimes relies upon data that is not publicly available due to significant privacy considerations. It is for that very reason that courts have recognized EPA's need to rely upon studies based on publicly undisclosed underlying data when considering the best science. American Trucking Ass'ns, 283 F.3d at 372 (explaining that curtailing EPA's ability to rely on published studies would exclude "plainly relevant scientific information" from regulatory decision-making processes); see also Coalition of Battery Recyclers Ass'n, 604 F.3d at 623 (finding that EPA is entitled to rely on published study results as "raw data is often unavailable due to proprietary interests of a study's scientific investigators or confidentiality agreements with study participants"). Forcing EPA to ignore high quality science controverts EPA's statutory obligations and will impair EPA's ability to protect public health and the environment.
Comment [9078-3551]: Commenter (6194) writes that the proposal is inconsistent with EPA's statutory obligations and conflicts with other applicable federal requirements.
The Proposal would prevent EPA from relying on the best available information in discharging its duties. EPA is charged with developing regulations that provide societal benefits by reducing harm to human health and the environment from the presence of chemicals in air, soil, drinking water, food, and consumer products. The Proposal implicates research that is central to making such determinations; creating an obstacle to the consideration of such science is contrary to both primary statutory directives and requirements to conduct cost benefit analyses. Likewise, EPA fails to address how it could implement a regulation that is inconsistent with federal requirements and standards governing the use of science across agencies. 
A. Statutory Requirements to Develop Health Based Standards and Conduct Cost-Benefit Analyses EPA acknowledges that it must use the "best available science" in all of its regulatory actions.
This directive is embodied in multiple statutes implemented by EPA and is reflected in other environmental statutes, further illustrating Congress' conceptualization of "best" available science. Broadly speaking, there are three basic types of statutory requirements that the EPA and other agencies consider scientific information. First, there are requirements that the agency consider the "best available science." Second, there are requirements that the agency consider particular factors, including scientific factors. Third, there are requirements that the agency balance economic costs or technological feasibility against public health, environmental, or other benefits that must be scientifically assessed. The precise terminology regarding the required use of science may, as illustrated below, vary across statutes, but what these provisions have in common are requirements for EPA and other agencies to use scientific information that is "best," not simply adequate, and "available." 
Table 1: Examples of Statutory Directives regarding Use of Science by EPA and other Agencies
 TSCA [15 U.S.C. § 2625(h)]: "In carrying out sections 2603, 2604, and 2605 of this title, to the extent that the Administrator makes a decision based on science, the Administrator shall use scientific information, technical procedures, measures, methods, protocols, methodologies, or models, employed in a manner consistent with the best available science".
This statutory provision does not link the concept of the "best available science" to the availability of underlying raw data, nor can such a limitation be read into the statutes. Where Congress wanted to direct EPA's decision regarding what science to consider or how -- it did so. When Congress wanted to link "best available science" to data being publicly available -- it did so. Thus, if Congress had wanted to put a blanket limitation on EPA's consideration of science for which underlying data is not publicly available, it clearly knew how to do so -- and it chose not to.
As EPA has previously explained, the agency would be unable to fulfill statutory mandates if it could not consider studies that follow federal laws and ethical standards regarding the privacy of patient and individual data: 
      If EPA and other governmental agencies could not rely on published studies without conducting an independent analysis of the enormous volume of raw data underlying them, then much plainly relevant scientific information would become unavailable to EPA for use in setting standards to protect public health and the environment. . . . [S]uch data are often the property of scientific investigators and are often not readily available because of the proprietary interests of the investigators or because of arrangements made to maintain confidentiality regarding personal health status and lifestyle information of individuals included in such data. Without provisions of confidentiality, the possibility of conducting such studies could be severely compromised.
The Proposal would also cut EPA off from information needed to complete the cost-benefit analysis of regulations required under certain statutes and government-wide requirements.
Not only would the Proposal impede EPA's fulfillment of its statutory duties, but it would also be contrary to judicial precedent, which holds that "best available science" includes "all existing scientific evidence relevant to the [agency] decision [in question]." Ecology Ctr., Inc. v. U.S. Forest Serv., 451 F.3d 1183, 1194 n.4 (10th Cir. 2006) (quoting Heartwood, Inc. v. U.S. Forest Serv., 380 F.3d 428, 436 (8th Cir. 2004)). While agencies need not produce new evidence, they must seek out information and "`cannot ignore existing data.'" Id. (emphasis added). Even if an agency is anticipating the arrival of better evidence, it must act on the basis of the evidence it currently has. Many of the fundamental public health studies on which EPA has based key rules and standards under the statutes are studies for which the raw data was not or could not have been released. Nonetheless these studies have been examined and validated, as will future studies. Arbitrarily cutting off consideration of a category of science is inconsistent with EPA's statutory obligations.
Comment [9217-3561]: Commenter (9225) writes that the proposed rule is not authorized by any authority delegated to EPA by Congress and is contrary to a number of statutes under EPA's authority. This includes, but is not limited to, the CAA; CWA; TSCA; LCSA; SDWA; FIFRA; and more. 
Comment [9289-3793]: Commenter (PH20) writes that the rule is counter to the mandates in the reformed TSCA to use the best available science and systematic reviews for chemical evaluations.
Comment [9310-4993]: Commenter (PH84) writes that they played a key role in the reform of the TSCA which requires that EPA use the best available science in the review and management of toxic chemicals. As EPA begins to review the tens of thousands of chemicals already on the market, they are concerned that they be able to take into consideration all information that is reasonably available. 
Comment [9312-5004]: Commenter (PH46) writes that the rule is counter to mandates in the amended TSCA, to use the best available science and systematic reviews for chemical evaluations. Specifically, the proposed rule inappropriately codifies particular data analysis approach such as dose response modeling that should be made based on empirical considerations. 
The rule is counter to the mandates in the amended TSCA to use the best available science and systematic reviews for chemical evaluations. In contrast, this proposed rule will have EPA ignore well-conducted, relevant studies simply because all the data are not publicly available and/or may not conform to the rule's invalid assumptions about good laboratory practices and guidelines, studies, and dose response modeling. This is inconsistent with modern science and the TSCA statutory mandates. Further, EPA's risk evaluation framework rules under TSCA mandate the use of systematic review methods. Well conducted systematic reviews consider the entire body of scientific evidence and the quality and strength of all relevant individual studies are considered to reach the overall conclusion. 
Comment [6857-2945]: Commenter (6879) writes that EPA rules and regulations have been critical to the protection of wildlife, natural habitats, and the ecosystem services they provide to communities and the economy. This includes protection of endangered and other wildlife from pesticides and other toxic substances under the FIFRA and the TSCA.
These laws require EPA to use the "best available" science, or all "reasonably available" science and information. By excluding high-quality peer-reviewed and validly published science from the regulatory decision-making process, the proposed regulation directly contravenes the specific mandate of these statutes. And by deliberately excluding the best science available, the rule also contravenes the Congressional mandates for these statutes and undermines their public health and environmental protection purposes.
Comment [6163-1276]: Commenter (6133) writes that the proposal's peer review provision lacks any statutory basis, is vague and contrary to existing requirements for peer review. 
In addition to addressing how and whether the agency will consider science, the Proposal also contains a seemingly unrelated provision regarding agency peer review. The Proposal, in § 30.7, reads: 
What role does independent peer review in this section?
EPA shall conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein. 
Because transparency in regulatory science includes addressing issues associated with assumptions used in models, EPA shall ask peer reviewers to articulate the strengths and weaknesses of EPA's justification for the assumptions applied and the implications of those assumptions for the results. 
83 Fed. Reg. at 18,774.
There is no statutory authority for EPA to "conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein." The federal statutes that EPA lists as putative authority for the Proposal provide no authority for proposed § 30.7. See TSCA, as amended, section 10, 15 U.S.C. 2609, and the APA). The claimed authorities that EPA lists do not mention peer review or even allude to the concept. Id. Neither the Proposal nor accompanying docket materials identify any provision of any federal statute that authorizes EPA to promulgate proposed § 30.7. Our own research revealed no provision of any federal statute that authorizes EPA to promulgate proposed § 30.7. 
When Congress writes federal statutes, Congress knows how to create legal authority for peer review, who shall conduct that peer review, what role, if any, that EPA or other parties will play, and how that peer review may be conducted. None of the provision in the law authorize EPA's Proposal in § 30.7. Instead, the statutes require that EPA use peer-reviewed science, regardless of whether it would meet EPA's definition of the term in § 30.7. And in cases where the law requires EPA to conduct the review, the statutes often spell out specifically how that should happen. 
Statute Provisions on Peer Review 
TSCA
§ 2625. Administration 
(h) Scientific standards In carrying out sections 2603, 2604, and 2605 of this title, to the extent that the Administrator makes a decision based on science, the Administrator shall use scientific information, technical procedures, measures, methods, protocols, methodologies, or models, employed in a manner consistent with the best available science, and shall consider as applicable-- (1) the extent to which the scientific information, technical procedures, measures, methods, protocols, methodologies, or models employed to generate the information are reasonable for and consistent with the intended use of the information; (2) the extent to which the information is relevant for the Administrator's use in making a decision about a chemical substance or mixture; (3) the degree of clarity and completeness with which the data, assumptions, methods, quality assurance, and analyses employed to generate the information are documented; (4) the extent to which the variability and uncertainty in the information, or in the procedures, measures, methods, protocols, methodologies, or models, are evaluated and characterized; and (5) the extent of independent verification or peer review of the information or of the procedures, measures, methods, protocols, methodologies, or models. 
15 U.S.C. § 2625 (emphasis added).
§ 2617. Preemption 
(b) New statutes, criminal penalties, or administrative actions creating prohibitions or other restrictions (1) In general Except as provided in subsections (c), (d), (e), (f), and (g), beginning on the date on which the Administrator defines the scope of a risk evaluation for a chemical substance under section 2605(b)(4)(D) of this title and ending on the date on which the deadline established pursuant to section 2605(b)(4)(G) of this title for completion of the risk evaluation expires, or on the date on which the Administrator publishes the risk evaluation under section 2605(b)(4)(C) of this title, whichever is earlier, no State or political subdivision of a State may establish a statute, criminal penalty, or administrative action prohibiting or otherwise restricting the manufacture, processing, distribution in commerce, or use of such chemical substance that is a high-priority substance designated under section 2605(b)(1)(B)(i) of this title.
(f) Waivers (2) Required exemptions Upon application of a State or political subdivision of a State, the Administrator shall exempt from subsection (b) a statute or administrative action of a State or political subdivision of a State that relates to the effects of exposure to a chemical substance under the conditions of use if the Administrator determines that-- (A)(i) compliance with the proposed requirement of the State or political subdivision of the State would not unduly burden interstate commerce in the manufacture, processing, distribution in commerce, or use of a chemical substance; (ii) compliance with the proposed requirement of the State or political subdivision of the State would not cause a violation of any applicable Federal law, rule, or order; and (iii) the State or political subdivision of the State has a concern about the chemical substance or use of the chemical substance based in peer-reviewed science; or (B) no later than the date that is 18 months after the date on which the Administrator has initiated the prioritization process for a chemical substance under the rule promulgated pursuant to section 2605(b)(1)(A) of this title, or the date on which the Administrator publishes the scope of the risk evaluation for a chemical substance under section 2605(b)(4)(D) of this title, whichever is sooner, the State or political subdivision of the State has enacted a statute or proposed or finalized an administrative action intended to prohibit or otherwise restrict the manufacture, processing, distribution in commerce, or use of the chemical substance. 
15 U.S.C. § 2617 (emphasis added).
§ 2605. Prioritization, risk evaluation, and regulation of chemical substances and mixtures 
(b) Risk evaluations (E) Metals and metal compounds In identifying priorities for risk evaluation and conducting risk evaluations of metals and metal compounds, the Administrator shall use the Framework for Metals Risk Assessment of the Office of the Science Advisor, Risk Assessment Forum, and dated March 2007, or a successor document that addresses metals risk assessment and is peer reviewed by the SAB. 
15 U.S.C. § 2605 (emphasis added).
Under longstanding federal case law, when Congress authorizes an approach in one section of a statute using specific language but does not do the same in other sections of a statute, courts presume that Congress acted purposefully and did not mean to address or authorize that approach in those other statutory sections. See, e.g., Dean v. United States, 556 U.S. 568 (2009) ("It is generally presumed that Congress acts intentionally when including particular language in one section of a statute but not in another." (citing Russello v. United States, 464 U.S. 16, 23 (1983)). Not only are there no implied grants of authority to an agency in the other statutory sections, but the congressional decisions to authorize the approach elsewhere in the statute give even greater force to the conclusion that the agency has not been given authority where Congress did not use the same or similar authorizing language. 
The EPA approach proposed in § 30.7 is even more unlawful than would be the case, independently, under this case law. Proposed § 30.7 says "EPA shall conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein." 83 Fed. Reg. at 18,774/2 (emphasis added). EPA proposes to bind itself ("shall") to conduct a particular form of peer review on "all pivotal regulatory science" based on an unenforceable, non-binding OMB bulletin that has never been the subject of notice and comment rulemaking and that itself is not authorized by any federal law. To the contrary, treating the content of the unenforceable bulletin as a binding regulation would itself violate the federal statutes that EPA implements, because those statutes do not codify the bulletin, and EPA would be unlawfully codifying a mere policy preference. This EPA may not do. 
This Proposal is unlawful, arbitrary, and capricious, and an abuse of EPA discretion. To reiterate, the Proposal identifies no statutory authority for EPA to conduct independent peer review for any, much less all, "pivotal regulatory science" consistent with the dense content (and exceptions) of the OMB bulletin. The Proposal identifies no statutory authority for EPA to bind itself, and future administrations, to conduct peer review only in this fashion, unless and until future notice-and-comment rulemaking is undertaken. The Proposal identifies no suggestion in statutory language or legislative history that Congress intended EPA to conduct binding peer review consistent with this OMB bulletin, notwithstanding that Congress has known about this bulletin since 2005. EPA has simply made up proposed § 30.7 -- with its the link to the OMB bulletin, and the putative authority for the Proposal -- out of whole cloth. This EPA may not do. 
1.3.4.2 TSCA (SNPRM Comments)
Comment [48-1959]: Commenter (12727) notes that several environmental statutes require EPA to ground its decision-making in scientific evidence. The TSCA requires the Administrator to "make decisions . . . based on the weight of the scientific evidence," 15 U.S.C. § 2625(i)."
The originally proposed regulatory language (§§ 30.3, 30.5) violated these statutory commands by preventing EPA from relying on a study as "pivotal regulatory science . . . used to justify significant regulatory decisions" if dose-response data or models underlying the study were not publicly available, without regard to whether the study had been validated by other means. The SNPRM exacerbates these violations by expanding the proposed regulation's scope (§§ 30.3, 30.5) to not just pivotal regulatory science used to justify significant regulatory decisions, but also "pivotal science supporting influential scientific information."
Comment [48-2183]: Commenter (12727) writes that just as EPA is required to consider all available science when making its regulatory decisions, governing environmental and public health statutes likewise require EPA to consider all available science when releasing ISI, which often forms the basis for the agency's regulatory decisions.
For example:
 The TSCA directs that, in implementing the Act's testing requirements, manufacturing and processing notice requirements, and requirements for the prioritization, risk evaluation, and regulation of chemical substances and mixtures, EPA must make its decisions "based on the weight of the scientific evidence." Likewise, in carrying out EPA's responsibilities under TSCA, "the Administrator shall use scientific information . . . employed in a manner consistent with the best available science." Numerous other TSCA provisions emphasize EPA's obligation to consider all reasonably available scientific information when implementing TSCA's requirements.
A rule that prohibits EPA from considering (or that downgrades) valid, high quality scientific studies when generating ISI such as the products identified above would contravene the above- noted statutory directives regarding EPA's obligation to consider all available science.
Comment [53-2007]: Commenter (11360) writes that Congress intentionally and wisely embedded peer-reviewed research in the foundation of the CAA, including requiring regular reviews of the science, explicitly recognizing that EPA needs the most current, peer-reviewed data to protect the public health. These expectations also are reflected in other public health laws that support the EPA's mission, including the TSCA. Limiting scientific evidence does not strengthen regulations or policies; rather, it is paramount that the full suite of relevant science, vetted through peer-review, inform the landscape of decision-making. Excluding relevant studies simply because they do not meet unnecessarily rigid transparency standards will adversely affect decision-making processes.
Comment [81-2191]: Commenter (12715) writes that the proposed rule conflicts with fundamental statutory requirements to use the best available science and citation to "housekeeping" authority cannot cure that defect.
Whether EPA is arbitrarily barring consideration of scientific studies or information with nonpublic underlying data or data that are not able to be independently validated, or arbitrarily ascribing less weight to such studies or information, neither approach constitutes "housekeeping," and neither comports with statutory requirements that EPA use the best available science. To reiterate from the 2018 Comments, EPA's obligation with respect to the use of scientific information is clear: it must base its decisions on such criteria as the latest scientific knowledge, the best available, peer-reviewed science, and/or generally accepted scientific principles or laboratory tests. To cite just a few examples, in performing its duties, EPA must rely on the "best available science," TSCA, 15 U.S.C. § 2625(h). EPA has repeatedly emphasized the importance of this fundamental precept, including in its 2018-2022 strategic plan, which states that one of the agency's priorities is to "identify, assess, conduct, and apply the best available science to address current and future environmental hazards."
These statutory requirements apply to all aspects of EPA's decision-making, including its scientific evaluations, adoption of regulatory standards, and development of policies. They do not permit what EPA proposes here: to either exclude or give less weight to scientific studies or information based on criteria -- availability of the underlying data and ability to independently validate underlying data -- that are not determinative of whether the studies or information constitute the best available science. Agencies may not adopt or implement regulations that conflict with the statutes under which they are promulgated, and an agency's interpretation of those statutes must always at least be reasonable. See Chevron, U.S.A., Inc. v. Nat. Res. Def. Council, Inc., 467 U.S. 837, 842-44 (1984). Furthermore, neither 5 U.S.C. § 301 nor any other purported housekeeping authority allows EPA to negate these statutes' fundamental, substantive, and common-sense commands. And even assuming EPA has some inherent housekeeping authority, it cannot rely on a general grant of rulemaking authority to promulgate regulations or develop policies that are otherwise inconsistent with more specific statutory directives. Glob. Van Lines, Inc. v. Interstate Commerce Comm'n, 714 F.2d 1290, 1293-97 (5th Cir. 1983).
Finally, EPA acknowledges that the statutes it administers would control in the event of a conflict between the proposed rule and those statutes, see 85 Fed. Reg. at 15,398, but that caveat cannot save the proposed rule. First, the core principle of the proposed rule is facially invalid, i.e., there is no circumstance in which it would be acceptable for EPA to exclude or give less weight to relevant, probative scientific studies, models, or other information that have been validated through peer review, on the sole basis that the underlying data are not publicly available or are not able to be independently validated. EPA cannot save an invalid rule by including what amounts to a waiver procedure, because the "essence of waiver is the assumed validity of the general rule." Alltel Corp. v. Fed. Commc'n Comm'n, 838 F.2d 551, 561 (D.C. Cir. 1988). Second, the agency's essential duty in rulemaking is to exercise its subject matter expertise to enact rules that implement and further the purposes of the statutes Congress has assigned it to administer. Chevron, 467 U.S. at 842-44. This does not include proposing rules without first determining the source of legal authority but instead asking commenters to supply that information. Third, the agency must propose its rules with specificity and be clear as to the programs, regulations, and information to which they will apply. 5 U.S.C. § 553(b)(3); Home Box Office, Inc., supra, 567 F.2d at 35-36.
Comment [89-2198]: Commenter (11403) writes that the proposed rule is not based on any valid statutory authority and is arbitrary and capricious.
a. The environmental statutory provisions cited by EPA do not authorize the proposed rule
EPA's 2018 version of this rule cited a hodgepodge of statutory authority, none of it valid. EPA at least corrected the obvious typographical errors from the previous version. However, those edits still do not grant EPA the authority to restrict science from all regulatory decision-making processes within the agency. EPA is attempting to use general grants of authority to create a completely novel approach to assessing science. However, as the Supreme Court has stated, "Congress . . . does not alter the fundamental details of a regulatory scheme in vague terms or ancillary provisions -- it does not, one might say, hide elephants in mouseholes." EPA does not have the authority to alter the fundamental details of all of its regulatory schemes based on such vague provisions of law.
Section 10 of the TSCA grants the Administrator authority on research, development, collection, dissemination, and utilization of data. There is no mention of any type of research or data that warrants exclusion in order to carry out the purposes of the statute.
In short, the statutory provision cited by EPA does not provide authority for the proposed rule. The proposed rule fails at step one under Chevron v. Natural Resources Defense Council since it is unreasonable to believe that any statutory general grant of authority or any mention of EPA's oversight of other science or research programs would enable EPA to conduct rulemaking in a way that is altogether inconsistent with the rule of law. None of the cited provisions provide the authority sought by EPA, and as such the proposed rule should be withdrawn.
Comment [93-2086]: Commenter (11395) notes that TSCA specifies that "(h)azard information related to potential health and environmental hazards of the chemical substance will be reviewed in a manner consistent with best available science and weight of scientific evidence" (40 CFR 702.41(d)(2)). The TSCA definition of "best available science" goes on to list five criteria that should be considered, as applicable, one of which is "(t)he extent of independent verification or peer review of the information or of the procedures, measures, methods, protocols, methodologies or models." While this definition acknowledges the merit of independent verification, that criterion is only one of several listed and peer reviews is identified as another method of review of data quality. This is inconsistent with the SNPRM, which stipulates that studies for which data cannot be made available for reanalysis will not be considered or will be considered as less valid, even when the validity of those studies is supported by other avenues, including peer review.
Comment [97-2148]: Commenter (11479) writes that the proposed rule, as modified in the supplemental notice, would clearly interfere with the agency's use of the best available science. For example, the policy violates laws like the TSCA and the SDWA because of the statutory requirement to use the "best available science." That requirement does not make exception for the agency to effectively ignore certain scientific information of high quality simply because the unanalyzed data cannot be made public for reasons of privacy, confidentiality, or intellectual property. According to judicial precedent, "best available science" is defined as "all existing scientific evidence relevant to the decision" and a body of evidence that "cannot ignore existing data." In addition to the "best available science" requirements, TSCA also requires EPA to take into consideration information "that is reasonably available to the Administrator." EPA's own regulations define that term as "information that EPA possesses or can reasonably generate, obtain, and synthesize . . . , whether or not that information is confidential business information, [sic] that is protected from public disclosure..." Not only would this proposed rule conflict with TSCA's mandate to consider the best available science, it would challenge the method by which EPA can review the weight of the scientific evidence, which in rulemaking for TSCA is defined by a systematic review method. These reviews consider the entire body of scientific evidence, so the exclusion of certain studies based on whether the raw data and models are publicly available would render these reviews, as mandated, impossible.
In fact, virtually all of EPA's statutory mandates require the use of the best available scientific evidence. Congress did not state in any of these laws that the agency shall only consider scientific evidence derived from studies where the underlying unanalyzed data is publicly available, nor that the evidence considered should be solely at the Administrator's discretion.
Comment [101-2085]: Commenter (11482) writes that the proposed rule likely violates federal laws that require the EPA to use the best scientific information available to it. EPA has a duty to use the best scientific information available under several laws. For example, the TSCA requires EPA to operate "in a manner consistent with the best available science" and make regulatory decisions based on the weight of the scientific evidence. Numerous court cases have weighed in on the best available science standard in the context of numerous different federal laws. For example, courts have found that the agency must use the best science that is available to it at the time of the decision, not the best that might be available to it at some time in the future.
Importantly, the critical actor under the best available science standard is the agency  --  the standard requires the use of the best science that is available to the agency, not necessarily available to the public. Excluding science that is available to the agency but not available to the public would thus be a violation of "best available science" provisions.
Under federal law, the agency must find a way to continue using the best available scientific data and models available to it, regardless of whether those data and models are publicly available. Excluding science whose raw data are not in the public domain runs counter to the "best available science" doctrine and is thus illegal.
Comment [125-2201]: Commenter (11495) writes that EPA's SNPRM is at odds with provisions of multiple environmental statutes which require the Agency to consider certain high-quality scientific information. The TSCA requires the EPA Administrator to ensure that evaluation and regulation of toxic substances is "consistent with the best available science" and to use all scientific information that is "reasonably available" when making related decisions. 
Each of these provisions directs EPA to incorporate a certain quality of scientific information into its rulemaking decisions made pursuant to the statute. In particular, these requirements are designed to ensure the scientific validity of the information upon which EPA bases its actions. Mandating a high-quality scientific basis for regulatory actions is necessary to guarantee the best possible protections for public health and welfare and to minimize partisan influence in scientific decision-making.
The SNPRM, however, would impermissibly prohibit the agency from considering of relevant and high-quality scientific studies whenever the data underlying the study are not publicly available -- in direct contravention of the statutory directives that mandate, for example, consideration of the "best available science." None of the provisions requires, or even suggests, that EPA only consider information that is publicly available. It strains the plain meaning of words like "best," "latest," and even "appropriate" to suggest that they permit the unscientific requirement that all data be publicly available.
EPA "cannot ignore available . . . information." Courts have held that "best available data" requirements "prohibit[] [an agency] from disregarding available scientific evidence that is in some way better than the evidence [it] relies on." Furthermore, a plaintiff may establish a violation of a "best available science" requirement by "point[ing] to any scientific evidence that the agency failed to consider." The SNPRM would undoubtedly preclude EPA from considering certain studies and scientific information that is "better" or the "best available," simply because the study's underlying data is not publicly available, thereby failing to satisfy Congressional directives and undermining the quality of regulatory decisions.
Likewise, EPA cannot avoid the statutory directives to consider high-quality scientific information simply by assigning a lesser weight to such information when the underlying data are not publicly available, as the Agency alternatively proposes. The SNPRM's alternative proposal is designed to have largely the same pernicious effects as flatly prohibiting consideration of studies lacking publicly available data. Just as with the primary proposal, EPA's alternative proposal would limit the agency's consideration of the best available science in making important regulatory decisions, and is, therefore, arbitrary, and capricious.
The effect of the Proposed Rule is inconsistent with EPA's view that the SNPRM is "intended to be consistent with the statutes that EPA administers and EPA plans to implement this procedural rulemaking in accordance with all applicable statutory and regulatory requirements." EPA attempts to resolve this inconsistency by stating that "in the event the procedures outlined in the proposed rulemaking conflict with the statutes that EPA administers, or their implementing regulations, the statutes and regulations will control." However, as discussed below, EPA lacks the statutory authority to issue a rule this broad, even when it claims that it will avoid statutory conflicts. 
Comment [133-1187]: Commenter (11894) notes that TSCA §5 requires EPA to review the likelihood that a new chemical will present an unreasonable risk to human health or the environment before it can be marketed. At present, much of the information that EPA reviews rests on CBI or otherwise would not meet the requirements of the proposal. To fill some of these gaps, EPA relies on models that likewise would often not meet those requirements.
Once again, EPA has failed to address how it can refuse to apply these new requirements to data and models used to support PMNs given express direction from Congress to the agency to consider data quality. Indeed, yet again, EPA has not addressed the issue of applicability to this program at all or addressed the impacts on the public health or the environment if it did. Nor, once again, has EPA explored the resources that would be needed to respond.
Comment [151-1348]: Commenter (12405) discusses Parkinson's disease research in TSCA determinations.
The TSCA is EPA's primary authority for regulating non-pesticide chemicals. Under TSCA, EPA can secure information on new and existing chemicals and regulate chemicals it determines pose an unreasonable risk to public health or the environment. All studies used would be subject to the proposed rule.
In late 2016, the EPA moved to ban toxic chemical TCE due to health risks, including a risk of Parkinson's disease, though this action is still pending. The original recommendation was based on hundreds of studies, many of which would not be considered under the proposed rule.
In these relatively small studies, a distinctive characteristic  --  people who worked together and twins  --  respectively, combined with the most basic additional medical information could render the participants identifiable. Both TCE studies are highly cited, and scientists have replicated the findings. To exclude evidence that TCE exposure is a risk factor for Parkinson's disease is illogical and does not serve the best health interests of the American public.
Furthermore, the proposed rule does not comply with the letter of TSCA. TSCA and other statutes administered by EPA requires the Agency use the "best available science" and none require the Agency to access to raw data. TSCA additionally requires that EPA consider all information that is reasonably available to the administrator. As drafted, the proposed rule violates these statutes because it would force the Agency to ignore some of the best information available.
Comment [158-1398]: Commenter (12423) writes that EPA has already developed and employed the systematic review process to screen and prioritize studies used in TSCA risk assessments. Like this proposed rule, that process discounts or excludes studies for which underlying data are not publicly available. Consequently, whether intended or not, many studies providing tribally relevant data are likely to be excluded. Like in the current rule, reporting issues are conflated with study quality. The process is inconsistent with best practices in systematic review, is unscientific and arbitrary, and open to bias.
As the EPA tribal partnership group that works on TSCA related issues, the National Tribal Toxics Council (NTTC) would like to know how this rule would and could affect risk evaluations and use limits done under TSCA authority. Together with the systematic review process, this rule would multiply bias against studies with relevant tribal data and increase the likelihood that studies with relevant tribal information may not be considered. Much of tribal data is published in various reports and grey literature that is already discounted via systematic review. With generally fewer studies and data available that are relevant for tribes and the myriad unique exposures to begin with, how might excluding valid studies impact the agency's ability to estimate risk to tribes?
NTTC requested a briefing from EPA on these points on 04/20/2020 and the request was not met. We are left to assume that this rule will apply to TSCA risk evaluations and other TSCA activities and, if so, this rule will be extremely detrimental to ongoing efforts by the Office of Pollution Prevention and Toxics (OPPT) and the Office of Research and Development (ORD) to represent tribes in risk assessment, as per TSCA mandate to consider potentially exposed and susceptible subpopulations.
Conclusion 3: The rule will significantly impact EPA's capacity to carry out the TSCA protections for potentially exposed and susceptible subpopulations, as mandated by Congress in its 2016 bipartisan amendment.
Comment [160-1456]: Commenter (1431) remains concerned that in its desire to ensure that the data underlying its significant regulatory actions and ISI are publicly available, EPA may exclude information that is most relevant to its assessment of risks to human health and the environment. In particular, excluding information that cannot be made publicly available would slow the development and implementation of alternative test methods and strategies to replace animal testing. Such approaches can provide information of equivalent or better scientific quality and relevance for risk assessment, and their development and implementation are mandated by the TSCA. This potential conflict between public accessibility and continued progress in regulatory science has been recognized by EPA's OPPT. In its Strategic Plan to Promote the Development and Implementation of Alternative Test Methods Within the TSCA Program, OPPT states that public access to the datasets used to build models that predict human outcomes may not be possible in some cases, because OPPT uses CBI to update and refine its models, and these activities are critical to its mission of evaluating chemicals under TSCA. Under the proposed rule and SNPRM, OPPT could be limited to using only publicly accessible information and open source models. As Robert DeWoskin, etioLogic LLC, notes in his comments on the proposed rule, in some cases, results from commercially available models can be used without the need to release proprietary code, and requiring commercial model developers to release this intellectual property would likely restrict their ability to continue developing and supporting their models. In addition, sponsors of new industrial chemicals and agrochemicals sometimes submit chemical and toxicological information derived from proprietary alternatives in lieu of certain animal tests. Under the proposed rule and SNPRM, EPA may be unable to consider this information, resulting in animal testing that would, in the case of industrial chemicals at least, be conducted in violation of TSCA provisions to use available nonanimal test methods and strategies. Finally, as numerous stakeholders have commented, the proposed rule could force EPA to exclude from its consideration epidemiologic studies that use confidential health data. Such studies are frequently the most relevant to protecting public health as they assess outcomes in humans following real world exposures. The exclusion of this information may lead EPA to increase its reliance on information from less relevant animal toxicity studies. 
Comment [172-1551]: Commenter (12465) writes that while the supplemental proposal notes that "in the event the procedures outlined in the proposed rulemaking conflict with the statutes that EPA administers, or their implementing regulations, the statutes and regulations will control," therein lies significant ambiguity as to the applicability and scope of regulation designed to underlie the basis of EPA regulatory science decisions. The details on how the Agency would navigate these potential regulatory conflicts should be considered and advanced concurrently with the current proposal to effect maximum stakeholder input. For example, EPA needs to fully understand the potential vulnerability this transparency rule would create for regulatory decisions made under the TSCA and the FIFRA that rely on CBI as well as the extent to which models relied on by both programs comply with the rule as proposed. It is possible that a significant number of EPA decisions under FIFRA and TSCA could be successfully challenged if this rule were finalized as proposed.
Comment [187-1709]: Commenter (12720) notes that EPA cites 15 U.S.C. § 2609 under the TSCA as support for this rule. But that section states only that the "Administrator shall, in consultation and cooperation with the Secretary of Health and Human Services and with other heads of appropriate departments and agencies, conduct such research, development, and monitoring as is necessary to carry out the purposes of this chapter." It does not allow EPA to limit the type of data considered in regulatory decisions, nor does it draw a distinction between dose-response data and other types of data. TSCA does not support the proposed rule. The absence of any such congressional distinction or restriction makes it all the more obvious that EPA invented and added the distinction and restrictions in the Proposal and Supplemental Proposal as a matter of its own policy preferences, contrary to the Act. This EPA may not do.
Comment [198-2209]: Commenter (13549) writes that as a baseline, whether working on "significant regulatory actions," ISI, or anything requiring the consideration of science, EPA must base its decisions on the best available science. Anything less is nonsensical, arbitrary, and in violation of the environmental laws EPA is charged with implementing. The SNPRM does not maintain this baseline, but instead provides for ways EPA would exclude and lessen consideration of the best available science. Therefore, the SNPRM, along with the proposed rule, must be withdrawn. See, e.g., 15 U.S.C. § 2625(h) (TSCA provision requiring the use of best available science); see also 83 Fed. Reg. at 18,769 ("The best available science must serve as the foundation of EPA's regulatory actions.").
Comment [199-2212]: Commenter (12403) writes that the proposal is also inconsistent with provisions in the TSCA.
1.3.5 Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA)/ Resource Conservation and Recovery Act (RCRA) (NPRM and SNPRM Comments).
Comment: Sections 1.3.5.1 and 1.3.5.2 of this Chapter present comments received on the NPRM (Section 1.3.5.1) and SNPRM (Section 1.3.5.2) related to the EPA's legal authority for the rule under the CERCLA and RCRA. The EPA provides the following general response to these comments:
Response: EPA appreciates your comments. As discussed in Section I.C of the final rule preamble and in this RTC document in Section 1.2, EPA has decided it is most appropriate to rely on its authority to promulgate housekeeping regulations, rather than any of the substantive environmental statutes cited in the 2018 proposal or the 2020 supplemental proposal as legal authority for the rule. EPA intends to promulgate future science transparency rules under its substantive environmental statutes. Additionally, the final rule states in § 30.3 that "[i]n the event the procedures outlined in this rule conflict with statutes that EPA administers, or their implementing regulations, the statutes and regulations will control."
1.3.5.1 CERCLA/RCRA (NPRM Comments)
1.3.5.1.1 CERCLA
Comment [6112-1554]: Commenter (6137) provides that EPA also cites to provisions under the CERCLA as authority for its Proposed Rule. However, upon examination, these provisions likewise provide no legal support for the Proposal.
The first provision upon which EPA relies  -  Section 115  -  is inapposite. It merely sets out goal dates for EPA to begin assessment and remediation of facilities on the NPL and is entirely irrelevant to the issues of the Proposed Rule. See 42 U.S.C. § 9616. While the second provision upon which EPA relies  -  Section 311  -  is at least relevant to the issues of the Proposed Rule, it nonetheless conflicts with the proposition that the Proposed Rule espouses. It requires the Department of HHS, in consultation with EPA, to establish and support a research program consisting of:
Basic research (including epidemiologic and ecologic studies) which may include each of the following:
(i) Advanced techniques for the detection, assessment, and evaluation of the effects on human health of hazardous substances.
(ii) Methods to assess the risks to human health presented by hazardous substances.
(iii) Methods and technologies to detect hazardous substances in the environment and basic biological, chemical, and physical methods to reduce the amount and toxicity of hazardous substances.
42 U.S.C. § 9660(a)(1)(A); see also id. § 9660(c) (authorizing EPA to conduct research on the "detection, assessment, and evaluation of the effects on and risks to human health of hazardous substances and detection of hazardous substances in the environment"). These provisions say nothing about authorizing EPA to adopt rules at all, much less rules limiting reliance on studies that do not meet the criteria of the Proposed Rule. Section 311(a) merely provides for the Department of HHS to establish and support certain research programs. It does not give EPA any authority at all, much less authority to limit the type of studies that can be relied upon for purposes of implementing CERCLA's operative provisions. Accordingly, CERCLA provides no support for EPA's actions here.
Comment [6112-1591]: Commenter (6137) contends that under CERCLA, Congress created a hazardous substance research and classification regime based largely on studies that "determine relationships between exposure to toxic substances and illness" -- the very dose response studies that the Proposed Rule would stifle. 42 U.S.C. § 9604(i)(1). The statute requires EPA and ATSDR to annually update a list of hazardous substances commonly found at facilities on the NPL that the agencies determine "pos[e] the most significant potential threat to human health due to their known or suspected toxicity to humans and the potential for human exposure to such substances . . ." Id. § 9604(i)(2)(A), (B). EPA must also develop guidelines for ATSDR's toxicological profiles of each listed substance, which must include "available toxicological information and epidemiologic evaluations . . . to ascertain the levels of significant human exposure for the substance and the associated acute, subacute, and chronic health effects." Id. § 9604(i)(3)(A). For any substance for which adequate information is unavailable, EPA and ATSDR must create a program of toxicological and epidemiological research to develop that information. Id. § 9604(i)(5).
Congress specified that CERCLA health assessments include: 
preliminary assessments of the potential risk to human health posed by individual sites and facilities, based on such factors as the nature and extent of contamination, the existence of potential pathways of human exposure (including ground or surface water contamination, air emissions, and food chain contamination), the size and potential susceptibility of the community within the likely pathways of exposure, the comparison of expected human exposure levels to the short-term and long-term health effects associated with identified hazardous substances and any available recommended exposure or tolerance limits for such hazardous substances, and the comparison of existing morbidity and mortality data on diseases that may be associated with the observed levels of exposure.
42 U.S.C. § 9604(i)(6)(F). And when assessing alternate remedial actions under CERCLA, EPA must "at a minimum, take into account . . . the persistence, toxicity, mobility, and propensity to bioaccumulate of such hazardous substances and their constituents [and] short- and long-term potential for adverse health effects from human exposure." Id. § 9621(b)(1). 
EPA's Proposed Rule would prevent the Agency from using the very types of health assessments that Congress mandates. It undermines both the letter and spirit of the statute and is therefore unlawful.
Comment [6118-1728]: Commenter (6143) asserts that EPA has issued its proposed rule on Strengthening Transparency in Regulatory Science pursuant to its authority under several environmental statutes that that it implements, including the CAA; CWA; SDWA; RCRA; CERCLA; EPCRA; FIFRA; and TSCA. This proposed action is fully consistent with the authorities and objectives laid out in those statutes. EPA must ensure that any final action it takes on this proposal continues to respect the balance reflected in those statutes between sound, transparent science, and legitimate privacy interests.
Comment [6152-642]: Commenter (6122) EPA cites multiple statutory provisions in an attempt to support the proposed rule that it believes authorize this type of rulemaking, "including provisions providing general authority to promulgate regulations necessary to carry out the Agency's functions under these statutes and provisions specifically addressing the Agency's conducting of and reliance on scientific activity to inform those functions . . ." None of the specific statutory provisions provided by EPA provide specific authority for EPA to fundamentally rework the foundational processes it uses to assess and include/reject science. Indeed, if such authority existed, one would have expected EPA to have discovered this authority many decades earlier. Instead, EPA is attempting to use general grants of authority to create a completely novel approach to assessing science. However, as the Supreme Court has stated, "Congress...does not alter the fundamental details of a regulatory scheme in vague terms or ancillary provisions -- it does not, one might say, hide elephants in mouseholes." And if this is true, neither can the EPA alter the fundamental details of all of its regulatory schemes based on such vague provisions of law. We review each of the provisions listed by EPA, none of which discuss transparency, dose response, pivotal regulatory science, benefit-cost calculation, or changing "agency culture," the stated goals of this rulemaking.
Section 115 of the CERCLA and Executive Order 12580 authorize the president to delegate duties under CERCLA and designate EPA as the statute's Administrator. This delegation provision provides the authority to prescribe regulations, but EPA has not provided a single rationale explaining how the proposed rule is "necessary" to the carrying out functions under CERCLA. EPA points to another section as a parallel citation, which addresses assessment, listing, and evaluation of facilities, and directs commencement of remedial investigations and feasibility studies and commencement of remedial actions. This provision is irrelevant to the proposed rule, we will assume that the inconsistency is a mere typographical error and not intended to cite additional statutory authority.
Section 311 of the CERCLA directs the Administrator in the use and development of research and training tools. The section does not mention transparency or any of the other undefined terms that EPA uses like "dose response" and "benefit-cost calculation."
Comment [6155-2632]: Commenter (6125) asserts that it is clear from even a brief look that these provisions require EPA to consider all the best available science, without authorizing, or even mentioning any of the per se restrictions based on "transparency" that EPA has proposed to use as a universal overriding rule. For these reasons, EPA must justify its proposal as a proper exercise of its authority under ​each ​of the statutory provisions whose operation it would affect - provisions that the agency does not even cite. It is the job of the proposer, not the commenters, to survey these provisions and provide the needed justification. 
In several statutes Congress has explicitly told EPA how to use science in decision-making. The commenter notes that the provisions all use mandatory language directing EPA to use reliable and relevant scientific information -- and further specify how to do so. They all direct that decisions regarding protection of public health be based on the full range of relevant scientific information, in some cases specifying how to identify what information to use. None of them give EPA discretion to ignore such information for any of the reasons set forth in the proposal.
Comment [6155-2724]: Commenter (6125) asserts that other provisions prescribe factors EPA must use in making decisions. The CERCLA requires that EPA rules setting cleanup priorities "shall be based upon relative risk or danger to public health or welfare or the environment... taking into account to the extent possible" seven enumerated factors related to risk. 
Like the express science provisions discussed above, both of these provisions are mandatory and prescribe specific factors EPA must assess. Neither offers any basis for failing to consider such factors or evidence regarding such factors because it is supported by science that is not "transparent." When Congress wished to limit available science to information that is publicly available, it has done so. See CWA Section 311(a)(27), 33 USC 1321(a)(27) (for purposes of addressing oil and hazardous substance liability, "best available science" is defined as evidence that "uses peer reviewed and publicly available data"). Its failure to impose such limits on the use of data or science in the provisions discussed above is thus purposeful. As noted, the proposal does not even mention any of these relevant statutory requirements governing the use of science or data or information, much less explain how any of them could authorize EPA to make a decision without using information that is relevant to decision-making, solely for lack of transparency and regardless of reliability or relevance. None of these statutes provide EPA with any authority to pick and choose what reliable and useful science to use or ignore specific studies based on transparency. While the proposal misleadingly pays lip service to using the best available science, it would expressly "preclude" the use of reliable and available science that is not, in its characterization, "transparent," regardless of whether it is the best available science. The proposal does not even mention the governing statutory requirements, much less explain how its approach is consistent with them.
Comment [6163-1107]: Commenter (6133) contends that under the CERCLA, EPA cites 42 U.S.C. § 9616 as authority, but that section merely provides a schedule for the assessment and remediation of Superfund sites. It is entirely unclear what this has to do with the subject matter of the Proposal. EPA does not state specifically which provision of 42 U.S.C. § 9616 it believes authorizes the Proposal, nor does the Proposal even explain the reference. Thus, the citation fails to provide sufficient notice for the public to comment on the proposed rule. 
EPA also cites 42 U.S.C. § 9660, which has many subsections. This broad citation also fails to provide sufficient notice for the public to comment on the proposed rule. Subsections (a), (b), and (c) require the Secretary of Health and Human Services (HHS) and the Administrator of EPA to establish research programs on the effects of hazardous substances on human health. But nothing in those sections limits EPA's consideration of studies in which the data can be made public or draws a line between dose-response data and other types of data. The absence of any such congressional distinction or restriction makes it all the more obvious that EPA invented and added the distinction and restrictions in the Proposal as a matter of its own policy preferences, contrary to the Act. This EPA may not do. CERCLA does not authorize the Proposal. 
Comment [6163-1139]: Commenter (6133) provides that under CERCLA, EPA has power to clean up sites that are contaminated with hazardous substances, and to assure that responsible parties pay for such clean up. CERCLA requires EPA to issue regulations that identify hazardous substances that "present substantial danger to the public health or welfare or the environment," and that specify the quantities of such substances that trigger the Act's notification requirements. 42 U.S.C. § 9602(a). The Proposal contradicts this statutory mandate because it allows EPA to arbitrarily exclude some studies solely because the underlying data cannot be made public. Under the statute, EPA is required to use all relevant studies in determining whether a substance presents a substantial danger to people or the environment. 
CERCLA also requires the President to promulgate and revise the National Contingency Plan for the removal of hazardous substances. Id. § 9605(a), (b). The President has delegated that authority to EPA. Exec. Order No. 12580, 52 Fed. Reg. 2923 (1987); Exec. Order No. 12777, 56 Fed. Reg. 54757. The Plan must include criteria for determining priorities "based upon relative risk or danger to public health or welfare or the environment," taking into account enumerated factors. 42 U.S.C. § 9605(a)(8)(A). The Proposal conflicts with this section because it would direct EPA to disregard relevant studies solely because the underlying data could not be made public, even if those studies shed light on the enumerated factors. 
CERCLA's non-rulemaking provisions also show that Congress did not intend for studies to be excluded from consideration simply because the underlying data cannot be made public. For example, CERCLA authorizes the President to address hazardous substance releases that pose an "imminent and substantial danger to the public health or welfare," and to "undertake such investigations, monitoring, surveys, testing, and other information gathering" as necessary to determine "the extent of danger to the public health or welfare or to the environment." Id. § 9604(a), (b). This shows that Congress's purpose in enacting CERCLA was to address the serious public health and environmental threats of hazardous substance releases. That purpose would be undermined if EPA could refuse to consider relevant studies only because the underlying data cannot be made public. EPA also has co-responsibility with the ATSDR to establish a registry of diseases relating to toxic substance exposure, as well as to create a list of hazardous substances found at Superfund sites, prepare a toxicological profile of those substances, and determine whether adequate information on the health effects of those substances exists. Id. § 9604(i). The statute specifically lists the types of studies and data that should be considered in determining whether adequate information exists and assessing the need for further research. Id. § 9604(i)(5); see also id. § 9604(i)(13). The statute does not exclude studies whose underlying data cannot be made public. In short, the Proposal contradicts both the statutory language and the purpose of CERCLA. 
Comment [6340-1457]: Commenter (6362) states that the provisions cited by EPA under the CAA, the CWA, the SDWA, the CERCLA, and the EPCRA in support of its authority to develop and implement its proposed rule all provide broad regulatory authority to promulgate regulations "as are necessary to carry out [the Administrator's] functions" under the statute. [...]
Comment [6343-2722]: Commenter (6465) provides that they further support EPA's objective of implementing this policy in a robust and consistent manner across its regulatory determinations. In particular, in order to robustly and consistently implement its scientific transparency standards, EPA should clarify that the science that is used in establishing remedial action performance standards for response actions under the of 1980 and for RCRA corrective actions is subject to EPA's proposed standards for ensuring transparency in regulatory science. 
Clean-up obligations under CERCLA and corrective action obligations under RCRA are among the most critical environmental issues that impact industrial manufacturing companies. Between 1980 and 2015, private parties committed more than $35 billion in cleanup costs, with several individual CERCLA matters having cleanup costs in excess of $1 billion. Consequently, many CERCLA clean-ups constitute the magnitude of significant regulatory actions under Executive Order 12866 for a "significant regulatory action."
Comment [6353-2323]: Commenter (6375) states that EPA solicits comment on whether additional or alternative sources of authority are appropriate bases for this proposed regulation. (pg. 18,771). EPA's statement concerning its legal authority could be enhanced by reference to statutory provisions that require it to rely on "accurate," "useful," or "best" data. For example, section 108(a)(2) requires that the Administrator issue air quality criteria that "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health and welfare . . .." 42 U.S.C. § 7408(a)(2). Increasing the transparency of regulatory science and ensuring that it is reproducible helps in evaluating its accuracy and usefulness.
Furthermore, making data and methods underlying regulatory decisions publicly available is not new. Guidelines issued by both the OMB and EPA implementing the IQA, Treasury and General Government Appropriations Act for Fiscal Year 2001, § 515, P.L. No. 106-554, 114 Stat. 2763 (Dec. 21, 2001), emphasize the importance of reproducibility in assessing the quality and utility of data used in making regulatory decisions. It appears the OMB's IQA guidelines recognize that the reproducibility of underlying science helps to ensure the integrity of agency decisions and explain that, "[m]aking the data and methods publicly available will assist in determining whether analytic results are reproducible." 67 Fed. Reg. 8452, 8460 (Feb. 22, 2002) (OMB IQA Guidelines).
EPA's IQA guideline states: "A higher degree of transparency about data and methods will facilitate the reproducibility of such information by qualitied third parties, to an acceptable degree of imprecision. . .. It is important that analytic results for influential information have a higher degree of transparency regarding (1) the source of the data used, (2) the various assumptions employed, (3) the analytic methods applied, and (4) the statistical procedures employed."
EPA should also consider replacing some of the citations it provides for its legal authority with others that might be more relevant. For example, the Agency may want to cite 42 U.S.C. § 6981, on research, instead of 42 U.S.C. § 6979, which concerns "Labor standards." In addition, section 115 of CERCLA, 42 U.S.C. § 9615, which authorizes regulations, might be a more appropriate citation than section 116 of that Act, 42 U.S.C. § 9616, which specifies schedules for assessment and evaluation of facilities subject to the Act's requirements.
Comment [6353-2328]: Commenter (6375) states that EPA solicits comment on whether alternative or additional regulatory or other policy vehicles are appropriate to establish and implement these policies, and whether further regulatory or other policy vehicles at the programmatic or statutory level would be appropriate as alternative or additional steps the agency may take to further the policies articulated in Section II. (pg. 18,771 of the NPRM FR notice).
The regulatory language that EPA proposes provides general requirements that apply to the Agency's use of science under the many statutes that it implements. The generality of the proposed regulations may be a function of the diverse purposes of those statutes Specifically, some of these statutes focus on removal from (e.g., CERCLA), the environment. The scientific analyses required for these different programs and the sources of the scientific information underlying those analyses differ. The Agency should consider supplementing the general language that it has proposed with statutory specific regulations. A statutory-specific rulemaking would provide greater certainty going forward.
Comment [6779-4123]: Commenter (6801) provides that the CERCLA, commonly known as Superfund, was enacted by Congress on December 11, 1980. This law created a tax on the chemical and petroleum industries and provided broad Federal authority to respond directly to releases or threatened releases of hazardous substances that may endanger public health or the environment. EPA, Superfund: CERCLA Overview, https://www.epa.gov/superfund/superfund-cercla-overview While the space allotted to this comment does not permit a thorough discussion of CERCLA's history, EPA discusses the many chemical spills that led to CERCLA and includes computerized graphics at this address: https://www.epa.gov/superfund/superfund-history
The EPA's numerous CERCLA regulations are in print in the CFR. Electronic versions can be found at https://www.gpo.gov/fdsys/browse/collectionCfr.action?collectionCode=CFR (official) or https://www.ecfr.gov/cgi-bin/retrieveECFR?gp=1&SID=d5cf8817453753f37820f1e458cfd52a&ty=HTML&h=L&mc=true&n=pt40.30.307&r=PART (unofficial, but more current).
The adoption or amendment of numerous EPA regulations require good scientific data including private medical data protected by privacy laws such as HIPAA, as well as data that is private and or difficult to reproduce for other reasons (for example, endangered wildlife surveys that may not be able to be repeated without impermissibly harming the wildlife or studies that require access to private land).
Examples of such good data and studies that led to CERCLA and/or the related regulations include: https://www.health.ny.gov/environmental/investigations/love_canal/lcreport.htm
https://www.health.ny.gov/environmental/investigations/love_canal/docs/report_public_comment_final.pdf https://archive.epa.gov/epa/aboutepa/love-canal-tragedy.html http://www.gendisasters.com/new-jersey/13273/logan-township-nj-chemical-explosion-dec-1977
An example of a regulation that may not have been adopted if the proposed regulation was in effect, 40 CFR Part 355 Sub Part C Section 355.33 provides:
      355.33What release quantities of EHSs and CERCLA hazardous substances trigger the emergency release notification requirements of this subpart?
      The release of a reportable quantity (RQ) of an EHS or CERCLA hazardous substance within any 24-hour period triggers the emergency release notification requirements. RQs for EHSs are listed in Appendices A and B of this part in the column labeled RQ. RQs for CERCLA hazardous substances are listed in Table 302.4 of 40 CFR 302.4 in the column labeled final RQ. 
Obviously, any revision of the rule or of the RQs listed in Table 302.4 of 40 CFR 302.4 in the column labeled 'final RQ' would require a good scientific basis. That would include data protected by privacy laws such as HIPAA.
Unfortunately, virtually all EPA's CERCLA regulations suffer from the same defect.
Comment [8750-1968]: Commenter (9227) contends that EPA arbitrarily failed to consider the far-reaching implications this Proposal could have on inter-agency coordination and consultation given that other agencies normally rely on research potentially excluded by the Proposal.
Consultation Provisions in the CERCLA
§311(a)(1) 
Research, Development, and Demonstration
The Secretary of HHS...in consultation with the Administrator, shall establish and support a basic research and training program...consisting of the following:
(A) Basic research (including epidemiologic and ecologic studies) which may include each of the following:
(i) Advanced techniques for the detection, assessment, and evaluation of the effects on human health of hazardous substances.
(ii) Methods to assess the risks to human health presented by hazardous substances.
(iii) Methods and technologies to detect hazardous substances in the environment and basic biological, chemical, and physical methods to reduce the amount and toxicity of hazardous substances.
(B) Training, which may include each of the following: 
(i) Short courses and continuing education for State and local health and environment agency personnel and other personnel engaged in the handling of hazardous substances, in the management of facilities at which hazardous substances are located, and in the evaluation of the hazards to human health presented by such facilities.
(ii) Graduate or advanced training in environmental and occupational health and safety and in the public health and engineering aspects of hazardous waste control.
(iii) Graduate training in the geosciences, including hydrogeology, geological engineering, geophysics, geochemistry, and related fields necessary to meet professional personnel needs in the public and private.
(a) sectors and to effectuate the purposes of this Act.
***
§311(a)(2)
Research, Development, and Demonstration
The Director of the NIEHS shall cooperate fully with the relevant Federal agencies referred to in subparagraph (A) of paragraph (5) in carrying out the purposes of this section. 
***
§311(a)(5)
Research, Development, and Demonstration
To assist in the implementation of this subsection and to aid in the coordination of research and demonstration and training activities funded from the Fund under this section, the Secretary shall appoint an advisory council (hereinafter in this subsection referred to as the ``Advisory Council'') which shall consist of representatives of the following:
 The relevant Federal agencies.
 The chemical industry.
 The toxic waste management industry.
 Institutions of higher education.
 State and local health and environmental agencies.
 The general public.
***
§311(a)(6)
Research, Development, and Demonstration
Within nine months after the date of the enactment of this subsection, the Secretary, acting through the Director of the NIEHS, shall issue a plan for the implementation of paragraph (1). The plan shall include priorities for actions under paragraph (1) and include research and training relevant to scientific and technological issues resulting from site specific hazardous substance response experience. The Secretary shall, to the maximum extent practicable, take appropriate steps to coordinate program activities under this plan with the activities of other Federal agencies in order to avoid duplication of effort. The plan shall be consistent with the need for the development of new technologies for meeting the goals of response actions in accordance with the provisions of this Act. The Advisory Council shall be provided an opportunity to review and comment on the plan and priorities and assist appropriate coordination among the relevant Federal agencies referred to in subparagraph (A) of paragraph (5).
***
§311(c)
Research, Development, and Demonstration
HAZARDOUS SUBSTANCE RESEARCH. -- The Administrator may conduct and support, through grants, cooperative agreements, and contracts, research with respect to the detection, assessment, and evaluation of the effects on and risks to human health of hazardous substances and detection of hazardous substances in the environment. The Administrator shall coordinate such research with the Secretary of HHS, acting through the advisory council established under this section, in order to avoid duplication of effort 
***
§104(i)(4)
Response Authorities
The Administrator of the ATSDR shall provide consultations upon request on health issues relating to exposure to hazardous or toxic substances, on the basis of available information, to the Administrator of EPA
***
§104(i)(5)(A)
Response Authorities
For each hazardous substance listed pursuant to paragraph (2), the Administrator of ATSDR (in consultation with the Administrator of EPA and other agencies and programs of the Public Health Service shall assess whether adequate information on the health effects of such substance is available. For any such substance for which adequate information is not available (or under development), the Administrator of ATSDR, in cooperation with the Director of the National Toxicology Program (NTP), shall assure the initiation of a program of research designed to determine the health effects (and techniques for development of methods to determine such health effects) of such substance.
***
§104(i)(6)(C)
Response Authorities
In determining the priority in which to conduct health assessments under this subsection, the Administrator of ATSDR, in consultation with the Administrator of EPA, shall give priority to those facilities at which there is documented evidence of the release of hazardous substances, at which the potential risk to human health appears highest, and for which in the judgment of the Administrator of ATSDR existing health assessment data are inadequate to assess the potential risk to human health as provided in subparagraph (F). In determining the priorities for conducting health assessments
***
§107(c)
Abatement Action 
Within one hundred and eighty days after enactment of this Act, the Administrator of the EPA shall, after consultation with the Attorney General, establish and publish guidelines for using the imminent hazard, enforcement, and emergency response authorities of this section and other existing statutes administered by the Administrator of the EPA to effectuate the responsibilities and powers created by this Act.
***
§120(e)(1)
Federal Facilities
Not later than 6 months after the inclusion of any facility on the NPL, the department, agency, or instrumentality which owns or operates such facility shall, in consultation with the Administrator and appropriate State authorities, commence a remedial investigation and feasibility study for such facility.
***
§120(e)(6)
Federal Facilities
Administrator, after consultation with other departments, may determine that remedial efforts should be done by another potentially responsible party and may enter into a settlement agreement with such party.
Comment [9066-1934]: Commenter (6188) asserts that the CERCLA § 311 requires EPA to carry out a research program to develop and demonstrate "alternative or innovative treatment technologies" for potential use in response actions. 42 U.S.C. § 9660(b). Information gathered as part of that research program is already required to be made publicly available, with the express exception of trade secrets or personal proprietary information, two categories of data that the Proposed Rule does not protect on its face. Indeed, in requiring that all studies' supporting data be made publicly available, the Proposed Rule's language contradicts this statutory mandate. In sum, the authority EPA has cited for the rulemaking not only fails to confer upon EPA the power to promulgate the rule, but also is inconsistent with the rule's requirements.
Comment [9078-3554]: Commenter (6194) states that the Proposal would prevent EPA from relying on the best available information in discharging its duties. EPA is charged with developing regulations that provide societal benefits by reducing harm to human health and the environment from the presence of chemicals in air, soil, drinking water, food, and consumer products. The Proposal implicates research that is central to making such determinations; creating an obstacle to the consideration of such science is contrary to both primary statutory directives and requirements to conduct cost benefit analyses. Likewise, EPA fails to address how it could implement a regulation that is inconsistent with federal requirements and standards governing the use of science across agencies.
This directive is embodied in multiple statutes implemented by EPA and is reflected in other environmental statutes, further illustrating Congress' conceptualization of "best" available science. Broadly speaking, there are three basic types of statutory requirements that the EPA and other agencies consider scientific information. First, there are requirements that the agency consider the "best available science." Second, there are requirements that the agency consider particular factors, including scientific factors. Third, there are requirements that the agency balance economic costs or technological feasibility against public health, environmental, or other benefits that must be scientifically assessed. The precise terminology regarding the required use of science may, as illustrated below, vary across statutes, but what these provisions have in common are requirements for EPA and other agencies to use scientific information that is "best," not simply adequate, and "available." 
As EPA has previously explained, the agency would be unable to fulfill statutory mandates if it could not consider studies that follow federal laws and ethical standards regarding the privacy of patient and individual data: 
      (If EPA and other governmental agencies could not rely on published studies without conducting an independent analysis of the enormous volume of raw data underlying them, then much plainly relevant scientific information would become unavailable to EPA for use in setting standards to protect public health and the environment. . . . [S]uch data are often the property of scientific investigators and are often not readily available because of the proprietary interests of the investigators or because of arrangements made to maintain confidentiality regarding personal health status and lifestyle information of individuals included in such data. Without provisions of confidentiality, the possibility of conducting such studies could be severely compromised.)
Examples of EPA statutory mandates that could be impeded by the Proposal include (i) the development of standards to control emissions from burning hazardous waste fuels under the RCRA "as may be necessary to protect human health and the environment," 42 U.S.C. § 6924(q)(1); and (ii) determinations of whether hazardous sites need to be placed on the CERCLA NPL. With respect to the latter, CERCLA provides that, if a health assessment indicates that a release or threatened release may pose a serious threat to human health, the ATSDR shall "notify the Administrator of EPA who shall promptly evaluate such release or threatened release in accordance with the hazard ranking system . . . to determine whether the site shall be placed on the National Priorities List." 42 U.S.C. § 9604 (i)(6)(H). As referenced in footnote 41 of the commenter's comment letter, health assessments provided by ATSDR to EPA on a scheduled basis are exempt from peer review as a prerequisite to submission.
The Proposal would also cut EPA off from information needed to complete the cost-benefit analysis of regulations required under certain statutes and government-wide requirements.
Not only would the Proposal impede EPA's fulfillment of its statutory duties, but it would also be contrary to judicial precedent, which holds that "best available science" includes "all existing scientific evidence relevant to the [agency] decision [in question]." Ecology Ctr., Inc. v. U.S. Forest Serv., 451 F.3d 1183, 1194 n.4 (10th Cir. 2006) (quoting Heartwood, Inc. v. U.S. Forest Serv., 380 F.3d 428, 436 (8th Cir. 2004)). While agencies need not produce new evidence, they must seek out information and "`cannot ignore existing data.'" Id. (emphasis added). Even if an agency is anticipating the arrival of better evidence, it must act on the basis of the evidence it currently has. Many of the fundamental public health studies on which EPA has based key rules and standards under the statutes are studies for which the raw data was not or could not have been released. Nonetheless these studies have been examined and validated, as will future studies. Arbitrarily cutting off consideration of a category of science is inconsistent with EPA's statutory obligations.
Comment [6168-4099]: Commenter (6178) raises the following concerns with the Transparency Rule proposed by the NPRM: 
1. The Agency cites specific sections of statutes it administers as the basis of the Agency's authority to adopt the Transparency Rule; specifically CERCLA (as delegated to the Administrator via Executive Order 12580) sections 115, 311, 42 U.S.C. 9616, 9660. The Agency rightly cites acts of Congress in an attempt to support the NPRM as such authority is necessary for the Agency to adopt regulations pursuant to the above listed Acts. 
2. However, Congress, in each of the Acts' sections cited, never instructs the Agency to consider transparency to the public of supporting data and/or models of the scientific research when deciding to rely on such research as support for regulations passed pursuant to any of the Acts. Consequently, the Transparency Rule requires the EPA to rely on factors which Congress has not intended it to consider when promulgating regulations pursuant to the Acts, rendering the Transparency Rule arbitrary and capricious under the APA. 
3. Further, not only does the Transparency Rule lack sufficient Congressional intent to avoid being deemed arbitrary and capricious, implementing the Rule would actually contradict Congressional instruction to conduct scientific research with a focus on `promotion' and `coordination' among itself and other institutions that produce scientific research material to the objectives of the Acts Congress requires the EPA to enforce. This provides additional cause for finding the Transparency Rule arbitrary and capricious under the APA.
Comment [6163-1276]: Commenter (6133) states that the Proposal contains a seemingly unrelated provision regarding agency peer review. The Proposal, in § 30.7, reads: 
 
What role does independent peer review in this section?
EPA shall conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein. 
 
Because transparency in regulatory science includes addressing issues associated with assumptions used in models, EPA shall ask peer reviewers to articulate the strengths and weaknesses of EPA's justification for the assumptions applied and the implications of those assumptions for the results. 
 
83 Fed. Reg. at 18,774. 
 
There is no statutory authority for EPA to "conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein." The federal statutes that EPA lists as putative authority for the Proposal provide no authority for proposed § 30.7. See CERCLA (as delegated to the Administrator via Executive Order 12580) sections 115, 311, 42 U.S.C. 9616, 9660. The claimed authorities that EPA lists do not mention peer review or even allude to the concept. Id. Neither the Proposal nor accompanying docket materials identify any provision of any federal statute that authorizes EPA to promulgate proposed § 30.7. Our own research revealed no provision of any federal statute that authorizes EPA to promulgate proposed § 30.7. 
 
When Congress writes federal statutes, Congress knows how to create legal authority for peer review, who shall conduct that peer review, what role, if any, that EPA or other parties will play, and how that peer review may be conducted. None of the provision in the law authorize EPA's Proposal in § 30.7. Instead, the statutes require that EPA use peer-reviewed science, regardless of whether it would meet EPA's definition of the term in § 30.7. And in cases where the law requires EPA to conduct the review, the statutes often spell out specifically how that should happen. 
CERCLA
§ 9604. Response authorities 
(i) ATSDR; establishment, functions, etc.
[...]
Any toxicological profile or revision thereof shall reflect the Administrator of ATSDR's assessment of all relevant toxicological testing which has been peer reviewed. The profiles required to be prepared under this paragraph for those hazardous substances listed under subparagraph (A) of paragraph (2) shall be completed, at a rate of no fewer than 25 per year, within 4 years after October 17, 1986. A profile required on a substance listed pursuant to subparagraph (B) of paragraph (2) shall be completed within 3 years after addition to the list. The profiles prepared under this paragraph shall be of those substances highest on the list of priorities under paragraph (2) for which profiles have not previously been prepared. Profiles required under this paragraph shall be revised and republished as necessary, but no less often than once every 3 years. Such profiles shall be provided to the States and made available to other interested parties.
[...]
(7)(A) Whenever in the judgment of the Administrator of ATSDR it is appropriate on the basis of the results of a health assessment, the Administrator of ATSDR shall conduct a pilot study of health effects for selected groups of exposed individuals in order to determine the desirability of conducting full scale epidemiological or other health studies of the entire exposed population. (B) Whenever in the judgment of the Administrator of ATSDR it is appropriate on the basis of the results of such pilot study or other study or health assessment, the Administrator of ATSDR shall conduct such full scale epidemiological or other health studies as may be necessary to determine the health effects on the population exposed to hazardous substances from a release or threatened release. If a significant excess of disease in a population is identified, the letter of transmittal of such study shall include an assessment of other risk factors, other than a release, that may, in the judgment of the peer review group, be associated with such disease, if such risk factors were not taken into account in the design or conduct of the study.
[...]
(13) All studies and results of research conducted under this subsection (other than health assessments) shall be reported or adopted only after appropriate peer review. Such peer review shall be completed, to the maximum extent practicable, within a period of 60 days. In the case of research conducted under the NTP, such peer review may be conducted by the Board of Scientific Counselors (BOSC). In the case of other research, such peer review shall be conducted by panels consisting of no less than three nor more than seven members, who shall be disinterested scientific experts selected for such purpose by the Administrator of ATSDR or the Administrator of EPA, as appropriate, on the basis of their reputation for scientific objectivity and the lack of institutional ties with any person involved in the conduct of the study or research under review. Support services for such panels shall be provided by the ATSDR, or by the EPA, as appropriate. 
 
42 U.S.C. § 9604 (emphasis added). 
 
Under longstanding federal case law, when Congress authorizes an approach in one section of a statute using specific language but does not do the same in other sections of a statute, courts presume that Congress acted purposefully and did not mean to address or authorize that approach in those other statutory sections. See, e.g., Dean v. United States, 556 U.S. 568 (2009) ("It is generally presumed that Congress acts intentionally when including particular language in one section of a statute but not in another." (citing Russello v. United States, 464 U.S. 16, 23 (1983)). Not only are there no implied grants of authority to an agency in the other statutory sections, but the congressional decisions to authorize the approach elsewhere in the statute give even greater force to the conclusion that the agency has not been given authority where Congress did not use the same or similar authorizing language. 
The EPA approach proposed in § 30.7 is even more unlawful than would be the case, independently, under this case law. Proposed § 30.7 says "EPA shall conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein." 83 Fed. Reg. at 18,774/2 (emphasis added). EPA proposes to bind itself ("shall") to conduct a particular form of peer review on "all pivotal regulatory science" based on an unenforceable, non-binding OMB bulletin that has never been the subject of notice and comment rulemaking and that itself is not authorized by any federal law. To the contrary, treating the content of the unenforceable bulletin as a binding regulation would itself violate the federal statutes that EPA implements, because those statutes do not codify the bulletin, and EPA would be unlawfully codifying a mere policy preference. This EPA may not do. 
This Proposal is unlawful, arbitrary, and capricious, and an abuse of EPA discretion. To reiterate, the Proposal identifies no statutory authority for EPA to conduct independent peer review for any, much less all, "pivotal regulatory science" consistent with the dense content (and exceptions) of the OMB bulletin. The Proposal identifies no statutory authority for EPA to bind itself, and future administrations, to conduct peer review only in this fashion, unless and until future notice-and-comment rulemaking is undertaken. The Proposal identifies no suggestion in statutory language or legislative history that Congress intended EPA to conduct binding peer review consistent with this OMB bulletin, notwithstanding that Congress has known about this bulletin since 2005. EPA has simply made up proposed § 30.7 -- with its the link to the OMB bulletin, and the putative authority for the Proposal -- out of whole cloth. This EPA may not do. 
1.3.5.1.2 RCRA
Comment [5410-4087]: Commenter (5421) objects to the proposal to limit the use of scientific evidence in rulemaking proceedings.
EPA estimates that more than 35 million people, roughly 12 percent of the U.S. population, live within one mile of a RCRA Corrective Action site. The presence of hazardous constituents in contaminated soil, sediments, groundwater, surface water, and air at, or emanating from, RCRA Corrective Action sites can increase the risk of adverse health effects to exposed populations. This can be especially critical for minority and poor communities as well as sensitive sub - populations such as children, pregnant women, and the elderly who can be disproportionately affected. Dangers include acute health effects, such as poisoning and injuries from fire or explosions, and long-term effects, such as cancers, birth defects, and other chronic non - carcinogenic effects (e.g., damage to kidney, liver, nervous and endocrine systems). By reducing exposures to contaminants listed in Table 1, as well as hundreds of others, RCRA Corrective Action cleanups protect the health of local residents, site workers, and others. Corrective Action sites can, and often do, contain more than one contaminant." RCRA Corrective Action: Case Studies Report, April 2013, p. 1 (PDF copy attached to comment letter excerpt).
In addition to helping humans, RCRA helps both surrounding communities and the environment. Id. at 2.
EPA adopted regulations to administer its duties under RCRA, see id. at p. 24 (Appendix A). Under previous administrations, EPA used its regulations and powers flexibly to maximize its effectiveness. Id.
Unfortunately, EPA has much to do. See, e.g., id. Even when construction of all RCRA facilities is completed, cleanup operations will continue. Id. at p. 25 (Appendix B).
A very scary problem with the EPA's proposed "Anti-Environment" Regulation is that it will eviscerate EPA's ability to regulate under RCRA to protect the millions of Americans living near RCRA sites. Good public policy requires both EPA and our elected officials to carefully consider, and if appropriate, to rely on the available data and studies. The proposed regulation is nothing short of a brazen attempt to undercut the progress we Americans have made. Ignoring such good studies as proposed would endanger millions.
Comment [6112-1558]: Commenter 6137 contends that the RCRA, 42 U.S.C. § 6901 et seq., gives EPA the authority and responsibility to manage and control solid and hazardous waste, including the generation, transportation, treatment, storage, and disposal of the waste. EPA points to two provisions in RCRA for support of this Proposed Rule, neither of which authorizes this action.
The first provision of RCRA cited as authority for this rule, § 6912(a)(1), provides the Administrator with the general authority to "prescribe, in consultation with Federal, State, and regional authorities, such regulations as are necessary to carry out his functions under this chapter." 42 U.S.C. § 6912(a)(1). RCRA defines the functions of EPA in the area covered by RCRA, and therefore, EPA cannot rely upon the general authority to make rules provided by the statute. Am. Petroleum Inst., 52 F.3d at 1119 ("EPA cannot rely on its general authority to make rules necessary to carry out its functions when a specific statutory directive defines the relevant functions of EPA in a particular area."). Moreover, limiting the consideration of reliable health science when promulgating regulations that have significant health and environmental impacts is in no way "necessary" for the Administrator to carry out his functions under RCRA, and thus, for this reason too, the general rulemaking provision does not authorize this rule. See Mourning, 411 U.S. at 369.
The second provision of RCRA upon which EPA relies, § 6979, is inapposite. This provision pertains to labor standards related to wages for laborers and mechanics. 42 U.S.C. § 6979. This provision has no relevance to the Proposed Rule whatsoever and certainly does not provide the authority for it. Thus, nothing that EPA cites to in RCRA provides the requisite authority for this Proposed Rule.
Comment [6112-1603]: Commenter (6137) asserts that not only do the provisions of RCRA upon which EPA relies not provide the requisite statutory authority, but other provisions of RCRA render the Proposed Rule unlawful. Specifically, section 8001 of RCRA provides that the Administrator shall conduct or otherwise assist in research, investigations, experiments, and other studies without limitation on what studies or data can be considered. 42 U.S.C. § 6981. The law mandates a broad and inclusionary role for science, requiring EPA to consider studies without limitation. The Proposed Rule's elimination from consideration of entire categories of scientific data conflicts with the requirements of this section of RCRA, and thus cannot stand. See Decker v. Nw. Envtl. Def. Ctr., 568 U.S. 597 (2013).
Comment [6118-1729]: Commenter (6143) provides that EPA has issued its proposed rule on Strengthening Transparency in Regulatory Science pursuant to its authority under several environmental statutes that that it implements, including the CAA; CWA; SDWA; RCRA; CERCLA; EPCRA; FIFRA; and TSCA. This proposed action is fully consistent with the authorities and objectives laid out in those statutes. EPA must ensure that any final action it takes on this proposal continues to respect the balance reflected in those statutes between sound, transparent science, and legitimate privacy interests.
Comment [6152-640]: Commenter (6122) states that EPA cites multiple statutory provisions in an attempt to support the proposed rule that it believes authorize this type of rulemaking, "including provisions providing general authority to promulgate regulations necessary to carry out the Agency's functions under these statutes and provisions specifically addressing the Agency's conducting of and reliance on scientific activity to inform those functions . . ."[30] None of the specific statutory provisions provided by EPA provide specific authority for EPA to fundamentally rework the foundational processes it uses to assess and include/reject science. Indeed, if such authority existed, one would have expected EPA to have discovered this authority many decades earlier. Instead, EPA is attempting to use general grants of authority to create a completely novel approach to assessing science. However, as the Supreme Court has stated, "Congress...does not alter the fundamental details of a regulatory scheme in vague terms or ancillary provisions -- it does not, one might say, hide elephants in mouseholes."[31] And if this is true, neither can the EPA alter the fundamental details of all of its regulatory schemes based on such vague provisions of law. We review each of the provisions listed by EPA, none of which discuss transparency, dose response, pivotal regulatory science, benefit-cost calculation, or changing "agency culture," the stated goals of this rulemaking.
Section 2002(a)(1) of the RCRA authorizes the Administrator to "prescribe, in consultation with Federal, State, and regional authorities, such regulations as are necessary to carry out his functions under this chapter. . ." This catch-all provision provides the authority to prescribe regulations, but EPA has not provided a single rationale explaining how the proposed rule is "necessary" to the carrying out functions under the RCRA.
Section 7009 of the RCRA states, "No grant for a project of construction under this chapter shall be made unless the Administrator finds that the application contains or is supported by reasonable assurance that all laborers and mechanics employed by contractors or subcontractors on projects of the type covered by sections 3141-3144, 3146, and 3147 of Title 40, will be paid wages at rates not less than those prevailing on similar work in the locality as determined by the Secretary of Labor in accordance with those sections; and the Secretary of Labor shall have with respect to the labor standards specified in this section the authority and functions set forth in Reorganization Plan Numbered 14 of 1950 (15 F.R. 3176) and section 3145 of Title 40." It is almost impossible to imagine how the EPA determined that this section gives authority for the content of this rulemaking.
Comment [6163-1106]: Commenter (6133) contends that EPA claims that 42 U.S.C. § 6912(a)(1) of the RCRA provides authority for the rule. But 42 U.S.C. § 6912(a)(1) merely states that the Administrator is authorized to "prescribe, in consultation with Federal, State, and regional authorities, such regulations as are necessary to carry out his functions under this chapter." As explained above, that general grant of authority cannot support the rule, especially when the rule conflicts with the Act. There is no support in RCRA for the Proposal. 
It appears that EPA's citation to 42 U.S.C. § 6979 is a mistake. That section deals with labor standards for construction and says nothing about research, data, or science. At any rate, EPA does not state specifically which provision of 42 U.S.C. § 6979 it believes authorizes the Proposal. Thus, the citation fails to provide sufficient notice for the public to comment on the proposed rule.
Comment [6163-1138]: Commenter (6133) notes that under the RCRA, EPA regulates the generation, transportation, treatment, storage, and disposal of hazardous waste. EPA must develop, and revise from time to time, "criteria for identifying the characteristics of hazardous waste" and "for listing hazardous waste" that should be subject to regulation, "taking into account toxicity, persistence, and degradability in nature, potential for accumulation in tissue, and other related factors such as flammability, corrosiveness, and other hazardous characteristics." 42 U.S.C. § 6921(a). EPA also must, in cooperation with ATSDR and the NTP, "identify or list those hazardous wastes" which must be subject to regulation because they contain "certain constituents (such as identified carcinogens, mutagens, or terat[o]gens) at levels in excess of levels which endanger human health." Id. § 6921(b)(1). Likewise, EPA must promulgate regulations establishing standards applicable to generators and transporters of hazardous waste, and owners and operators of hazardous waste treatment, storage, and disposal facilities, "as may be necessary to protect human health and the environment." Id. §§ 6922(a), 6923(a), 6924(a); see also id. § 6924(b), (d), (g).
The Proposal conflicts with RCRA's statutory mandate. RCRA requires EPA to evaluate and regulate hazardous waste based on whether it will endanger human health and the environment, while the Proposal allows EPA to disregard relevant science simply because the underlying data cannot be made public. Under RCRA, EPA cannot ignore studies for that reason. Thus, the Proposal violates RCRA.
Comment [6340-1460]: Commenter (6362) contends that the citation to the RCRA speaks to Labor Standards in the issuance of grants and does not appear applicable to this rulemaking authority.
Comment [6343-2710]: Commenter (6365) asserts that they are committed to meeting their obligations to operate in a responsible manner to avoid adversely impacting the environment and public health and safety. Both lead itself and the production of lead are extensively regulated by the EPA, including under the TSCA, CAA, CWA, and RCRA. As a result of these programs and other actions taken by industry, "[t]he United States have made tremendous progress in reducing lead concentrations in the outdoor air," with the national average concentrations dropping "more than 90 percent since 1980," and "EPA expects that most of the areas previously designated as nonattainment will have attained or are on track to attain by the end of 2016." 
In order to ensure that the regulatory standards that the ABR's members are subject to are supported by the best possible science, the ABR supports EPA's objective of Strengthening Transparency in Regulatory Science by ensuring that the data and models underlying regulatory science is publicly available in a manner sufficient for validation and analysis. Furthermore, EPA should establish clear criteria for data and model availability so that stakeholders can effectively determine whether the data and models underlying each regulatory proposal are available for review.
Comment [6343-2721]: Commenter (6365) supports EPA's objective of implementing this policy in a robust and consistent manner across its regulatory determinations. In particular, in order to robustly and consistently implement its scientific transparency standards, EPA should clarify that the science that is used in establishing remedial action performance standards for response actions under the CERCLA of 1980 and for RCRA corrective actions is subject to EPA's proposed standards for ensuring transparency in regulatory science.
Clean-up obligations under CERCLA and corrective action obligations under RCRA are among the most critical environmental issues that impact industrial manufacturing companies. Between 1980 and 2015, private parties committed more than $35 billion in cleanup costs, with several individual CERCLA matters having cleanup costs in excess of $1 billion. Consequently, many CERCLA clean-ups constitute the magnitude of significant regulatory actions under Executive Order 12866 for a "significant regulatory action."
Comment [8750-1969]: Commenter (9227) asserts that EPA arbitrarily failed to consider the far-reaching implications this Proposal could have on inter-agency coordination and consultation given that other agencies normally rely on research potentially excluded by the Proposal.
Consultation Provisions in the RCRA
§2002(a)(1)
Authorities of Administrator
In carrying out this Act, the Administrator is authorized to --  (1) prescribe, in consultation with Federal, State, and regional authorities, such regulations as are necessary to carry out his functions under this Act.
***
§1008(a)
Solid Waste Management Information and Guidelines
Administrator shall consult with Federal agencies, among others, to develop and publish guidelines for solid waste management.
***
§2001
Office of Solid Waste and Interagency Coordinating Committee
Establishing an Interagency Coordinating Committee for RCRA between EPA, Department of Energy (DOE), Department of Commerce, and all other Federal agencies. Includes coordinating research and projects.
***
§2002(a)(2)- (6)
Authorities of Administrator
(2) consult with or exchange information with other Federal agencies undertaking research, development, demonstration projects, studies, or investigations relating to solid waste; ...
(5) utilize the information, facilities, personnel and other resources of Federal agencies, including the National Bureau of Standards 1 and the National Bureau of the Census, on a reimbursable basis, to perform research and analyses and conduct studies and investigations related to resource recovery and conservation and to otherwise carry out the Administrator's functions under this Act; and
(6) to delegate to the Secretary of Transportation the performance of any inspection or enforcement function under this Act relating to the transportation of hazardous waste where such delegation would avoid unnecessary duplication of activity and would carry out the objectives of this Act and of the Hazardous Materials Transportation Act.
***
§4002(b)
Federal Guidelines for Plans
Not later than 18 months after enactment, Administrator shall consult with appropriate agencies to promulgate guidelines for the development and implementation of State plans. Such guidelines should be reviewed and revised at least every three years.
***
§8001(a)
Research, Demonstrations, Training, and Other Activities
The Administrator, alone or after consultation with the [DOE], or [FERC], shall conduct, and encourage, cooperate with, and render financial and other assistance to appropriate public (whether Federal, State, interstate, or local) authorities, agencies, and institutions, private agencies and institutions, and individuals in the conduct of, and promote the coordination of, research, investigations, experiments, training, demonstrations, surveys, public education programs, and studies relating to -- 
(1) any adverse health and welfare effects of the release into the environment of material present in solid waste, and methods to eliminate such effects... 
***
§8001(b)(2) (D)
Research, Demonstrations, Training, and Other Activities
any activities undertaken under provisions of sections 8002 and 8003 as related to energy; as related to energy or synthetic fuels recovery from waste; or as related to energy conservation shall be accomplished through coordination and consultation with the [DOE]
Comment [9078-3553]: Commenter (6194) contends that the Proposal would prevent EPA from relying on the best available information in discharging its duties. EPA is charged with developing regulations that provide societal benefits by reducing harm to human health and the environment from the presence of chemicals in air, soil, drinking water, food, and consumer products. The Proposal implicates research that is central to making such determinations; creating an obstacle to the consideration of such science is contrary to both primary statutory directives and requirements to conduct cost benefit analyses. Likewise, EPA fails to address how it could implement a regulation that is inconsistent with federal requirements and standards governing the use of science across agencies.
This directive is embodied in multiple statutes implemented by EPA and is reflected in other environmental statutes, further illustrating Congress' conceptualization of "best" available science. Broadly speaking, there are three basic types of statutory requirements that the EPA and other agencies consider scientific information. First, there are requirements that the agency consider the "best available science." Second, there are requirements that the agency consider particular factors, including scientific factors. Third, there are requirements that the agency balance economic costs or technological feasibility against public health, environmental, or other benefits that must be scientifically assessed. The precise terminology regarding the required use of science may, as illustrated below, vary across statutes, but what these provisions have in common are requirements for EPA and other agencies to use scientific information that is "best," not simply adequate, and "available." 
The Proposal would also cut EPA off from information needed to complete the cost-benefit analysis of regulations required under certain statutes and government-wide requirements.
Not only would the Proposal impede EPA's fulfillment of its statutory duties, but it would also be contrary to judicial precedent, which holds that "best available science" includes "all existing scientific evidence relevant to the [agency] decision [in question]." Ecology Ctr., Inc. v. U.S. Forest Serv., 451 F.3d 1183, 1194 n.4 (10th Cir. 2006) (quoting Heartwood, Inc. v. U.S. Forest Serv., 380 F.3d 428, 436 (8th Cir. 2004)). While agencies need not produce new evidence, they must seek out information and "`cannot ignore existing data.'" Id. (emphasis added). Even if an agency is anticipating the arrival of better evidence, it must act on the basis of the evidence it currently has. Many of the fundamental public health studies on which EPA has based key rules and standards under the statutes are studies for which the raw data was not or could not have been released. Nonetheless these studies have been examined and validated, as will future studies. Arbitrarily cutting off consideration of a category of science is inconsistent with EPA's statutory obligations.
Comment [6168-4099]: Commenter (6178) raises the following concerns with the Transparency Rule proposed by the NPRM: 
1. The Agency cites specific sections of statutes it administers as the basis of the Agency's authority to adopt the Transparency Rule; specifically RCRA sections 2002(a)(1), 7009, 42 U.S.C. 6912(a)(1), 6979. The Agency rightly cites acts of Congress in an attempt to support the NPRM as such authority is necessary for the Agency to adopt regulations pursuant to RCRA.
2. However, Congress, in each of the Acts' sections cited, never instructs the Agency to consider transparency to the public of supporting data and/or models of the scientific research when deciding to rely on such research as support for regulations passed pursuant to any of the Acts. Consequently, the Transparency Rule requires the EPA to rely on factors which Congress has not intended it to consider when promulgating regulations pursuant to the Acts, rendering the Transparency Rule arbitrary and capricious under the APA. 
3. Further, not only does the Transparency Rule lack sufficient Congressional intent to avoid being deemed arbitrary and capricious, implementing the Rule would actually contradict Congressional instruction to conduct scientific research with a focus on `promotion' and `coordination' among itself and other institutions that produce scientific research material to the objectives of the Acts Congress requires the EPA to enforce. This provides additional cause for finding the Transparency Rule arbitrary and capricious under the APA.
[6163-1276]: Commenter (6133) states that in addition to addressing how and whether the agency will consider science, the Proposal also contains a seemingly unrelated provision regarding agency peer review. The Proposal, in § 30.7, reads: 

What role does independent peer review in this section?
EPA shall conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein. 

Because transparency in regulatory science includes addressing issues associated with assumptions used in models, EPA shall ask peer reviewers to articulate the strengths and weaknesses of EPA's justification for the assumptions applied and the implications of those assumptions for the results. 
 
83 Fed. Reg. at 18,774. 
 
There is no statutory authority for EPA to "conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein." The federal statutes that EPA lists as putative authority for the Proposal provide no authority for proposed § 30.7. See RCRA sections 2002(a)(1), 7009, 42 U.S.C. 6912(a)(1), 6979; CERCLA (as delegated to the Administrator via Executive Order 12580) sections 115, 311, 42 U.S.C. 9616, 9660. The claimed authorities that EPA lists do not mention peer review or even allude to the concept. Id. Neither the Proposal nor accompanying docket materials identify any provision of any federal statute that authorizes EPA to promulgate proposed § 30.7. Our own research revealed no provision of any federal statute that authorizes EPA to promulgate proposed § 30.7. 
 
When Congress writes federal statutes, Congress knows how to create legal authority for peer review, who shall conduct that peer review, what role, if any, that EPA or other parties will play, and how that peer review may be conducted. None of the provision in the law authorize EPA's Proposal in § 30.7. Instead, the statutes require that EPA use peer-reviewed science, regardless of whether it would meet EPA's definition of the term in § 30.7. And in cases where the law requires EPA to conduct the review, the statutes often spell out specifically how that should happen. The commenter lists several statutes. Only comments related to RCRA are excerpted here.
      RCRA 
      § 6939a. Exposure information and health assessments 
 
(b) Health assessments
      (2) Whenever in the judgment of the Administrator, or the State (in the case of a State with an authorized program), a landfill or a surface impoundment poses a substantial potential risk to human health, due to the existence of releases of hazardous constituents, the magnitude of contamination with hazardous constituents which may be the result of a release, or the magnitude of the population exposed to such release or contamination, the Administrator or the State (with the concurrence of the Administrator) may request the Administrator of the ATSDR to conduct a health assessment in connection with such facility and take other appropriate action with respect to such risks as authorized by section 9604(b) and (i) of this title. If funds are provided in connection with such request the Administrator of such Agency shall conduct such health assessment.
      [...]
      (e) Periodic reports The Administrator of such Agency shall issue periodic reports which include the results of all the assessments carried out under this section. Such assessments or other activities shall be reported after appropriate peer review. 
      42 U.S.C. § 6939a (emphasis added).
1.3.5.2 CERCLA/RCRA (SNPRM Comments)
Comment [48-442]: Commenter (12727) notes that the Supplemental Notice adds a reference to the CWA, and amends its references to the CERCLA and RCRA, but these additions provide no additional support or authorization for the proposal.
The references to CWA and CERCLA cite to the general authorities of the EPA Administrator to issue regulations to fulfill the respective requirements of the statutes. These authorities cannot authorize the proposal's efforts to violate those statutes or act beyond the scope of their requirements. Similarly, the cited provision of RCRA contains the Administrator's general authorities to promote research and training, offering no basis for the proposal's requirements or reconciliation of where it may conflict with RCRA's obligations -- even requirements in the very same subsection it presumes to cite for authority.
Comment [73-413]: Commenter (11576) contends Section 8001 of the RCRA does not allow the EPA to issue the proposed rule. Instead, the provision provides the agency with the authority:
 To conduct, and assist others in conducting, "research, investigations, experiments, training, demonstrations, surveys, public education programs, and studies" relating to solid-waste management;
 To establish a management program or system to coordinate activities relating to solid-waste research, and to facilitate and accelerate the development of new technology;
 To make grants to, and enter into contracts with, public agencies and authorities or private persons relating to solid-waste research; and
 To detail EPA personnel to agencies eligible for assistance under Section 8001.
Notably absent from the provision is any grant of authority for the EPA to fundamentally change its approach to reviewing scientific research by selectively disregarding or devaluing certain studies. The EPA has also failed to provide a citation to any particular paragraph or subsection of the provision, making it difficult to even understand what language the EPA believes does give it such authority. While Section 8001 does provide the EPA with authority to coordinate research activities, this is plainly intended to increase scientific knowledge in pertinent areas and ensure efficiency -- not to exclude scientific information from consideration or to provide substantive limits on the science that can be used by the agency at the expense of public health. The proposed rule is thus at odds with Section 8001, which accordingly provides no support for the agency's action.
Comment [73-414]: Commenter (11576) asserts that there is also no plausible reading of CERCLA Section 115 that would provide the EPA with the authority to limit the science it relies on. Section 115 provides that "[t]he President is authorized to delegate and assign any duties or powers imposed upon or assigned to ... [the President] and to promulgate any regulations necessary to carry out the provisions of this subchapter." This section accordingly allows the EPA -- to whom the President has delegated implementation of CERCLA -- to "promulgate any regulations necessary to carry out" the statute. The EPA has not shown that the proposed rule is in any way necessary to carry out CERCLA. Nor could it. In other words, the EPA has not identified any problems in implementing CERLCA that have resulted from relying on all available scientific knowledge.
Indeed, courts have rejected the view that such general rulemaking provisions authorize the promulgation of any regulation that is "`reasonably related to the purposes of the enabling legislation[,]'" noting that "[a]n agency's general rulemaking authority does not mean that the specific rule the agency promulgates is a valid exercise of that authority." A regulation promulgated under such a provision "cannot stand if it is arbitrary, capricious, or manifestly contrary to the statute." The agency may not use such a provision "to contravene Congress'
will[.]"
As explained elsewhere in these and previous comments, the proposed rule's restrictions on the use of scientific studies where underlying data is not available will prevent the EPA from considering sound science in its rulemakings and analyses. This is manifestly at odds with CERCLA's focus on protecting human health and the environment. Thus, the EPA cannot rely on Section 115 of CERCLA to support the proposal.
Comment [82-837]: Commenter (198105) asserts that none of the other statutory provisions identified by the Supplemental Notice authorize EPA to promulgate the Proposal. The Supplemental Notice asserts that RCRA Section 8001, 42 U.S.C. § 6981, CERCLA Section 115, 42 U.S.C. § 9615, and CWA Section 501, 33 U.S.C. § 1361, empower EPA to promulgate the Proposal. 85 Fed. Reg. at 15,397. However, there are two major problems with these assertions of statutory authority. First, the Supplemental Notice does not provide any explanation for how these provisions purportedly authorize the Proposal. Second, an examination of the cited provisions demonstrates that they provide no support for the Proposal; indeed, the Proposal would actively hinder, rather than promote, the statutory purposes of RCRA, CERCLA, and the CWA.
Comment [82-838]: Commenter (198105) asserts that the Supplemental Notice makes no effort at all to tie the Proposal to the statutory schemes of RCRA, CERCLA, or the CWA. Instead, it merely asserts that the cited provisions authorize the Proposal without any explanation of how the Proposal relates to the language or purposes of the three statutes. Additionally, both the Initial Proposal and the Supplemental Notice refer to a range of different provisions from different statutes. 83 Fed. Reg. at 18,769; 85 Fed. Reg. at 15,397. EPA must explain how distinct provisions in multiple statutes can authorize the agency to promulgate precisely the same rule for its review of science under all of these schemes, as well as how the Proposal would comply with each statutory scheme.
Comment [82-851]: Commenter (198105) provides that "[I]t is `axiomatic' that `administrative agencies may act only pursuant to authority delegated to them by Congress.'" None of the cited provisions, however, provide EPA with authority to ignore relevant scientific information. RCRA Section 8001 relates only to research and research funding, and the cited provisions of CERCLA and the CWA only authorize rulemakings that are necessary to achieve the statutes' broader purposes. The Proposal, however, is not related to EPA research or research funding, and will actively hinder EPA's ability to achieve the statutes' underlying goals by limiting EPA's ability to rely on the best available science.
EPA has long accepted that its decisions must be guided by the best available science. EPA even reiterated its commitment to this standard in the Initial Proposal. 83 Fed. Reg. at 18,769. However, the Proposal would directly undermine this longstanding commitment to use the best available science. As we pointed out in a comment letter in response to the Initial Proposal, the Proposal would prevent EPA from relying on crucial studies that underlie a number of past, present, and future regulations. This radical departure from EPA's standard practice cannot be justified by any of the statutes EPA identifies because it is contrary to the goals of those statutes.
Comment [82-852]: Commenter (198105) asserts that RCRA Section 8001, 42 U.S.C. § 6981, only empowers EPA to assist and conduct research; it says nothing about EPA's use of data in its regulatory capacity and certainly does not authorize EPA to limit its ability to rely on certain scientific data. In relevant part, RCRA Section 8001 grants EPA the authority to "conduct, and encourage, cooperate with, and render financial and other assistance" to institutions and individuals conducting research. 42 U.S.C. § 6981(a). The section also confers on EPA the authority to "establish a management program or system to insure the coordination of all such authorities," 42 U.S.C. § 6981(b), and to "make grants to or enter into contracts (including contracts for construction) with, public agencies and authorities or private persons" for the development of research. 42 U.S.C. § 6981(c).
None of these provisions empower EPA to issue a broad rulemaking whose primary effect will be to limit the agency's ability to rely on the best available science. Nothing in the statutory text purports to confer this power. On the contrary, the powers to "conduct," "encourage," "cooperate with," and assist in research efforts have nothing to do with the power to ignore research when acting in a regulatory capacity. The Supplemental Notice makes no attempt to connect the Proposal to the language of this provision or RCRA's statutory purposes.
Comment [82-855]: Commenter (195105) contends that section 115 of CERCLA, 42 U.S.C. § 9615, only authorizes the President, and by extension EPA, to promulgate regulations that are "necessary to carry out the provisions of this subchapter." 42 U.S.C. § 9615 (emphasis added). Section 115 does not authorize EPA to promulgate the Proposal for two related reasons.
First, the Supplemental Notice does not offer any reason why the Proposal is necessary to achieve CERCLA's purposes. Provisions in other environmental statutes with language identical to that in Section 115 have been interpreted to limit the Administrator's discretion in promulgating regulations. Similarly, the language of Section 115 of CERCLA is clear on its face: the provision only authorizes regulations that are necessary to achieve the statutory purposes of CERCLA. EPA, however, offers no justification for why the Proposal is necessary to achieve CERCLA's goals; indeed, it makes no effort to tie the Proposal to CERCLA's legislative purposes at all.
Second, Section 115 does not authorize EPA to promulgate the Proposal because it would hinder, rather than advance, CERCLA's underlying purposes. The discretion afforded to EPA under CERCLA only extends to actions that further the statute's goals, which include the "timely cleanup of hazardous waste sites" and ensuring that polluters are held financially responsible for cleanup efforts. In Eagle-Picher Industries, Inc. v. U.S. EPA, for example, the D.C. Circuit Court of Appeals upheld EPA's use of a particular model to determine priority sites only because the model was "reasonable and consistent with congressional intent."
Allowing EPA to limit its use of scientific data, however, would hinder rather than further CERCLA's purposes by precluding EPA from relying on studies merely because their underlying data and models cannot be made available to the public for privacy or confidentiality reasons. Indeed, courts have recognized that valid scientific data is crucial in promulgating regulations under CERCLA. In City of Stoughton, Wisconsin v. U.S. EPA, for example, the D.C. Circuit Court of Appeals underscored that EPA must rely on valid data as a basis for administrative action under CERCLA.
The Proposal, if finalized, would also undermine specific actions that EPA has considered taking under CERCLA. For example, at the time of the Initial Proposal in 2018, EPA was planning to designate two chemicals -- perfluorooctanoic acid (PFOA) and perfluorooctanoic sulfonate (PFOS) -- as hazardous chemicals under CERCLA. Then-Administrator Pruitt had announced that EPA viewed these contaminants as public health risks and intended to develop MCL for them. However, this determination was based on epidemiological studies generated, e.g., by the C8 Health Project, all of which included confidential human health data. The Proposal would severely weaken EPA's effort to promulgate MCL for PFOA and PFOS because it would limit the agency's ability to rely on studies whose underlying health data cannot be made publicly available.
Comment [89-2195]: Commenter (11403) states that EPA's 2018 version of this rule cited a hodgepodge of statutory authority, none of it valid. EPA at least corrected the obvious typographical errors from the previous version. However, those edits still do not grant EPA the authority to restrict science from all regulatory decision-making processes within the agency. EPA is attempting to use general grants of authority to create a completely novel approach to assessing science. However, as the Supreme Court has stated, "Congress . . . does not alter the fundamental details of a regulatory scheme in vague terms or ancillary provisions -- it does not, one might say, hide elephants in mouseholes." EPA does not have the authority to alter the fundamental details of all of its regulatory schemes based on such vague provisions of law.
Section 2002(a)(1) of the RCRA authorizes the Administrator to "prescribe, in consultation with Federal, State, and regional authorities, such regulations as are necessary to carry out his functions under this chapter. . ." This catch-all provision provides the authority to prescribe regulations, but EPA has not provided a single rationale explaining how the proposed rule is "necessary" to carrying out functions under the RCRA.
Section 8001 of the RCRA provides EPA with authority to conduct and assist with research relating to the handling of waste management and recovery. The section includes a provision stating, "the Administrator shall establish a management program or system to insure the coordination of all such activities and to facilitate the process of development of sound new technology (or other discoveries) from the research phase, through development, and into the demonstration phase." Not only does section 8001 not provide EPA with authority to promulgate this regulation, the requirement to facilitate the development of new science contradicts the proposed rule's restrictions on science. Limiting scientific research only serves to hinder development of new science, not facilitate it.
Section 115 of the CERCLA and Executive Order 12580 authorize the President to delegate duties under CERCLA and designate EPA as the statute's administrator. This delegation provides the authority to prescribe regulations, but EPA has not provided a single rationale explaining how the proposed rule is necessary to carrying out the functions under CERCLA.
Section 311 of the CERCLA directs the Administrator in the use and development of research and training tools. The section does not mention transparency of data or give any other indication that it authorizes EPA to promulgate the proposed rule.
Comment [133-1189]: Commenter (11894) states that EPA, for many years, has issued recommendations on the soil and groundwater purity levels to be attained in CERCLA cleanups. These rest on detailed and constantly updated literature reviews. In themselves, they have no direct legal effect, and they are very rarely absolutely legally binding. However, they undeniably shape regulatory decisions. Once again, EPA has not given any indication whether or not the issuance of these standards would be "agency action" exempt from the new requirements. Nor - once again - has EPA evaluated in any way the impact on the program or the resources needed to mitigate that impact.
Comment [159-1410]: Commenter (12430) contends that the SNPRM cites to one additional RCRA statute  -  42 U.S.C. section 6981 ("Section 6981"). Section 6981 consists of three subdivisions, the most relevant of which mandates that "[t]he Administrator shall establish a management program or system to insure the coordination of all such activities and to facilitate and accelerate the process of development of sound new technology (or other discoveries) from the research phase, through development, and into the demonstration phase." The SNPRM -- which would often screen out the best available science and which has alarmed leading scientific bodies -- would not "facilitate" or "accelerate" the advancement of "sound" science, for the reasons discussed throughout this comment.
Comment [159-1411]: Commenter (12430) notes that the SNPRM cites to an additional CERCLA provision  -  42 U.S.C. section 9615  -  that consists of a single statement of delegation: "The President is authorized to delegate and assign any duties or powers imposed upon or assigned to him and to promulgate any regulations necessary to carry out the provisions of this subchapter." The SNPRM does not identify any "duties or powers imposed upon or assigned to" the president under RCRA that would authorize U.S. EPA to depart from accepted practices in the scientific community and to exclude and/or deprioritize the best available scientific studies for arbitrary reasons.
Comment [187-1705]: Commenter (12720) contends that EPA also claims that 42 U.S.C. § 6912(a)(1) of the RCRA provides authority for the rule. But 42 U.S.C. § 6912(a)(1) merely states that the Administrator is authorized to "prescribe, in consultation with Federal, State, and regional authorities, such regulations as are necessary to carry out his functions under this chapter." As explained above, that general grant of authority cannot support the rule, especially when the rule conflicts with the Act. There is no support in RCRA for the Proposal or the Supplemental Proposal.
In the Supplemental Proposal, EPA states that its citation to 42 U.S.C. § 6979 is a mistake and should be corrected to Section 8001, 42 U.S.C. § 6981. Just as the previous incorrect citation failed to support the Proposal, this new citation fails to support the Proposal or Supplemental Proposal. As explained above, a general grant of authority cannot support the instant rulemaking, especially when the proposed text conflicts with the Act. Section 8001 outlines general authority under the statute to fund or conduct research, and to implement management programs. It does not remotely authorize the proposed rulemaking's various provisions, definitions, requirements, or exemptions The RCRA does not authorize the proposed rule.
Comment [187-1706]: Commenter (12720) provides that EPA states in the Supplemental Proposal that it finds authority under the CERCLA section 42 U.S.C. § 9615. The entirety of this section reads that "[t]he President is authorized to delegate and assign any duties or powers imposed upon or assigned to him and to promulgate any regulations necessary to carry out the provisions of this subchapter." As explained above, that general grant of authority cannot support the rule, especially when the rule conflicts with the Act. The CERCLA does not authorize the proposed rule.
EPA also cites 42 U.S.C. § 9660, which has many subsections. This broad citation also fails to provide sufficient notice for the public to comment on the proposed rule. Subsections (a), (b), and (c) require the Secretary of HHS and the Administrator of EPA to establish research programs on the effects of hazardous substances on human health. But nothing in those sections limits EPA's consideration of studies in which the data can be made public or draws a line between dose-response data and other types of data. The absence of any such congressional distinction or restriction makes it all the more obvious that EPA invented and added the distinction and restrictions in the Proposal and Supplemental Proposal as a matter of its own policy preferences, contrary to the Act. This EPA may not do. CERCLA does not authorize the rulemaking.
1.3.6 Emergency Planning and Community Right-to-Know Act (EPCRA) (NPRM and SNPRM Comments).
Comment: Sections 1.3.6.1 and 1.3.6.2 of this Chapter present comments received on the NPRM (Section 1.3.6.1) and SNPRM (Section 1.3.6.2) related to the EPA's legal authority for the rule under EPCRA. The EPA provides the following general response to these comments:
Response: The EPA appreciates your comments. As discussed in Section I.C of the final rule preamble and in this RTC document in Section 1.2, EPA has decided it is most appropriate to rely on its authority to promulgate housekeeping regulations, rather than any of the substantive environmental statutes cited in the 2018 proposal or the 2020 supplemental proposal as legal authority for the rule. EPA intends to promulgate future science transparency rules under its substantive environmental statutes. Additionally, the final rule states in § 30.3 that "[i]n the event the procedures outlined in this rule conflict with statutes that EPA administers, or their implementing regulations, the statutes and regulations will control."
None of the statutory provisions cited by the commenter prohibits EPA from conducting additional peer review of studies the agency evaluates as support for significant regulatory decisions and ISI. That said, the final rule provides that if EPA determines that studies have already undergone adequate peer review, EPA will not conduct de novo independent peer review. Adequate peer review is peer review that is consistent with both OMB's and EPA's peer review guidance. The intent of the rule is to encourage EPA to go beyond current OMB/EPA guidance by recognizing that peer review that actually examined the underlying data will lead to even more confidence in the results of a peer reviewed study and that pivotal science that has had this deeper peer review will be given greater consideration in study quality evaluation. 
1.3.6.1 EPCRA (NPRM Comments)
Comment [4827-2073]: Commenter (4838) writes that the proposed rule violates several environmental statutes because it hinders EPA's ability to rely on best available science, or the most up-to-date information, as they require. The CAA, CWA, TSCA, and EPCRA all require certain decisions or regulatory criteria be based on the most up-to-date science. These criteria are described as "best available science," "latest scientific knowledge," and "best available public health information." The proposed rule would illegally limit EPA's ability to rely on best available science in violation of these statutes.
Comment [6112-1555]: Commenter (6137) writes that EPA likewise relies upon a provision in the EPCRA that in no way authorizes this Proposed Rule. Specifically, Section 328 of EPCRA  -  upon which EPA relies  -  merely authorizes EPA to "prescribe such regulations as may be necessary to carry out" the statute. 42 U.S.C. § 11048. But this provision does not "empower[] [EPA] to establish regulations which run far afield from the substance of the Act." Kaw Valley, Inc. v. EPA 844 F. Supp. 705, 708 (D. Kan. 1994) (citing Central Forwarding, Inc. v. Interstate Commerce Comm'n, 698 F.2d 1266, 1277 (5th Cir. 1983)). Given that the Proposed Rule is contrary to the purposes of EPCRA, see infra, Section IV.A, this general rulemaking provision cannot be considered "necessary," and thus does not do the work that EPA ascribes to it. See Mourning, 411 U.S. at 369; Ragsdale, 535 U.S. at 86.
Comment [6112-1593]: Commenter (6137) writes that the Proposed Rule also violates EPCRA. EPCRA requires EPA to make determinations about whether to list new chemicals in the statute's Toxic Release Inventory (TRI) program "based on generally accepted scientific principles or laboratory tests, or appropriately designed and conducted epidemiological or other population studies, available to [EPA]." 42 U.S.C. § 11023(d)(2). Specifically, Congress instructs EPA to add a chemical to the TRI list when:
(A) The chemical is known to cause or can reasonably be anticipated to cause significant adverse acute human health effects at concentration levels that are reasonably likely to exist beyond facility site boundaries as a result of continuous, or frequently recurring, releases.
(B) The chemical is known to cause or can reasonably be anticipated to cause in humans -- 
      (i) cancer or teratogenic effects, or
      (ii) serious or irreversible -- 
      (I) reproductive dysfunctions,
      (II) neurological disorders,
      (III) heritable genetic mutations, or
      (IV) other chronic health effects.
(C) The chemical is known to cause or can reasonably be anticipated to cause, because of -- 
      (i) its toxicity,
      (ii) its toxicity and persistence in the environment, or
      (iii) its toxicity and tendency to bioaccumulate in the environment, a significant adverse effect on the environment of sufficient seriousness, in the judgment of the Administrator, to warrant reporting under this section.
Id. EPA thus has a mandate from Congress to consider the types of toxicological studies that EPA's Proposed Rule would prevent the Agency from considering. For this reason, the Proposed Rule cannot withstand scrutiny.
Comment [6118-1730]: Commenter (6143) notes that EPA has issued its proposed rule on Strengthening Transparency in Regulatory Science pursuant to its authority under several environmental statutes that that it implements, including the CAA; CWA; SDWA; RCRA; CERCLA; EPCRA; FIFRA; and TSCA. This proposed action is fully consistent with the authorities and objectives laid out in those statutes. EPA must ensure that any final action it takes on this proposal continues to respect the balance reflected in those statutes between sound, transparent science, and legitimate privacy interests.
Comment [6152-650]: Commenter (6122) writes that the proposed rule is not based on any valid statutory authority and is arbitrary and capricious.
EPA cites multiple statutory provisions in an attempt to support the proposed rule that it believes authorize this type of rulemaking, "including provisions providing general authority to promulgate regulations necessary to carry out the Agency's functions under these statutes and provisions specifically addressing the Agency's conducting of and reliance on scientific activity to inform those functions . . ." None of the specific statutory provisions provided by EPA provide specific authority for EPA to fundamentally rework the foundational processes it uses to assess and include/reject science. Indeed, if such authority existed, one would have expected EPA to have discovered this authority many decades earlier. Instead, EPA is attempting to use general grants of authority to create a completely novel approach to assessing science. However, as the Supreme Court has stated, "Congress...does not alter the fundamental details of a regulatory scheme in vague terms or ancillary provisions -- it does not, one might say, hide elephants in mouseholes." And if this is true, neither can the EPA alter the fundamental details of all of its regulatory schemes based on such vague provisions of law. We review each of the provisions listed by EPA, none of which discuss transparency, dose response, pivotal regulatory science, benefit-cost calculation, or changing "agency culture," the stated goals of this rulemaking.
Section 328 of the EPCRA states, "The Administrator may prescribe such regulations as may be necessary to carry out this chapter." This catch-all provision provides the authority to prescribe regulations, but EPA has not provided a single rationale explaining how the proposed rule is "necessary" to the carrying out functions under the EPCRA.
Comment [6163-1109]: Commenter (6133) notes that the only authority EPA cites under the EPCRA is 42 U.S.C. § 11048, which states that the "Administrator may prescribe such regulations as may be necessary to carry out this chapter." The citation fails to provide sufficient notice for the public to comment on the Proposal. EPA does not identify any statutory authority for why the proposed rule is necessary to carry out the chapter. As explained above, that general grant of authority cannot support the rule, especially when the rule conflicts with the Act. EPCRA does not authorize the proposed rule. 
Comment [6163-1140]: Commenter (6133) notes that EPCRA establishes requirements for state and local emergency planning and reporting on hazardous chemicals. It requires EPA to publish a list of extremely hazardous substances and set, by regulation, a threshold planning quantity for each substance on the list. 42 U.S.C. § 11002(a). "Any revisions to the list shall take into account the toxicity, reactivity, volatility, dispersibility, combustibility, or flammability of a substance." Id. § 11002(a)(4). Notably, in defining the criteria that EPA must consider for the list, EPCRA affirmatively directs EPA to consider the toxicity of the substance, among other things, and says nothing about excluding relevant studies for the reasons stated in the Proposal.
EPCRA also contains reporting requirements for owners or operators who manufacture, process, or use hazardous chemicals. Id. § 11023. EPA "may by rule add or delete a chemical from the list" of covered chemicals if there is sufficient evidence that the "chemical is known to cause or can reasonably be anticipated to cause significant adverse acute human health effects" or if the "chemical is known to cause or can reasonably be anticipated to cause in humans . . . cancer or teratogenic effects, or...serious or irreversible . . . reproductive dysfunctions[,] neurological disorders[,] heritable genetic mutations[,] other chronic health effects." Id. § 11023(d). A chemical can also be added if it "is known to cause or can reasonably be anticipated to cause . . . a significant adverse effect on the environment of sufficient seriousness" due to its toxicity. Id. Of critical importance here, this determination "shall be based on generally accepted scientific principles or laboratory tests, or appropriately designed and conducted epidemiological or other population studies, available to the Administrator." Id.
The Proposal directly conflicts with EPCRA's requirement to use "generally accepted scientific principles or laboratory tests," or "appropriately designed and conducted epidemiological or other population studies." See id. § 11023(d). As explained throughout these comments, there is no reason the underlying data must be public for these tests and studies to be "generally accepted" or "appropriated designed and conducted." Thus, the Proposal is -- on its face -- contrary to EPCRA's mandate that EPA use these tests and studies when making determinations under the statute.
Comment [6340-1458]: Commenter (6362) writes that the provisions cited by EPA under the CAA, the CWA, SDWA, CERCLA, and EPCRA in support of its authority to develop and implement its proposed rule all provide broad regulatory authority to promulgate regulations "as are necessary to carry out [the Administrator's] functions" under the statute. [...]
Comment [6894-3608]: Commenter (6915) writes that the EPCRA requires that a determination to add a chemical to the TRI "be based on generally accepted scientific principles or laboratory tests, or appropriately designed and conducted epidemiological or other population studies, available to the Administrator." § 313(d)(2), 42 U.S.C. § 11023(d)(2) (emphasis added).
Comment [6340-1457]: Commenter (6362) writes that the provisions cited by EPA under the CAA, the CWA, the SDWA, the CERCLA, and the EPCRA in support of its authority to develop and implement its proposed rule all provide broad regulatory authority to promulgate regulations "as are necessary to carry out [the Administrator's] functions" under the statute. [...]
Comment [6155-2632]: Commenter (6125) writes that it is clear from even a brief look that these provisions require EPA to consider all the best available science, without authorizing, or even mentioning any of the per se restrictions based on "transparency" that EPA has proposed to use as a universal overriding rule. For these reasons, EPA must justify its proposal as a proper exercise of its authority under each of the statutory provisions whose operation it would affect - provisions that the agency does not even cite. It is the job of the proposer, not the commenters, to survey these provisions and provide the needed justification.
      In several of the listed statutes Congress has explicitly told EPA how to use science in decision-making. For example, for EPCRA: EPCRA, 42 USC 11023(d)(2)(C) mandates that a determination whether to add a substance to list of "extremely hazardous substances" "shall be based on generally accepted scientific principles or laboratory tests or appropriately designed and conducted epidemiological or population studies available to the Administrator."
These provisions all use mandatory language directing EPA to use reliable and relevant scientific information - and further specify how to do so. They all direct that decisions regarding protection of public health be based on the full range of relevant scientific information, in some cases specifying how to identify what information to use. None of them give EPA discretion to ignore such information for any of the reasons set forth in the proposal.
Comment [6163-1276]: Commenter (6133) writes that the Proposal's peer review provision lacks any statutory basis, is vague and contrary to existing requirements for peer review.
In addition to addressing how and whether the agency will consider science, the Proposal also contains a seemingly unrelated provision regarding agency peer review. The Proposal, in § 30.7, reads: 
      What role does independent peer review in this section?
      EPA shall conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein. 
      Because transparency in regulatory science includes addressing issues associated with assumptions used in models, EPA shall ask peer reviewers to articulate the strengths and weaknesses of EPA's justification for the assumptions applied and the implications of those assumptions for the results. 
      83 Fed. Reg. at 18,774.
There is no statutory authority for EPA to "conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein." The federal statutes that EPA lists as putative authority for the Proposal provide no authority for proposed § 30.7. See 83 Fed. Reg. at 18,769/2 (citing CAA sections 103, 301(a), 42 U.S.C. 7403, 7601(a); CWA sections 104, 501, 33 U.S.C. 1254, 1361; SDWA sections 1442, 1450(a)(1), 42 U.S.C. 300j - 1, 300j -  9(a)(1); RCRA sections 2002(a)(1), 7009, 42 U.S.C. 6912(a)(1), 6979; CERCLA (as delegated to the Administrator via Executive Order 12580) sections 115, 311, 42 U.S.C. 9616, 9660; EPCRA section 328, 42 U.S.C. 11048; FIFRA sections 25(a)(1), 136r(a), 7 U.S.C. 136r(a), 136w; and TSCA, as amended, section 10, 15 U.S.C. 2609, and the APA). The claimed authorities that EPA lists do not mention peer review or even allude to the concept. Id. Neither the Proposal nor accompanying docket materials identify any provision of any federal statute that authorizes EPA to promulgate proposed § 30.7. Our own research revealed no provision of any federal statute that authorizes EPA to promulgate proposed § 30.7. 
When Congress writes federal statutes, Congress knows how to create legal authority for peer review, who shall conduct that peer review, what role, if any, that EPA or other parties will play, and how that peer review may be conducted. None of the provision in the law authorize EPA's Proposal in § 30.7. Instead, the statutes require that EPA use peer-reviewed science, regardless of whether it would meet EPA's definition of the term in § 30.7. And in cases where the law requires EPA to conduct the review, the statutes often spell out specifically how that should happen. EPCRA does not mention peer review.
The EPA approach proposed in § 30.7 is even more unlawful than would be the case, independently, under this case law. Proposed § 30.7 says "EPA shall conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein." 83 Fed. Reg. at 18,774/2 (emphasis added). EPA proposes to bind itself ("shall") to conduct a particular form of peer review on "all pivotal regulatory science" based on an unenforceable, non-binding OMB bulletin that has never been the subject of notice and comment rulemaking and that itself is not authorized by any federal law. To the contrary, treating the content of the unenforceable bulletin as a binding regulation would itself violate the federal statutes that EPA implements, because those statutes do not codify the bulletin, and EPA would be unlawfully codifying a mere policy preference. This EPA may not do. 
This Proposal is unlawful, arbitrary, and capricious, and an abuse of EPA discretion. To reiterate, the Proposal identifies no statutory authority for EPA to conduct independent peer review for any, much less all, "pivotal regulatory science" consistent with the dense content (and exceptions) of the OMB bulletin. The Proposal identifies no statutory authority for EPA to bind itself, and future administrations, to conduct peer review only in this fashion, unless and until future notice-and-comment rulemaking is undertaken. The Proposal identifies no suggestion in statutory language or legislative history that Congress intended EPA to conduct binding peer review consistent with this OMB bulletin, notwithstanding that Congress has known about this bulletin since 2005. EPA has simply made up proposed § 30.7 -- with its the link to the OMB bulletin, and the putative authority for the Proposal -- out of whole cloth. This EPA may not do. 
1.3.6.2 EPCRA (SNPRM Comments)
Comment [81-2190]: Commenter (12715) writes that the proposed rule conflicts with fundamental statutory requirements to use the best available science and citation to "housekeeping" authority cannot cure that defect.
Whether EPA is arbitrarily barring consideration of scientific studies or information with nonpublic underlying data or data that are not able to be independently validated, or arbitrarily ascribing less weight to such studies or information, neither approach constitutes "housekeeping," and neither comports with statutory requirements that EPA use the best available science. To reiterate from the 2018 Comments, EPA's obligation with respect to the use of scientific information is clear: it must base its decisions on such criteria as the latest scientific knowledge, the best available, peer-reviewed science, and/or generally accepted scientific principles or laboratory tests. To cite just a few examples, in performing its duties, EPA must rely on: "the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices," SDWA of 1996, 42 U.S.C. § 300g-1(b)(3)(A)(i); the "best available science," TSCA, 15 U.S.C. § 2625(h); "the latest scientific knowledge," CWA of 1977, 33 U.S.C. § 1314(a)(1), and CAA of 1970, 42 U.S.C. § 7408(a)(2); and "generally accepted scientific principles or laboratory tests, or appropriately designed and conducted epidemiological or other population studies," EPCRA, 42 U.S.C. § 11023(d)(2). EPA has repeatedly emphasized the importance of this fundamental precept, including in its 2018-2022 strategic plan, which states that one of the agency's priorities is to "identify, assess, conduct, and apply the best available science to address current and future environmental hazards."
These statutory requirements apply to all aspects of EPA's decision-making, including its scientific evaluations, adoption of regulatory standards, and development of policies. They do not permit what EPA proposes here: to either exclude or give less weight to scientific studies or information based on criteria -- availability of the underlying data and ability to independently validate underlying data -- that are not determinative of whether the studies or information constitute the best available science. Agencies may not adopt or implement regulations that conflict with the statutes under which they are promulgated, and an agency's interpretation of those statutes must always at least be reasonable. See Chevron, U.S.A., Inc. v. Nat. Res. Def. Council, Inc., 467 U.S. 837, 842-44 (1984). Furthermore, neither 5 U.S.C. § 301 nor any other purported housekeeping authority allows EPA to negate these statutes' fundamental, substantive, and common-sense commands. And even assuming EPA has some inherent housekeeping authority, it cannot rely on a general grant of rulemaking authority to promulgate regulations or develop policies that are otherwise inconsistent with more specific statutory directives. Glob. Van Lines, Inc. v. Interstate Commerce Comm'n, 714 F.2d 1290, 1293-97 (5th Cir. 1983).
Finally, EPA acknowledges that the statutes it administers would control in the event of a conflict between the proposed rule and those statutes, see 85 Fed. Reg. at 15,398, but that caveat cannot save the proposed rule. First, the core principle of the proposed rule is facially invalid, i.e., there is no circumstance in which it would be acceptable for EPA to exclude or give less weight to relevant, probative scientific studies, models, or other information that have been validated through peer review, on the sole basis that the underlying data are not publicly available or are not able to be independently validated. EPA cannot save an invalid rule by including what amounts to a waiver procedure, because the "essence of waiver is the assumed validity of the general rule." Alltel Corp. v. Fed. Commc'n Comm'n, 838 F.2d 551, 561 (D.C. Cir. 1988). Second, the agency's essential duty in rulemaking is to exercise its subject matter expertise to enact rules that implement and further the purposes of the statutes Congress has assigned it to administer. Chevron, 467 U.S. at 842-44. This does not include proposing rules without first determining the source of legal authority but instead asking commenters to supply that information. Third, the agency must propose its rules with specificity and be clear as to the programs, regulations, and information to which they will apply. 5 U.S.C. § 553(b)(3); Home Box Office, Inc., supra, 567 F.2d at 35-36.
Comment [89-2196]: Commenter (11403) writes that the proposed rule is not based on any valid statutory authority and is arbitrary and capricious.
a. The environmental statutory provisions cited by EPA do not authorize the proposed rule
EPA's 2018 version of this rule cited a hodgepodge of statutory authority, none of it valid. EPA at least corrected the obvious typographical errors from the previous version. However those edits still do not grant EPA the authority to restrict science from all regulatory decision-making processes within the agency. EPA is attempting to use general grants of authority to create a completely novel approach to assessing science. However, as the Supreme Court has stated, "Congress . . . does not alter the fundamental details of a regulatory scheme in vague terms or ancillary provisions -- it does not, one might say, hide elephants in mouseholes." EPA does not have the authority to alter the fundamental details of all of its regulatory schemes based on such vague provisions of law.
Section 328 of the EPCRA states "[t]he Administrator may prescribe such regulations as may be necessary to carry out this chapter." This catch-all provision provides the authority to prescribe regulations, but EPA has not provided a single rationale explaining how the proposed rule is "necessary" to carrying out functions under the EPCRA.
In short, none of the statutory provisions cited by EPA provide authority for the proposed rule. The proposed rule fails at step one under Chevron v. Natural Resources Defense Council since it is unreasonable to believe that any statutory general grant of authority or any mention of EPA's oversight of other science or research programs would enable EPA to conduct rulemaking in a way that is altogether inconsistent with the rule of law. None of the cited provisions provide the authority sought by EPA, and as such the proposed rule should be withdrawn.
Comment [125-2204]: Commenter (11495) writes that EPA's Supplemental Notice is at odds with provisions of multiple environmental statutes which require the Agency to consider certain high-quality scientific information. For example, the SDWA requires that the issuance of national drinking water regulations be based on the "best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices." TSCA requires the EPA Administrator to ensure that evaluation and regulation of toxic substances is "consistent with the best available science" and to use all scientific information that is "reasonably available" when making related decisions. The CWA and the CAA require that water and air quality criteria determinations be based on the "latest scientific knowledge." Lastly, EPCRA requires that some actions "be based on generally accepted scientific principles or laboratory tests, or appropriately designed and conducted epidemiological or other population studies."
Each of these provisions directs EPA to incorporate a certain quality of scientific information into its rulemaking decisions made pursuant to the statute. In particular, these requirements are designed to ensure the scientific validity of the information upon which EPA bases its actions. Mandating a high-quality scientific basis for regulatory actions is necessary to guarantee the best possible protections for public health and welfare and to minimize partisan influence in scientific decision-making.
The Supplemental Notice, however, would impermissibly prohibit the agency from considering of relevant and high-quality scientific studies whenever the data underlying the study are not publicly available -- in direct contravention of the statutory directives that mandate, for example, consideration of the "best available science." None of the provisions requires, or even suggests, that EPA only consider information that is publicly available. It strains the plain meaning of words like "best," "latest," and even "appropriate" to suggest that they permit the unscientific requirement that all data be publicly available.
EPA "cannot ignore available . . . information." Courts have held that "best available data" requirements "prohibit[] [an agency] from disregarding available scientific evidence that is in some way better than the evidence [it] relies on." Furthermore, a plaintiff may establish a violation of a "best available science" requirement by "point[ing] to any scientific evidence that the agency failed to consider." The Supplemental Notice would undoubtedly preclude EPA from considering certain studies and scientific information that is "better" or the "best available," simply because the study's underlying data is not publicly available, thereby failing to satisfy Congressional directives and undermining the quality of regulatory decisions.
Likewise, EPA cannot avoid the statutory directives to consider high-quality scientific information simply by assigning a lesser weight to such information when the underlying data are not publicly available, as the Agency alternatively proposes. The Supplemental Notice's alternative proposal is designed to have largely the same pernicious effects as flatly prohibiting consideration of studies lacking publicly available data. Just as with the primary proposal, EPA's alternative proposal would limit the agency's consideration of the best available science in making important regulatory decisions, and is, therefore, arbitrary, and capricious.
The effect of the Proposed Rule is inconsistent with EPA's view that the Supplemental Notice is "intended to be consistent with the statutes that EPA administers and EPA plans to implement this procedural rulemaking in accordance with all applicable statutory and regulatory requirements." EPA attempts to resolve this inconsistency by stating that "in the event the procedures outlined in the proposed rulemaking conflict with the statutes that EPA administers, or their implementing regulations, the statutes and regulations will control." However, as discussed below, EPA lacks the statutory authority to issue a rule this broad, even when it claims that it will avoid statutory conflicts. 
Comment [187-1707]: Commenter (12720) writes that the only authority EPA cites under the EPCRA is 42 U.S.C. § 11048, which states that the "Administrator may prescribe such regulations as may be necessary to carry out this chapter." The citation fails to provide sufficient notice for the public to comment on the Proposal or Supplemental Proposal. EPA does not identify any statutory authority for why the proposed rule is necessary to carry out the chapter. As explained above, that general grant of authority cannot support the rule, especially when the rule conflicts with the Act. EPCRA does not authorize the proposed rule.
1.3.7 Clean Water Act (CWA) (NPRM and SNPRM comments).
Comment: Sections 1.3.7.1 and 1.3.7.2 of this Chapter present comments received on the NPRM (Section 1.3.7.1) and SNPRM (Section 1.3.7.2) related to the EPA's legal authority for the rule under the CWA. The EPA provides the following general response to these comments:
Response: EPA appreciates your comments. As discussed in Section I.C of the final rule preamble and in this RTC document in Section 1.2, EPA has decided it is most appropriate to rely on its authority to promulgate housekeeping regulations, rather than any of the substantive environmental statutes cited in the 2018 proposal or the 2020 supplemental proposal as legal authority for the rule. EPA intends to promulgate future science transparency rules under its substantive environmental statutes. Additionally, the final rule states in § 30.3 that "[i]n the event the procedures outlined in this rule conflict with statutes that EPA administers, or their implementing regulations, the statutes and regulations will control."
1.3.7.1 CWA (NPRM Comments)
Comment [4827-2072]: Commenter (4838) writes that the proposed rule violates several environmental statutes because it hinders EPA's ability to rely on best available science, or the most up-to-date information, as they require. The CAA, CWA, SDWA, TSCA, And EPCRA all require certain decisions or regulatory criteria be based on the most up-to-date science. These criteria are described as "best available science," "latest scientific knowledge," and "best available public health information." The proposed rule would illegally limit EPA's ability to rely on best available science in violation of these statutes.
Comment [6112-1551]: Commenter (6137) notes that EPA cites sections 104 and 501 of the CWA, 33 U.S.C. §§ 1254, 1361, as statutory authority for its Proposed Rule. Upon examination, these sections do not provide the authority EPA suggests.
Section 104, 33 U.S.C. § 1254, entitled "Research, investigations, training, and information," addresses the Administrator's authority as it relates to the establishment of national programs, cooperation, investigations, water quality surveillance system, and reports. It requires the Administrator to, among other things: "conduct and promote the coordination and acceleration of, research, investigations, experiments, training, demonstrations, surveys, and studies relating to the causes, effects, extent, prevention, reduction, and elimination of pollution"; and to "initiate and promote the coordination and acceleration of research designed to develop the most effective practicable tools and techniques for measuring the social and economic costs and benefits of activities which are subject to regulation under this chapter." 33 U.S.C. § 1254(a)(1), (6). 
Toward that end, the provision authorizes the Administrator to: "collect and make available, through publications and other appropriate means, the results of and other information, including appropriate recommendations by him in connection therewith, pertaining to such research and other activities referred to in" 1254(a)(1); and "cooperate with other Federal departments and agencies, State water pollution control agencies, interstate agencies, other public and private agencies, institutions, organizations, industries involved, and individuals, in the preparation and conduct of such research and other activities referred to in" 1254(a)(1). Id. § 1254(b)(1), (2). It also requires the Administrator to "conduct research on, and survey the results of other scientific studies on, the harmful effects on the health or welfare of persons caused by pollutants." Id. § 1254(c). And it requires the Administrator to conduct and update a variety of studies, including, but not limited to, studies on oil pollution controls, id. § 1254(i), effects and control of pesticides in water, id. § 1254(l), waste oil disposal, id. § 1254(m), effects of pollution on estuaries and estuarine zones, id. § 1254(n), pollution from agriculture, id. § 1254(p), and effects and methods of controlling thermal discharges, id. § 1254(t).
Thus, section 1254 discusses research and studies in great detail, but it does so by setting forth requirements for cooperation and promotion of research. This section does not grant EPA any rulemaking authority at all, nor does it say anything about the Administrator's ability to screen or otherwise define the parameters for research that EPA can rely on for regulatory purposes. Instead, it describes the different areas for research and study and requires the Administrator to conduct research and studies in these areas.
The second CWA provision upon which EPA relies fares no better. EPA cites to section 501, 33 U.S.C. § 1361, as additional statutory authority for this rule. This is the provision generally authorizing the Administrator to "prescribe such regulations as are necessary to carry out his functions under this chapter." 33 U.S.C. § 1361(a) (emphasis added). Such authority only exists if the regulation is, in fact, "necessary to carry out" the provisions under the CWA. Mourning v. Family Publ'n Serv., Inc., 411 U.S. 356, 369 (1973) ("[w]here the empowering provision of a statute states simply that the agency may `make . . . such rules and regulations as may be necessary to carry out the provisions of [an] Act,' . . . the validity of a regulation promulgated thereunder will be sustained [only] so long as it is `reasonably related to the purposes of the enabling legislation.'" (citation omitted)). The Proposed Rule is decidedly not necessary at all. As with the CAA discussed supra, the CWA regulatory authority enables the Agency to carry out its functions and fill any statutory gaps. The Proposed Rule is not needed to fill any "gaps" in the CWA, as Congress has already provided  -  in great detail  -  the Administrator's regulatory authority as it relates to research, emphasizing the need for the use and promotion of inclusive research. See § 1254. Moreover, a rule "devised pursuant to Congress' directive to issue regulations `necessary to carry out' [an] Act . . . cannot stand if it is `arbitrary, capricious, or manifestly contrary to the statute.'" Ragsdale v. Wolverine World Wide, Inc., 535 U.S. 81, 86 (2002) (citations omitted); see also Am. Petroleum Inst., 52 F.3d at 1119 ("EPA cannot rely on its general authority to make rules necessary to carry out its functions when a specific statutory directive defines the relevant functions of EPA in a particular area."). As discussed in Section IV.A of the commenter's comment letter, not only is the regulation unnecessary to carry out EPA's functions under the CWA or to fill any gaps, but it is arbitrary and antithetical to the objectives of the CWA. Thus, the Proposed Rule is not authorized under this general rulemaking provision.
Comment [6112-1588]: Commenter (6137) argues that the Proposed Rule violates the CWA in two ways: first, it runs afoul of its requirement to use all relevant science and the best technology available; and second, it undermines its mandate to protect public health. For each of these reasons, the rule cannot stand. 
First, pursuant to Section 1251 of the CWA, the primary objective of the CWA is "to restore and maintain the chemical, physical, and biological integrity of the Nation's waters." 33 U.S.C. § 1251(a). Toward that end, a fundamental policy underlying the CWA is "to support and aid research relating to the prevention, reduction, and elimination of pollution." Id. § 1251(b). The CWA thus promotes the use of good science. The Proposed Rule handicaps EPA from accomplishing these broad goals and objectives by limiting the available science and research.
For example, section 1313(c) of the CWA governs the establishment and modification of water quality standards. Pursuant to this provision, these standards "shall be such as to protect the public health or welfare, enhance the quality of water and serve the purposes of" the CWA. Id. § 1313(c)(2)(A). In setting these standards, EPA must "tak[e] into consideration their use and value for public water supplies, propagation of fish and wildlife, recreational purposes, and agricultural, industrial, and other purposes, and also taking into consideration their use and value for navigation." Id. If EPA limits the type of science acceptable for these purposes, it is not fulfilling this obligation of the CWA as it is not using all means to accomplish this requirement.
Similarly, section 1313(d) requires states to establish "the total maximum daily load" for certain identified pollutants, and it must do so "at a level necessary to implement the applicable water quality standards with seasonal variations and a margin of safety which takes into account any lack of knowledge concerning the relationship between effluent limitations and water quality." Id. § 1313(d)(1)(C). As with Section 1313(c), if EPA is required to do this without all available science, it is not fulfilling its obligations under this provision. 
Not only do the limitations on science undermine the Agency's ability to most effectively fulfill the obligations of the CWA, but they likewise contradict provisions in the Act that require use of the best technology available. For example, section 1311(p)(1) requires that any modified requirements of effluent limitations in certain permits apply "the best available technology economically achievable." Failure to do so will render the effluent limitations invalid. See, e.g., Nat. Res. Def. Council v. EPA, 808 F.3d 556, 564 (2d Cir. 2015). Several other provisions likewise require the use of the best technology to carry out the purpose of the Act. See, e.g., 33 U.S.C. § 1314(b)(1) (regulations establishing or revising effluent limitations must apply "the best practicable control technology currently available" to identify "the degree of effluent reduction attainable"); id. § 1314(b)(2)(A) (regulations establishing or revising effluent limitations must apply "the best control measures and practices achievable" to identify "the degree of effluent reduction attainable . . . including treatment techniques, process and procedure innovations, operating methods, and other alternatives for classes and categories of point sources")); id. § 1314(b)(4)(A) (regulations establishing or revising effluent limitations must apply "the best conventional pollutant control technology" to identify "the degree of effluent reduction attainable . . . for classes and categories of point sources"). 
Second, the CWA also contains several additional provisions that demonstrate its overarching goal of protecting the public health. For example, one of the provisions in section 1254  -  one of the two sections cited by EPA as authorizing this Proposed Rule  -  addresses the "collection and dissemination of scientific knowledge on the effects and control of pesticides in water." 33 U.S.C. § 1254(l). Pursuant to this provision, the Administrator is charged with developing and issuing to the States for the purpose of carrying out the CWA "the latest scientific knowledge available in indicating the kind and extent of effects on health and welfare which may be expected from the presence of pesticides in the water in varying quantities," and updating that information "whenever necessary to reflect developing scientific knowledge." Id. There are no qualifications or other limiting factors in the type of scientific knowledge that must be considered. Rather, the provision contemplates inclusiveness to most effectively accomplish the CWA's objectives.
Several other provisions likewise address science and research as they relate to the public health goals of the CWA. For example, section 1254a requires the Administrator to "conduct research on the harmful effects on the health and welfare of persons caused by pollutants in water." A provision addressing protection of the Great Lakes states, in part, that "[t]he Administrator may not carry out a project under this paragraph for remediation of contaminated sediments located in an area of concern --  (i) if an evaluation of remedial alternatives for the area of concern has not been conducted, including a review of the short-term and long-term effects of the alternatives on human health and the environment." See 33 U.S.C. § 1268(c)(11)(D). Section 1311(g)(2), which addresses requirements for modifications to effluent limitations, requires that such modifications not result in "the discharge of pollutants in quantities which may reasonably be anticipated to pose an unacceptable risk to human health or the environment." And section 1314(a)(9) provides that the Administrator "shall publish new or revised water quality criteria for pathogens and pathogen indicators (including a revised list of testing methods, as appropriate), based on the results of the studies conducted under section 1254(v)," for the purpose of protecting human health in coastal recreation waters, and that at least once every five years, the Administrator must review and if necessary revise the water quality criteria. And EPA has long recognized that the National Pollutant Discharge Elimination System (NPDES) (§ 402 permit) and fill discharges (§ 404 permit) programs require protection of public health. See, e.g., 33 U.S.C. §§ 1342, 1344 (authorizing EPA to prohibit, withdraw, or veto a discharge that "will have an unacceptable adverse effect on municipal water supplies, shellfish beds and fishery areas . . . wildlife, or recreational areas") and implementing regulations, including Section 404(b)(1) Guidelines (40 C.F.R. Part 230).
Taken together, these provisions demonstrate that the overall focus of the CWA is to promote and protect the public health and water quality in the most comprehensive way possible. Thus, any measure that could limit science or research supporting these objectives is antithetical to the Act. For these reasons, EPA's reliance on the CWA to support its restrictions on science is entirely misplaced.
Comment [6118-1727]: Commenter (6143) notes that EPA has issued its proposed rule on Strengthening Transparency in Regulatory Science pursuant to its authority under several environmental statutes that that it implements, including the CAA; CWA; SDWA; RCRA; CERCLA; EPCRA; FIFRA; and TSCA. This proposed action is fully consistent with the authorities and objectives laid out in those statutes. EPA must ensure that any final action it takes on this proposal continues to respect the balance reflected in those statutes between sound, transparent science, and legitimate privacy interests.
Comment [6155-590]: Commenter (6125) writes that when Congress wished to limit available science to information that is publicly available, it has done so. See CWA Section 311(a)(27), 33 USC 1321(a)(27) (for purposes of addressing oil and hazardous substance liability, "best available science" is defined as evidence that "uses peer reviewed and publicly available data"). Its failure to impose such limits on the use of data or science in the provisions discussed above is thus purposeful. As noted, the proposal does not even mention any of these relevant statutory requirements governing the use of science or data or information, much less explain how any of them could authorize EPA to make a decision without using information that is relevant to decision-making, solely for lack of transparency and regardless of reliability or relevance. None of these statutes provide EPA with any authority to pick and choose what reliable and useful science to use or ignore specific studies based on transparency. While the proposal misleadingly pays lip service to using the best available science, it would expressly "preclude" the use of reliable and available science that is not, in its characterization, "transparent," regardless of whether it is the best available science. The proposal does not even mention the governing statutory requirements, much less explain how its approach is consistent with them.
Comment [6155-2633]: Commenter (6125) argues that it is clear from even a brief look that these provisions require EPA to consider all the best available science, without authorizing, or even mentioning any of the per se restrictions based on "transparency" that EPA has proposed to use as a universal overriding rule. For these reasons, EPA must justify its proposal as a proper exercise of its authority under each of the statutory provisions whose operation it would affect - provisions that the agency does not even cite. It is the job of the proposer, not the commenters, to survey these provisions and provide the needed justification.
In several of the listed statutes Congress has explicitly told EPA how to use science in decision-making. Specifically, for the CWA:
 CWA, 33 USC 1314(a)(1) mandates that the Administrator "shall develop and publish ... criteria for water quality accurately reflecting the latest scientific knowledge" "on the kind and extent of all identifiable effects on health or welfare" in setting water quality standards; 
These provisions all use mandatory language directing EPA to use reliable and relevant scientific information -- and further specify how to do so. They all direct that decisions regarding protection of public health be based on the full range of relevant scientific information, in some cases specifying how to identify what information to use. None of them give EPA discretion to ignore such information for any of the reasons set forth in the proposal.
Comment [6163-1104]: Commenter (6133) notes that EPA cites sections 104, 33 U.S.C. § 1254, and 501, 33 U.S.C. § 1361, of the CWA as putative authority for the Proposal. Nothing in these sections authorize the Proposal's limitations on scientific evidence. 
With respect to section 104, the Proposal tellingly fails to specify which of its 22 subsections supposedly supports the restrictions EPA has proposed. This deficiency reflects a lack of authority for the Proposal in section 104. And even if EPA thinks that it can cobble together language in section 104 to support the Proposal, the agency's complete failure to identify in the Proposal how section 104 authorizes this rulemaking means that EPA did not provide sufficient notice for the public to comment on the Proposal.
None of the subsections in section 104 states or suggests that, in promulgating regulations under the CWA, EPA may limit its consideration of "regulatory science underlying its actions" only to studies or analyses "are publicly available in a manner sufficient for independent validation." See 83 Fed. Reg. at 18,773 (proposed § 30.5). To the contrary, several subsections indicate that Congress intended EPA to consider available scientific evidence in order to carry out the Act.
First, subsection (b) authorizes EPA to collect and publicize results and information related to studies about water pollution but does not say anything about limiting consideration of science simply because data cannot be made public, either as part of rulemakings or otherwise. Nor does it draw any distinction between dose-response data and other types of data. 
Second, subsection (c) directs EPA to "conduct research on, and survey the results of other scientific studies on, the harmful effects on the health or welfare of persons caused by pollutants." It provides no authority whatsoever for limiting consideration of studies, models or data, dose-response or otherwise, during rulemakings; indeed, by directing EPA to "survey the results of other scientific studies," rather than the publicly-available dose-response data underlying those results, this subsection contradicts the Proposal's limitations and conditions.
Third, subsection (l)(1) indicates that EPA should be inclusive with respect to considering evidence, as it directs EPA to "develop and issue to the States for the purpose of carrying out this Act the latest scientific knowledge available in indicating the kind and extent of effects on health and welfare which may be expected from the presence of pesticides in the water in varying quantities. He shall revise and add to such information whenever necessary to reflect developing scientific knowledge."
Fourth, subsection (n) directs EPA to cooperate with various entities to "conduct and promote, encourage contributions to, continuing comprehensive studies of the effects of pollution, including sedimentation, in the estuaries and estuarine zones of the United States on fish and wildlife, on sport and commercial fishing, on recreation, on water supply and water power, and on other beneficial purposes." Importantly, subsection (n)(2) reveals Congress's intention that EPA will consider information broadly, by instructing the agency to "assemble, coordinate, and organize all existing pertinent information on the Nation's estuaries and estuarine zones . . ."
EPA also cites 33 U.S.C. § 1361 as a basis for the Proposal, but it does not provide the agency with the authority it desires. Subsection (a) merely states that the "Administrator is authorized to prescribe such regulations as are necessary to carry out his functions under this chapter." As explained above, that general grant of authority cannot support the rule, especially when the rule conflicts with the Act. Moreover, EPA casually invokes this provision, but does not make any effort to justify the proposed restrictions as necessary to any particular CWA statutory function, so it has not made the case that this provision provides authority to adopt the Proposal's limits.
Finally, the Act contains other indications that Congress intended EPA's consideration of science to be inclusive. In particular, section 304(a)(1) of the Act states:
      The Administrator, after consultation with appropriate Federal and State agencies and other interested persons, shall develop and publish, within one year after the date of enactment of this title (and from time to time thereafter revise) criteria for water quality accurately reflecting the latest scientific knowledge (A) on the kind and extent of all identifiable effects on health and welfare including, but not limited to, plankton, fish, shellfish, wildlife, plant life, shorelines, beaches, esthetics, and recreation which may be expected from the presence of pollutants in any body of water, including ground water; (B) on the concentration and dispersal of pollutants, or their byproducts, through biological, physical, and chemical processes; and (C) on the effects of pollutants on biological community diversity, productivity, and stability, including information on the factors affecting rates of eutrophication and rates of organic and inorganic sedimentation for varying types of receiving waters.
Although water quality criteria EPA develops are not issued as regulations, such that the Proposal as written would likely not apply to them, the salient point -- illustrated by the italicized language above -- is that Congress refused to limit EPA's consideration of available evidence in discharging one of its core functions aimed at protecting the nation's waters. EPA provides no reason in the Proposal why the regulations the Proposal targets should be any different.
Accordingly, the CWA does not authorize the Proposal. 
Comment [6163-1135]: Commenter (6133) argues that the Proposal, if adopted, would imperil the effective implementation of the CWA. Several provisions of the Act direct EPA to consider a range of data in promulgating regulations to effectuate its goals, and the development of these regulations would be hamstrung by the Proposal's restrictions on considering valid scientific evidence. As discussed in these comments, identifying, and excluding valid scientific evidence is time- and resource-intensive and has not been demonstrated to improve the quality of the science EPA considers or its science-based decisions. Accordingly, applying the proposed limitations to the myriad of regulatory decisions the agency is supposed to make would be a recipe for complete paralysis on multiple fronts under the CWA. Some examples of the water regulations that could be adversely affected by the far-reaching the Proposal follow. 
Under sections 301 and 304, EPA must develop effluent limitation guidelines, setting out nationally applicable pollution discharge standards for various industries. These Effluent Guidelines Plan (ELG) "identify, in terms of amounts of constituents and chemical, physical, and biological characteristics of pollutants, the degree of effluent reduction attainable through the application of [particular levels of pollution control stringency] for classes and categories of point sources . . . ." 33 U.S.C. § 1314(b)(1)(A). EPA is to specify the factors used to determine the controls to be used, including "the age of equipment and facilities involved, the process employed, the engineering aspects of the application of various types of control techniques, process changes, the cost of achieving such effluent reduction, non-water quality environmental impact (including energy requirements), and such other factors as the Administrator deems appropriate . . . ." Id. § 1314(b)(1)(B). Making these judgments and formulating the proper control levels that industrial dischargers must meet will obviously depend on data collected about the processes used in a given industry, control technology performance, cost, and energy use, among other things. 
Under section 303, the Act charges EPA with issuing initial water quality standards for states that fail to submit their own, and with developing such standards if EPA determines submitted standards are not consistent with the Act. Id. § 1313(b). Congress required these standards to take account of a wide range of evidence, and the Proposal would therefore curtail EPA's actions pursuant to the Act. Specifically, standards:
      shall be such as to protect the public health or welfare, enhance the quality of water and serve the purposes of this chapter. Such standards shall be established taking into consideration their use and value for public water supplies, propagation of fish and wildlife, recreational purposes, and agricultural, industrial, and other purposes, and also taking into consideration their use and value for navigation.
      Id. § 1313(c)(2)(A).
In addition, section 303's water quality standards process illustrates a hypocritical element of the Proposal. When states develop water quality standards, they must submit to EPA "[g]eneral information which will aid the Agency in determining the adequacy of the scientific basis of the standards," 40 C.F.R. § 131.6(f), and EPA's review of such a submission considers "[w]hether the State standards . . . are based upon appropriate technical and scientific data and analyses," id. § 131.5(a)(4), such that states can consider a wide range of information in establishing standards and EPA's review of the states' standards looks simply to whether the information on which they are based is "appropriate." By contrast, if EPA were obliged to develop standards for a state (either because of a failure to submit or an inadequate submission), the Proposal would require EPA to consider a much more limited universe of information. 
Pursuant to section 307 of the Act, EPA may issue category-wide effluent standards for listed toxic pollutants that go beyond the minimum level of control the Act mandates. These more stringent standards "shall take into account the toxicity of the pollutant, its persistence, degradability, the usual or potential presence of the affected organisms in any waters, the importance of the affected organisms, the nature and extent of the effect of the toxic pollutant on such organisms, and the extent to which effective control is being or may be achieved under other regulatory authority." Further, "[a]ny effluent standard promulgated under this section shall be at that level which the Administrator determines provides an ample margin of safety." 33 U.S.C. § 1317(a)(4). Obviously, it takes a substantial effort for EPA to assess these various factors and determine what level of pollution is acceptable, with an "ample margin of safety," and to do so for numerous categories of dischargers (multiplied by numerous different toxic pollutants). If EPA adopts the Proposal, it would make each element of this analysis that much more cumbersome and difficult, and thus make it harder for EPA to effectively protect the public from toxic pollution. 
Section 311 includes a further example of the kinds of regulatory analyses into which the Proposal would inject confusion and administrative burden. That section charges EPA with issuing "regulations designating as hazardous substances, other than oil as defined in this section, such elements and compounds which, when discharged in any quantity into . . . [various water resources] present an imminent and substantial danger to the public health or welfare, including, but not limited to, fish, shellfish, wildlife, shorelines, and beaches." 33 U.S.C. § 1321(b)(2)(A). Indeed, answering these kinds of questions seems particularly likely to be undermined by the Proposal, as data relevant to determining the conditions under which hazardous substances may be an "imminent and substantial danger" could well come from prior accidental releases that could fail the Proposal's "reproducibility" trigger. 
The foregoing examples are merely illustrative. The CWA imposes numerous regulatory duties on EPA, and the Proposal threatens to make carrying out those obligations harder. The Act's foundational purpose --  "to restore and maintain the chemical, physical, and biological integrity of the Nation's waters," 33 U.S.C. § 1251(a) -- would thus be ill-served by finalizing the Proposal.
Comment [6340-1456]: Commenter (6362) writes that the provisions cited by EPA under the CAA, the CWA, the SDWA, the CERCLA, and the EPCRA in support of its authority to develop and implement its proposed rule all provide broad regulatory authority to promulgate regulations "as are necessary to carry out [the Administrator's] functions" under the statute. 
Comment [6343-2709]: Commenter (6365) writes that their members are committed to meeting their obligations to operate in a responsible manner to avoid adversely impacting the environment and public health and safety. Both lead itself and the production of lead are extensively regulated by the EPA, including under the TSCA, CAA, CWA, and RCRA. As a result of these programs and other actions taken by industry, "[t]he United States have made tremendous progress in reducing lead concentrations in the outdoor air," with the national average concentrations dropping "more than 90 percent since 1980," and "EPA expects that most of the areas previously designated as nonattainment will have attained or are on track to attain by the end of 2016." 
In order to ensure that the regulatory standards that their members are subject to are supported by the best possible science, the commenter supports EPA's objective of strengthening transparency in regulatory science by ensuring that the data and models underlying regulatory science is publicly available in a manner sufficient for validation and analysis. Furthermore, EPA should establish clear criteria for data and model availability so that stakeholders can effectively determine whether the data and models underlying each regulatory proposal are available for review.
Comment [6353-2322]: Commenter (6375) notes that EPA solicits comment on whether additional or alternative sources of authority are appropriate bases for this proposed regulation. (pg. 18,771). EPA's statement concerning its legal authority could be enhanced by reference to statutory provisions that require it to rely on "accurate," "useful," or "best" data. For example, section 108(a)(2) requires that the Administrator issue air quality criteria that "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health and welfare . . .." 42 U.S.C. § 7408(a)(2). Similarly, section 304 of the CWA requires water quality criteria that "accurately reflect [] the latest scientific knowledge . . .." 33 U.S.C. § 1314(a)(1). Increasing the transparency of regulatory science and ensuring that it is reproducible helps in evaluating its accuracy and usefulness.
Furthermore, making data and methods underlying regulatory decisions publicly available is not new. Guidelines issued by both the OMB and EPA implementing the IQA, Treasury and General Government Appropriations Act for Fiscal Year 2001, § 515, P.L. No. 106-554, 114 Stat. 2763 (Dec. 21, 2001), emphasize the importance of reproducibility in assessing the quality and utility of data used in making regulatory decisions. It appears the OMB's IQA guidelines recognize that the reproducibility of underlying science helps to ensure the integrity of agency decisions and explain that, "[m]aking the data and methods publicly available will assist in determining whether analytic results are reproducible." 67 Fed. Reg. 8452, 8460 (Feb. 22, 2002) (OMB IQA Guidelines).
EPA's IQA guideline states: "A higher degree of transparency about data and methods will facilitate the reproducibility of such information by qualitied third parties, to an acceptable degree of imprecision. . . It is important that analytic results for influential information have a higher degree of transparency regarding (1) the source of the data used, (2) the various assumptions employed, (3) the analytic methods applied, and (4) the statistical procedures employed."
Comment [6353-2327]: Commenter (6375) notes that EPA solicits comment on whether alternative or additional regulatory or other policy vehicles are appropriate to establish and implement these policies, and whether further regulatory or other policy vehicles at the programmatic or statutory level would be appropriate as alternative or additional steps the agency may take to further the policies articulated in Section II. (pg. 18,771).
The regulatory language that EPA proposes provides general requirements that apply to the Agency's use of science under the many statutes that it implements. The generality of the proposed regulations may be a function of the diverse purposes of those statutes Specifically, some of these statutes primarily concern regulation of emissions or releases to (e.g., CAA and CWA). The scientific analyses required for these different programs and the sources of the scientific information underlying those analyses differ. The Agency should consider supplementing the general language that it has proposed with statutory specific regulations. A statutory-specific rulemaking would provide greater certainty going forward.
Comment [6873-1717]: Commenter (6895) writes that EPA's proposed rule would eliminate the use of best and vetted science to other areas of rulemaking beyond just CAA issues. For example, the proposed rule could negatively impact the development of Water Quality Standards (WQS) for the protection of human health and aquatic life, which forms the basis of New York's State Pollutant Discharge Elimination Systems (SPDES) program. This proposed rule could delay the development and permit implementation of WQS for all pollutants for which standards do not already exist, especially priority Contaminants of Emerging Concern (CEC) in New York State, as the best available peer-reviewed research would be excluded from consideration by EPA. As noted above, the proposed rule includes language regarding "dose response data and models, point of departure and reference value", which are toxicological principles and variables used specifically in the derivation of WQS.
Furthermore, the proposed rule could impede EPA research and guidance to the states pertaining to Water Treatment Chemical (INTC) use, as the formulations are typically confidential and proprietary. In recent years, EPA has focused on WTC use at potable water plants, and the discharge of untreated potentially WTC-laden effluents from these facilities. New York is similarly addressing this concern through our SPDES Whole Effluent Toxicity (WET) program in the absence of applicable WQS and/or analytical methods with sufficient levels of detection for the protection of aquatic life.
Comment [6894-3591]: Commenter (6915) writes that the proposed rule would violate the very federal laws EPA is required to uphold. To cite just a few examples, in performing its duties, EPA must rely on "the latest scientific knowledge," 33 U.S.C. § 1314(a)(1) (CWA). Indeed, no federal environmental law so much as suggests that, in setting standards, EPA can ignore the "latest" or "best" or "appropriately designed and conducted" scientific studies whenever any portion of the underlying data is not public -- which is often the case for important privacy reasons.
Comment [6894-3606]: Commenter (6915) notes that the CWA requires that water quality criteria "accurately reflect [] the latest scientific knowledge." § 304(a)(1), 33 U.S.C. § 1314(a)(1) (emphasis added).
Comment [6894-3609]: Commenter (6915) writes that even statutory provisions that EPA chose to cite as authority for the proposed action prohibit the Agency from promulgating the proposed rule. For example, CWA § 104(l) explicitly requires that "[t]he Administrator shall . . . develop and issue . . . the latest scientific knowledge available in indicating the kind and extent of effects on health and welfare which may be expected from the presence of pesticides in water." 33 U.S.C. § 1254(l) (emphasis added). It strains credulity to believe that a directive to issue the "latest scientific knowledge available" somehow imposes a requirement that the Administrator only issue knowledge based on publicly available data, and EPA has not supplied any substantive argument that it does.
Comment [6901-4443]: Commenter (6922) notes in the CWA: "The Administrator, after consultation with appropriate Federal and State agencies and other interested persons, shall develop and publish . . . criteria for water quality reflecting the latest scientific knowledge (A) on the kind and extent of all identifiable effects on health and welfare including, but not limited to, plankton, fish, shellfish, wildlife, plant life, shorelines, beaches, esthetics, and recreation which may be expected from the presence of pollutants in any body of water, including ground water; (B) on the concentration and dispersal of pollutants, or their byproducts, through biological, physical, and chemical processes; and (C) on the effects of pollutants on biological community diversity, productivity, and stability, including information on the factors affecting rates of eutrophication and rates of organic and inorganic sedimentation for varying types of receiving waters."
Comment [8750-1945]: Commenter (9227) writes that EPA arbitrarily failed to consider the far-reaching implications this Proposal could have on inter-agency coordination and consultation given that other agencies normally rely on research potentially excluded by the Proposal. The commenter provides the following consultation provisions in the CWA:
      Consultation Provisions in CWA
      §304(c)
      Information and Guidelines
      Consult with appropriate Federal and State agencies to issue information on pollution-reducing procedures and operating methods to implement standards of performance under §306.
      ***
      §304(d)(1)-(2)
      Information and Guidelines
      Consult with appropriate Federal and State agencies to publish the amount of reduction attainable through secondary treatment and information on alternative waste treatment management techniques.
      ***
      §304(e)
      Information and Guidelines
      Consult with appropriate Federal and State agencies to publish supplemental regulations to control plant site runoff, leaks/spillage, sludge/waste disposal, and drainage
      ***
      §304(f)
      Information and Guidelines
      Consult with Federal and State agencies to issue guidelines for evaluating nonpoint sources and methods to control pollution from those sources.
      ***
      §307(a)(7)
      Toxic Pretreatment Effluent Standards
      Consult with Federal departments and agencies prior to publishing regulations pursuant to this section
      ***
      §404(d)(1)
      Disposal of Sewage Sludge
      Administrator must consult with Federal agencies on regulations providing guidelines for the disposal of sludge and the utilization of sludge for various purposes.
      ***
      §118(a)
      Lake Tahoe Study
      Coordinate with Secretary of Agriculture and other Federal agencies regarding adequacy and need for extending Federal oversight of Lake Tahoe
      ***
      §311(d)(2)(M)
      Oil and Hazardous Substance Liability
      Consultation with FWS and NOAA for a fish and wildlife response plan
      ***
      §312(e)
      Marine Sanitation Devices
      "Before the standards and regulations under this section are promulgated, the Administrator and the Secretary of the department in which the Coast Guard is operating shall consult with the Secretary of State; the Secretary of Health, Education, and Welfare; the Secretary of Defense; the Secretary of the Treasury; the Secretary of Commerce; other interested Federal agencies...."
Comment [9066-1933]: Commenter (9066) writes that the Proposed Rule also conflicts with provisions of federal environmental laws that EPA specifically cites as its authority for promulgating the rule. For example, the Proposed Rule cites CWA § 104, which authorizes EPA to establish national programs for the prevention, reduction, and elimination of pollution and to conduct and promote the coordination and acceleration of research and studies in connection with establishing those programs. 33 U.S.C. § 1254(a)(1). The activities authorized by this section do not include placing limitations on the use of scientific research for regulatory decision-making. Instead, this section requires EPA to undertake continuing comprehensive studies of the effects of pollution on estuaries and estuarine zones, considering "all pertinent information" -- a mandate at odds with the Proposed Rule's push to exclude relevant peer-reviewed research from the regulatory process.
Comment [9078-3550]: Commenter (6194) writes that the Proposal would prevent EPA from relying on the best available information in discharging its duties. EPA is charged with developing regulations that provide societal benefits by reducing harm to human health and the environment from the presence of chemicals in air, soil, drinking water, food, and consumer products. The Proposal implicates research that is central to making such determinations; creating an obstacle to the consideration of such science is contrary to both primary statutory directives and requirements to conduct cost benefit analyses. Likewise, EPA fails to address how it could implement a regulation that is inconsistent with federal requirements and standards governing the use of science across agencies. 
The commenter notes that in developing statutory requirements to develop health based standards and conduct cost-benefit analyses EPA acknowledges that it must use the "best available science" in all of its regulatory actions. This directive is embodied in multiple statutes implemented by EPA and is reflected in other environmental statutes, further illustrating Congress' conceptualization of "best" available science. Broadly speaking, there are three basic types of statutory requirements that the EPA and other agencies consider scientific information. First, there are requirements that the agency consider the "best available science." Second, there are requirements that the agency consider particular factors, including scientific factors. Third, there are requirements that the agency balance economic costs or technological feasibility against public health, environmental, or other benefits that must be scientifically assessed. The precise terminology regarding the required use of science may, as illustrated below, vary across statutes, but what these provisions have in common are requirements for EPA and other agencies to use scientific information that is "best," not simply adequate, and "available." 
CWA Example of Statutory Directives regarding Use of Science by EPA and other Agencies
 CWA [33 U.S.C. § 1321(a)(27)]: Some provisions dealing with oil and hazardous substances on the Gulf Coast require "best available science."
CWA statutory provisions do not link the concept of the "best available science" to the availability of underlying raw data, nor can such a limitation be read into the statutes. Where Congress wanted to direct EPA's decision regarding what science to consider or how -- it did so. When Congress wanted to link "best available science" to data being publicly available -- it did so. Thus, if Congress had wanted to put a blanket limitation on EPA's consideration of science for which underlying data is not publicly available, it clearly knew how to do so -- and it chose not to.
As EPA has previously explained, the agency would be unable to fulfill statutory mandates if it could not consider studies that follow federal laws and ethical standards regarding the privacy of patient and individual data: 
      If EPA and other governmental agencies could not rely on published studies without conducting an independent analysis of the enormous volume of raw data underlying them, then much plainly relevant scientific information would become unavailable to EPA for use in setting standards to protect public health and the environment. . . . [S]uch data are often the property of scientific investigators and are often not readily available because of the proprietary interests of the investigators or because of arrangements made to maintain confidentiality regarding personal health status and lifestyle information of individuals included in such data. Without provisions of confidentiality, the possibility of conducting such studies could be severely compromised.)
The Proposal would also cut EPA off from information needed to complete the cost-benefit analysis of regulations required under certain statutes and government-wide requirements.
Not only would the Proposal impede EPA's fulfillment of its statutory duties, it would also be contrary to judicial precedent, which holds that "best available science" includes "all existing scientific evidence relevant to the [agency] decision [in question]." Ecology Ctr., Inc. v. U.S. Forest Serv., 451 F.3d 1183, 1194 n.4 (10th Cir. 2006) (quoting Heartwood, Inc. v. U.S. Forest Serv., 380 F.3d 428, 436 (8th Cir. 2004)). While agencies need not produce new evidence, they must seek out information and "cannot ignore existing data." Id. (emphasis added). Even if an agency is anticipating the arrival of better evidence, it must act on the basis of the evidence it currently has. Many of the fundamental public health studies on which EPA has based key rules and standards under the statutes are studies for which the raw data was not or could not have been released. Nonetheless these studies have been examined and validated, as will future studies. Arbitrarily cutting off consideration of a category of science is inconsistent with EPA's statutory obligations.
Comment [9217-3560]: Commenter (9225) writes that the proposed rule is not authorized by any authority delegated to EPA by Congress and is contrary to a number of statutes under EPA's authority. This includes, but is not limited to, the CAA; CWA; TSCA; LCSA; SDWA; FIFRA; and more. 
Comment [9283-3584]: Commenter (8281-PH6) writes that the proposed rule is inconsistent with the Agency's statutory obligation to use the best available science as required in the TSCA, SDWA, and CWA.
Comment [9290-3814]: Commenter (8281-PH24) writes that the EPA has administered successfully, the CWA, and the CAA, and these acts should be expanded based on scientific research. The EPA should not be working to undermine scientific research. Instead, this EPA should be working to provide running water to the 630,000 American families who do not have running water in their homes.
Comment [9316-5014]: Commenter (8281-PH85) supports the application of this rule to EPA's environmental impact analyses, particularly Total Maximum Daily Loads (TMDL), and the NPDES permits under the CWA.
These legally binding permits include ethylene limits for wastewater treatment facilities for pollutants such as lead, mercury, or phosphorus. Slight alterations in these permit limits can cost a single wastewater facility tens of millions of dollars, the cost of which is passed on to individual local rate bearers. These permit limits are supposed to be derived in a manner similar to dose-response relationships as mentioned in the rule where, for example, a lower level of the pollutant in the discharge will result in a measurable increase in receiving water quality working with health. However, we have dealt with instances throughout the country where environmental agencies have based regulations on publicly unavailable data, outdated science or faulty science, even in the face of data or studies which indicate stringent permit limits imposed by these agencies are not anticipated to result in any quantifiable environmental or human health benefit despite the cost. We hope that this rule would remedy these shortcomings.
We also strongly support the use of independent expert peer reviews as an additional level of review for fiscal regulatory science. Our firm has been involved in independent peer reviews of various CWA-related EPA regulations which have concluded that the technical basis for EPA's regulations and permit limits were scientifically indefensible. Had no peer reviews occurred, these regulations would have imposed hundreds of millions of dollars of wastewater treatment costs to rate bearers with no anticipated benefit. As a science-based Agency applying science-based statutes it is critical to both receiving water quality and rate payers throughout the country that these permits, and regulations are based on sound science and not speculation. In this regard, we support application of EPA's proposed rule to CWA regulations.
Comment [6141-2554]: Commenter (6166) writes that the proposed rule is not authorized by any authority delegated to EPA by Congress and is contrary to a number of statutes under EPA's authority. This includes, but is not limited to, the CAA; CWA; TSCA; LCSA; SDWA; FIFRA; and more. Substantively, the rule violates numerous public health and environmental provisions contained in these laws, as well as requirements to use the best available science or to consider all available information.
Comment [6168-4099]: Commenter (6168) raises the following concerns with the Transparency Rule proposed by the NPRM: 
 The Agency cites specific sections of statutes it administers as the basis of the Agency's authority to adopt the Transparency Rule; including CWA sections 104, 501, 33 U.S.C. 1254, 1361. The Agency rightly cites acts of Congress in an attempt to support the NPRM as such authority is necessary for the Agency to adopt regulations pursuant to the above listed Acts. 
 However, Congress, in each of the Acts' sections cited, never instructs the Agency to consider transparency to the public of supporting data and/or models of the scientific research when deciding to rely on such research as support for regulations passed pursuant to any of the Acts. Consequently, the Transparency Rule requires the EPA to rely on factors which Congress has not intended it to consider when promulgating regulations pursuant to the Acts, rendering the Transparency Rule arbitrary and capricious under the APA. 
 Further, not only does the Transparency Rule lack sufficient Congressional intent to avoid being deemed arbitrary and capricious, implementing the Rule would actually contradict Congressional instruction to conduct scientific research with a focus on `promotion' and `coordination' among itself and other institutions that produce scientific research material to the objectives of the Acts Congress requires the EPA to enforce. This provides additional cause for finding the Transparency Rule arbitrary and capricious under the APA.
Comment [7869-3372]: Commenter (8272) writes that the proposed rule is not authorized by any authority delegated to EPA by Congress and is contrary to a number of statutes under EPA's authority. This includes, but is not limited to, the CAA; CWA; TSCA; LCSA; SDWA; and the FIFRA. The rule violates numerous public health and environmental provisions contained in these laws, as well as requirements to use the best available science or to consider all available information.
Comment [6163-1276]: Commenter (6133) writes that the Proposal's peer review provision lacks any statutory basis, is vague and contrary to existing requirements for peer review.
In addition to addressing how and whether the agency will consider science, the Proposal also contains a seemingly unrelated provision regarding agency peer review. The Proposal, in § 30.7, reads: 
      What role does independent peer review in this section?
      EPA shall conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein. 
      Because transparency in regulatory science includes addressing issues associated with assumptions used in models, EPA shall ask peer reviewers to articulate the strengths and weaknesses of EPA's justification for the assumptions applied and the implications of those assumptions for the results. 
      83 Fed. Reg. at 18,774.
There is no statutory authority for EPA to "conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein." The federal statutes that EPA lists as putative authority for the Proposal provide no authority for proposed § 30.7. See CWA sections 104, 501, 33 U.S.C. 1254, 1361. The claimed authorities that EPA lists do not mention peer review or even allude to the concept. Id. Neither the Proposal nor accompanying docket materials identify any provision of any federal statute that authorizes EPA to promulgate proposed § 30.7. Our own research revealed no provision of any federal statute that authorizes EPA to promulgate proposed § 30.7. 
When Congress writes federal statutes, Congress knows how to create legal authority for peer review, who shall conduct that peer review, what role, if any, that EPA or other parties will play, and how that peer review may be conducted. None of the provision in the law authorize EPA's Proposal in § 30.7. Instead, the statutes require that EPA use peer-reviewed science, regardless of whether it would meet EPA's definition of the term in § 30.7. And in cases where the law requires EPA to conduct the review, the statutes often spell out specifically how that should happen. For example, for the CWA:
      CWA
      § 1321. Oil and hazardous substance liability 
      (a) Definitions (27) the term "best available science" means science that--(A) maximizes the quality, objectivity, and integrity of information, including statistical information; (B) uses peer-reviewed and publicly available data; and (C) clearly documents and communicates risks and uncertainties in the scientific basis for such projects; 
      33 U.S.C. § 1321 (emphasis added).
Under longstanding federal case law, when Congress authorizes an approach in one section of a statute using specific language but does not do the same in other sections of a statute, courts presume that Congress acted purposefully and did not mean to address or authorize that approach in those other statutory sections. See, e.g., Dean v. United States, 556 U.S. 568 (2009) ("It is generally presumed that Congress acts intentionally when including particular language in one section of a statute but not in another." (citing Russello v. United States, 464 U.S. 16, 23 (1983)). Not only are there no implied grants of authority to an agency in the other statutory sections, but the congressional decisions to authorize the approach elsewhere in the statute give even greater force to the conclusion that the agency has not been given authority where Congress did not use the same or similar authorizing language. 
The EPA approach proposed in § 30.7 is even more unlawful than would be the case, independently, under this case law. Proposed § 30.7 says "EPA shall conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein." 83 Fed. Reg. at 18,774/2 (emphasis added). EPA proposes to bind itself ("shall") to conduct a particular form of peer review on "all pivotal regulatory science" based on an unenforceable, non-binding OMB bulletin that has never been the subject of notice and comment rulemaking and that itself is not authorized by any federal law. To the contrary, treating the content of the unenforceable bulletin as a binding regulation would itself violate the federal statutes that EPA implements, because those statutes do not codify the bulletin, and EPA would be unlawfully codifying a mere policy preference. This EPA may not do. 
This Proposal is unlawful, arbitrary, and capricious, and an abuse of EPA discretion. To reiterate, the Proposal identifies no statutory authority for EPA to conduct independent peer review for any, much less all, "pivotal regulatory science" consistent with the dense content (and exceptions) of the OMB bulletin. The Proposal identifies no statutory authority for EPA to bind itself, and future administrations, to conduct peer review only in this fashion, unless and until future notice-and-comment rulemaking is undertaken. The Proposal identifies no suggestion in statutory language or legislative history that Congress intended EPA to conduct binding peer review consistent with this OMB bulletin, notwithstanding that Congress has known about this bulletin since 2005. EPA has simply made up proposed § 30.7 -- with its the link to the OMB bulletin, and the putative authority for the Proposal -- out of whole cloth. This EPA may not do. 
[6857-2945] Commenter (6879) notes that EPA rules and regulations have been critical to the protection of wildlife, natural habitats, and the ecosystem services they provide to communities and the economy. This includes protection of wetlands under the CWA.These laws require EPA to use the "best available" science, or all "reasonably available" science and information. By excluding high-quality peer-reviewed and validly published science from the regulatory decision-making process, the proposed regulation directly contravenes the specific mandate of these statutes. And by deliberately excluding the best science available, the rule also contravenes the Congressional mandates for these statutes and undermines their public health and environmental protection purposes.
1.3.7.2 CWA (SNPRM Comments)
Comment [48-1947]: Commenter (12727) notes that the Supplemental Notice adds a reference to the CWA, and amends its references to the CERCLA and RCRA, but these additions provide no additional support or authorization for the proposal.
The references to CWA and CERCLA cite to the general authorities of the EPA Administrator to issue regulations to fulfill the respective requirements of the statutes. These authorities cannot authorize the proposal's efforts to violate those statutes or act beyond the scope of their requirements. Similarly, the cited provision of RCRA contains the Administrator's general authorities to promote research and training, offering no basis for the proposal's requirements or reconciliation of where it may conflict with RCRA's obligations -- even requirements in the very same subsection it presumes to cite for authority. 
[48-2184] Commenter (12727) notes that just as EPA is required to consider all available science when making its regulatory decisions, governing environmental and public health statutes likewise require EPA to consider all available science when releasing ISI, which often forms the basis for the agency's regulatory decisions. For example, for the CWA:
 The CWA instructs that EPA's water quality criteria must "accurately reflect[] the latest scientific knowledge" on a variety of factors. 
A rule that prohibits EPA from considering (or that downgrades) valid, high quality scientific studies when generating ISI such as the products identified above would contravene the above- noted statutory directives regarding EPA's obligation to consider all available science.
Comment [73-415]: Commenter (11576) writes that in its supplemental notice, the EPA also points to Section 501 of the CWA, a provision that was first referenced in the agency's original proposal. Like Section 115 of CERCLA, Section 501 grants general rulemaking authority, authorizing the administrator "to prescribe such regulations as are necessary to carry out" the agency's functions under the CWA.
For the reasons explained above, and as previously noted, Section 501's general grant of rulemaking authority does not provide support for the proposed rule. Once again, the EPA has failed to explain why the regulation would be "necessary" for implementing the CWA. It plainly is not, as it would rather undermine the EPA's ability to protect public health and the environment by limiting the scientific studies the agency could consider when setting critical limits for toxic pollutants. And the agency has failed to provide any evidence that its current process for reviewing scientific evidence has been failing it. Limiting the science the EPA can rely on would compromise, not help, the agency's ability to "restore and maintain the chemical, physical, and biological integrity of the Nation's waters" -- the very purpose of the CWA. If the proposed rule is finalized, the EPA will undermine its ability to consider what may be the best or only scientific evidence available. The language of Section 501 does not provide authority for rules that are arbitrary and contrary to the statute -- and it accordingly offers no support for the proposed rule.
Comment [81-2188]: Commenter (12715) writes that whether EPA is arbitrarily barring consideration of scientific studies or information with nonpublic underlying data or data that are not able to be independently validated, or arbitrarily ascribing less weight to such studies or information, neither approach constitutes "housekeeping," and neither comports with statutory requirements that EPA use the best available science. To reiterate from the 2018 Comments, EPA's obligation with respect to the use of scientific information is clear: it must base its decisions on such criteria as the latest scientific knowledge, the best available, peer-reviewed science, and/or generally accepted scientific principles or laboratory tests. To cite just a few examples, in performing its duties, EPA must rely on: "the latest scientific knowledge," CWA of 1977, 33 U.S.C. § 1314(a)(1). EPA has repeatedly emphasized the importance of this fundamental precept, including in its 2018-2022 strategic plan, which states that one of the agency's priorities is to "identify, assess, conduct, and apply the best available science to address current and future environmental hazards."
These statutory requirements apply to all aspects of EPA's decision-making, including its scientific evaluations, adoption of regulatory standards, and development of policies. They do not permit what EPA proposes here: to either exclude or give less weight to scientific studies or information based on criteria -- availability of the underlying data and ability to independently validate underlying data -- that are not determinative of whether the studies or information constitute the best available science. Agencies may not adopt or implement regulations that conflict with the statutes under which they are promulgated, and an agency's interpretation of those statutes must always at least be reasonable. See Chevron, U.S.A., Inc. v. Nat. Res. Def. Council, Inc., 467 U.S. 837, 842-44 (1984). Furthermore, neither 5 U.S.C. § 301 nor any other purported housekeeping authority allows EPA to negate these statutes' fundamental, substantive, and common-sense commands. And even assuming EPA has some inherent housekeeping authority, it cannot rely on a general grant of rulemaking authority to promulgate regulations or develop policies that are otherwise inconsistent with more specific statutory directives. Glob. Van Lines, Inc. v. Interstate Commerce Comm'n, 714 F.2d 1290, 1293-97 (5th Cir. 1983).
Finally, EPA acknowledges that the statutes it administers would control in the event of a conflict between the proposed rule and those statutes, see 85 Fed. Reg. at 15,398, but that caveat cannot save the proposed rule. First, the core principle of the proposed rule is facially invalid, i.e., there is no circumstance in which it would be acceptable for EPA to exclude or give less weight to relevant, probative scientific studies, models, or other information that have been validated through peer review, on the sole basis that the underlying data are not publicly available or are not able to be independently validated. EPA cannot save an invalid rule by including what amounts to a waiver procedure, because the "essence of waiver is the assumed validity of the general rule." Alltel Corp. v. Fed. Commc'n Comm'n, 838 F.2d 551, 561 (D.C. Cir. 1988). Second, the agency's essential duty in rulemaking is to exercise its subject matter expertise to enact rules that implement and further the purposes of the statutes Congress has assigned it to administer. Chevron, 467 U.S. at 842-44. This does not include proposing rules without first determining the source of legal authority but instead asking commenters to supply that information. Third, the agency must propose its rules with specificity and be clear as to the programs, regulations, and information to which they will apply. 5 U.S.C. § 553(b)(3); Home Box Office, Inc., supra, 567 F.2d at 35-36.
Comment [82-856]: Commenter (12464) notes that section 501 of the CWA, 33 U.S.C. § 1361, does not authorize the Proposal for the same reasons. First, like Section 115 of CERCLA, Section 501(a) of the CWA only authorizes the EPA administrator to promulgate regulations "as are necessary to carry out his functions under this chapter." 33 U.S.C. § 1361(a). The EPA Administrator's discretion to promulgate regulations is limited by the goal of the CWA to "restore and maintain the chemical, physical, and biological integrity of the Nation's waters." EPA, however, makes no effort to explain why the Proposal is necessary to further the CWA's goals. On the contrary, the Proposal will limit the agency's ability to rely on the best available science in promulgating regulations under the CWA.
Second, the Proposal is contrary to the broader statutory scheme of the CWA. Like many other environmental statutes, the CWA embraces science and seeks to incorporate the best available science into EPA's decision-making. For example, in developing water quality criteria under the CWA, EPA must "accurately reflect[] the latest scientific knowledge." 33 U.S.C. § 1314(a)(1). The Proposal, by arbitrarily blocking EPA from considering relevant scientific studies, is inconsistent with this mandate.
Comment [82-2081]: Commenter (12464) writes that none of the other statutory provisions identified by the Supplemental Notice authorize EPA to promulgate the Proposal. The Supplemental Notice asserts that RCRA Section 8001, 42 U.S.C. § 6981, CERCLA Section 115, 42 U.S.C. § 9615, and CWA Section 501, 33 U.S.C. § 1361, empower EPA to promulgate the Proposal. 85 Fed. Reg. at 15,397. However, there are two major problems with these assertions of statutory authority. First, the Supplemental Notice does not provide any explanation for how these provisions purportedly authorize the Proposal. Second, an examination of the cited provisions demonstrates that they provide no support for the Proposal; indeed, the Proposal would actively hinder, rather than promote, the statutory purposes of RCRA, CERCLA, and the CWA.
Comment [82-2082]: Commenter (12464) argues that the Supplemental Notice makes no effort at all to tie the Proposal to the statutory schemes of RCRA, CERCLA, or the CWA. Instead, it merely asserts that the cited provisions authorize the Proposal without any explanation of how the Proposal relates to the language or purposes of the three statutes. Additionally, both the Initial Proposal and the Supplemental Notice refer to a range of different provisions from different statutes. 83 Fed. Reg. at 18,769; 85 Fed. Reg. at 15,397. EPA must explain how distinct provisions in multiple statutes can authorize the agency to promulgate precisely the same rule for its review of science under all of these schemes, as well as how the Proposal would comply with each statutory scheme.
Comment [82-2083]: Commenter (12464) writes, "[I]t is `axiomatic' that `administrative agencies may act only pursuant to authority delegated to them by Congress.'" None of the cited provisions, however, provide EPA with authority to ignore relevant scientific information. The CWA only authorizes rulemakings that are necessary to achieve the statutes' broader purposes. The Proposal, however, is not related to EPA research or research funding, and will actively hinder EPA's ability to achieve the statutes' underlying goals by limiting EPA's ability to rely on the best available science.
EPA has long accepted that its decisions must be guided by the best available science. For example, in 2002, EPA issued Information Quality Guidelines in which it took the position that the standard set forth in the SDWA -- to use "the best available, peer-reviewed science" -- should apply to all of the agency's risk assessments. EPA even reiterated its commitment to this standard in the Initial Proposal. 83 Fed. Reg. at 18,769. However, the Proposal would directly undermine this longstanding commitment to use the best available science. As we pointed out in a comment letter in response to the Initial Proposal, the Proposal would prevent EPA from relying on crucial studies that underlie a number of past, present, and future regulations. This radical departure from EPA's standard practice cannot be justified by any of the statutes EPA identifies because it is contrary to the goals of those statutes.
Comment [89-2199]: Commenter (11403) writes that EPA's 2018 version of this rule cited a hodgepodge of statutory authority, none of it valid. EPA at least corrected the obvious typographical errors from the previous version. However, those edits still do not grant EPA the authority to restrict science from all regulatory decision-making processes within the agency. EPA is attempting to use general grants of authority to create a completely novel approach to assessing science. However, as the Supreme Court has stated, "Congress . . . does not alter the fundamental details of a regulatory scheme in vague terms or ancillary provisions -- it does not, one might say, hide elephants in mouseholes." EPA does not have the authority to alter the fundamental details of all of its regulatory schemes based on such vague provisions of law. 
As an example, the commenter states that Section 501 of the CWA states, "The Administrator is authorized to prescribe such regulations as are necessary to carry out his functions under this chapter." This catch-all provision provides the authority to prescribe regulations, but EPA has not provided a single rationale explaining how the proposed rule is "necessary" to carrying out functions under the CWA.
In short, the commenter notes that none of the statutory provisions cited by EPA provide authority for the proposed rule. The proposed rule fails at step one under Chevron v. Natural Resources Defense Council since it is unreasonable to believe that any statutory general grant of authority or any mention of EPA's oversight of other science or research programs would enable EPA to conduct rulemaking in a way that is altogether inconsistent with the rule of law. None of the cited provisions provide the authority sought by EPA, and as such the proposed rule should be withdrawn.
Comment [125-2203]: Commenter (11495) notes that EPA's Supplemental Notice is at odds with provisions of multiple environmental statutes which require the Agency to consider certain high-quality scientific information. The CWA and the CAA require that water and air quality criteria determinations be based on the "latest scientific knowledge." 
Each of these provisions directs EPA to incorporate a certain quality of scientific information into its rulemaking decisions made pursuant to the statute. In particular, these requirements are designed to ensure the scientific validity of the information upon which EPA bases its actions. Mandating a high-quality scientific basis for regulatory actions is necessary to guarantee the best possible protections for public health and welfare and to minimize partisan influence in scientific decision-making.
The Supplemental Notice, however, would impermissibly prohibit the agency from considering of relevant and high-quality scientific studies whenever the data underlying the study are not publicly available -- in direct contravention of the statutory directives that mandate, for example, consideration of the "best available science." None of the provisions requires, or even suggests, that EPA only consider information that is publicly available. It strains the plain meaning of words like "best," "latest," and even "appropriate" to suggest that they permit the unscientific requirement that all data be publicly available.
EPA "cannot ignore available . . . information." Courts have held that "best available data" requirements "prohibit[] [an agency] from disregarding available scientific evidence that is in some way better than the evidence [it] relies on." Furthermore, a plaintiff may establish a violation of a "best available science" requirement by "point[ing] to any scientific evidence that the agency failed to consider." The Supplemental Notice would undoubtedly preclude EPA from considering certain studies and scientific information that is "better" or the "best available," simply because the study's underlying data is not publicly available, thereby failing to satisfy Congressional directives and undermining the quality of regulatory decisions.
Likewise, EPA cannot avoid the statutory directives to consider high-quality scientific information simply by assigning a lesser weight to such information when the underlying data are not publicly available, as the Agency alternatively proposes. The Supplemental Notice's alternative proposal is designed to have largely the same pernicious effects as flatly prohibiting consideration of studies lacking publicly available data. Just as with the primary proposal, EPA's alternative proposal would limit the agency's consideration of the best available science in making important regulatory decisions, and is, therefore, arbitrary, and capricious.
The effect of the Proposed Rule is inconsistent with EPA's view that the Supplemental Notice is "intended to be consistent with the statutes that EPA administers and EPA plans to implement this procedural rulemaking in accordance with all applicable statutory and regulatory requirements." EPA attempts to resolve this inconsistency by stating that "in the event the procedures outlined in the proposed rulemaking conflict with the statutes that EPA administers, or their implementing regulations, the statutes and regulations will control." However, as discussed below, EPA lacks the statutory authority to issue a rule this broad, even when it claims that it will avoid statutory conflicts.
Comment [133-1194]: Commenter (11894) writes that the CWA requires states, or, if they default, EPA, to establish "total maximum daily [pollution] loads" for individual water bodies to specify the pollution levels that they can tolerate. About 65,000 such TMDL have been established, mostly by states, but some by EPA. TMDL rest on extensive bodies of scientific information and modeling in several different technical fields. Once again, EPA has completely failed to even specify whether or not its new requirements would apply to this program.
The question whether these new requirements would apply to TMDL raises a fundamental issue that goes far beyond the TMDL program itself. Perhaps the new requirements would not apply to EPA-issued TMDL because of the specific regulatory language exempting EPA actions. But would they apply to the extensive technical guidance that EPA issues to states on how to write TMDL? And if they did, how could EPA exempt its own requirements from the same standards?
These questions are relevant, not just to the TMDL program, but to the entire structure of "cooperative federalism" under several EPA statutes. They would apply, for example, in almost exactly the same form, to permit decisions under the CWA or to permit and state implementation plan decisions under the CAA. Typically, under each of these statutes, EPA issues extensive technical guidance to state programs even though they retain the power to make regulatory decisions. Yet EPA's proposal does not mention this issue in any way. Nor does it discuss the impact of these new requirements on the TMDL program or how the agency might address them.
Comment [133-1238]: Commenter (11894) notes that the CWA requires each state to develop TMDL for every waterbody the state identifies as having uses impaired because of pollution. The objective of the TMDL is to determine the pollutant loading capacity of the waterbody and to allocate that load among the pollutant sources so that the appropriate control actions can be taken to achieve the state's water quality standards. States are responsible for developing TMDL and submitting them to EPA for approval. Even if a third party develops the TMDL with a proprietary model, the state is responsible for submitting that TMDL to EPA. If EPA disapproves a state TMDL, EPA must develop a replacement TMDL. TMDL are developed using a range of techniques from simple mass balance calculations to complex fate and transport mathematical models. To date, over 65,000 TMDL have been developed and approved for use in regulating point sources of pollution and controlling pollutant discharges from nonpoint sources. If the final science transparency rule applies retroactively to ISI, every one of the TMDL will have to be re-examined. If the model underlying a TMDL fails to meet the new tests of transparency and reproducibility, the enforceability of the state's point source permit limits and nonpoint source controls based on that TMDL will be undermined. If the science transparency rule applies only prospectively to new TMDL, EPA will need to disapprove any new TMDL based on a proprietary model since such a model would not meet the new transparency and reproducibility requirements. Whether the science transparency rule applies retroactively or prospectively, EPA will incur significant costs and time replacing TMDL. Despite this major impact, the supplemental notice provides no estimation of the costs and benefits.
Comment [159-1414]: Commenter (12430) notes that the SNPRM cites to one additional CWA provision  -  33 U.S.C. section 1361. The most arguably relevant subdivision simply states that "[t]he Administrator is authorized to prescribe such regulations as are necessary to carry out his functions under this chapter." It does not address the use of science, and the SNPRM fails to explain how the proposed regulation would be "necessary to carry out" the CWA, particularly when another section of the Act requires that water quality criteria "accurately reflect[] the latest scientific knowledge."
Comment [159-2091]: Commenter (12430) writes that in the SNPRM, U.S. EPA proposes to consider only (or, in the alternative, more heavily weight) studies and models with publicly available data. However, restricting agency consideration of scientific studies and models would run afoul of U.S. EPA's obligations under many of its operating statutes to rely on the best available science. For example, The CWA requires that water quality criteria "accurately reflect[] the latest scientific knowledge."
Comment [161-1464]: Commenter (12434) writes that the Proposal directly conflicts with requirements under the core federal environmental laws, each of which requires that decisions be made using the "best available science" or some similar restriction. By requiring that publicly available data be prioritized over other scientific data, regardless of any other merits, the EPA is arbitrarily constrained and will be unable to comply with the statutory requirements to meet the "best available science."
The CWA, for example, requires that the EPA develop water quality criteria "accurately reflecting the latest scientific knowledge." 33 U.S.C. § 1314(a)(1). In fact, throughout the CWA, the Agency is required to look at a broad range of scientific information when promulgating the numerous regulatory decisions, it entails. When reviewing states' water quality standards, the EPA must review any submitted "general information which will aid the Agency in determining the adequacy of the scientific basis of the standards." 40 C.F.R. § 131.6(f). The CWA does not limit the review of this information and other scientific information by whether the data set is publicly available anywhere in the statute. In fact, given that many of these regulatory decisions directly consider "public health or welfare," such as is the case for water quality standards, 33 U.S.C. § 1313(c)(2)(A), it is possible that the relevant scientific studies include data sets that are not fully publicly available because of public health data involved. The Proposal does not provide any such justification, beyond a stated intent of transparency, as to why a robust, and even peer-reviewed, scientific study that reflects the "latest scientific knowledge" could be insufficient or even ignored when making such critical decisions such as the establishment of water quality standards.
Public health data, most vulnerable to the Proposal's exclusions, are used for the development of water quality standards, which are critical for protecting the health of communities in the Long Island Sound region. For example, NYC has developed a Long-Term Control Plan in order to reduce combined sewer overflow events. The success of the Long-Term Control Plan is matched against New York State's water quality standards. If the latest scientific knowledge is not used when establishing these water quality standards because of confidential public health data sets, then NYC might work towards an inadequate standard that is not properly protective of public health, a serious consideration when speaking of the frequent overflow of raw sewage into local waterways during storm events.
Comment [187-1703]: Commenter (12720) notes that EPA cites sections 104, 33 U.S.C. § 1254, and 501, 33 U.S.C. § 1361, of the CWA as putative authority for the Proposal. Nothing in these sections authorize the Proposal's limitations on scientific evidence.
With respect to section 104, the Proposal tellingly fails to specify which of its 22 subsections supposedly supports the restrictions EPA has proposed. This deficiency reflects a lack of authority for the Proposal in section 104. And even if EPA thinks that it can cobble together language in section 104 to support the Proposal, the agency's complete failure to identify in the Proposal how section 104 authorizes this rulemaking means that EPA did not provide sufficient notice for the public to comment on the Proposal.
None of the subsections in section 104 states or suggests that, in promulgating regulations under the CWA, EPA may limit its consideration of "regulatory science underlying its actions" to studies or analyses that "are publicly available in a manner sufficient for independent validation." See 83 Fed. Reg. at 18,773 (proposed § 30.5) (and modified at 85 Fed. Reg. at 15,402). To the contrary, several subsections indicate that Congress intended EPA to consider available scientific evidence in order to carry out the Act. 
First, subsection (b) authorizes EPA to collect and publicize results and information related to studies about water pollution but does not say anything about limiting consideration of science simply because data cannot be made public, either as part of rulemakings or otherwise. Nor does it draw any distinction between dose-response data and other types of data.
Second, subsection (c) directs EPA to "conduct research on, and survey the results of other scientific studies on, the harmful effects on the health or welfare of persons caused by pollutants." It provides no authority whatsoever for limiting consideration of studies, models or data, dose-response or otherwise, during rulemakings; indeed, by directing EPA to "survey the results of other scientific studies," rather than the publicly-available dose-response data underlying those results, this subsection contradicts the Proposal's (and Supplemental Proposal's) limitations and conditions.
Third, subsection (l)(1) indicates that EPA should be inclusive with respect to considering evidence, as it directs EPA to "develop and issue to the States for the purpose of carrying out this Act the latest scientific knowledge available in indicating the kind and extent of effects on health and welfare which may be expected from the presence of pesticides in the water in varying quantities. He shall revise and add to such information whenever necessary to reflect developing scientific knowledge."
Fourth, subsection (n) directs EPA to cooperate with various entities to "conduct and promote, encourage contributions to, continuing comprehensive studies of the effects of pollution, including sedimentation, in the estuaries and estuarine zones of the United States on fish and wildlife, on sport and commercial fishing, on recreation, on water supply and water power, and on other beneficial purposes." Importantly, subsection (n)(2) reveals Congress's intention that EPA will consider information broadly, by instructing the agency to "assemble, coordinate, and organize all existing pertinent information on the Nation's estuaries and estuarine zones. . ."
EPA also cites 33 U.S.C. § 1361 as a basis for the Proposal, but it does not provide the agency with the authority it desires. Subsection (a) merely states that the "Administrator is authorized to prescribe such regulations as are necessary to carry out his functions under this chapter." As explained above, that general grant of authority cannot support the rule, especially when the rule conflicts with the Act. Moreover, EPA casually invokes this provision, but does not make any effort to justify the proposed restrictions as necessary to any particular CWA statutory function, so it has not made the case that this provision provides authority to adopt the Proposal's limits.
The Act contains other indications that Congress intended EPA's consideration of science to be inclusive. In particular, section 304(a)(1) of the Act states:
The Administrator, after consultation with appropriate Federal and State agencies and other interested persons, shall develop and publish, within one year after the date of enactment of this title (and from time to time thereafter revise) criteria for water quality accurately reflecting the latest scientific knowledge (A) on the kind and extent of all identifiable effects on health and welfare including, but not limited to, plankton, fish, shellfish, wildlife, plant life, shorelines, beaches, esthetics, and recreation which may be expected from the presence of pollutants in any body of water, including ground water; (B) on the concentration and dispersal of pollutants, or their byproducts, through biological, physical, and chemical processes; and (C) on the effects of pollutants on biological community diversity, productivity, and stability, including information on the factors affecting rates of eutrophication and rates of organic and inorganic sedimentation for varying types of receiving waters. 
As the italicized language above reveals, Congress refused to limit EPA's consideration of available evidence in discharging one of its core functions aimed at protecting the nation's waters. EPA provides no reason in the Proposal why the any other action under the CWA the Proposal targets should be any different.
1.3.8 National Environmental Policy Act (NEPA) (SNPRM Comments) 
Comment [Comment 20-1294]: Commenter (10753) cited comments as stating that NEPA requires preparation of an Environmental Impact Statement (EIS) before EPA adopts the Rule and/or the SNPRM. This is especially true because the SNPRM expressly extends the Rule's coverage to rules and models that may not have been otherwise covered. EPA states:
      This SNPRM proposes that the scope of the rulemaking apply to ISI as well as significant regulatory decisions. This notice proposes definitions and clarifies that the proposed rulemaking applies to data and models underlying both pivotal science and pivotal regulatory science. In this SNPRM, EPA is also proposing a modified approach to the public availability provisions for data and models that would underly significant regulatory decisions and an alternate approach. Finally, EPA is taking comment on whether to use its housekeeping authority independently or in conjunction with appropriate environmental statutory provisions as authority for taking this action.
      https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-9322 
Put another way, by expanding the Rule's reach, the SNPRM requires an EIS more (and a more thorough EIS) than does the Rule by itself.
Response: NEPA does not apply to this rulemaking because the rule is not a "major Federal action significantly affecting the quality of the human environment." NEPA section 102(2)(c); 42 USC § 4332(2)(c). The rule is one of internal agency procedure that describes the consideration the agency will give to studies based on data availability and the factors EPA uses to evaluate studies that may be used for regulatory purposes and ISI and, as such, will not affect the quality of the human environment.
Comment [Comment 89-814]: Commenter (11403) states that section 7(a)(2) of the ESA requires that "each federal agency shall, in consultation with and with the assistance of the Secretary, insure that any action authorized, funded, or carried out by such agency is not likely to jeopardize the continued existence of any endangered or threatened species or result in the destruction or adverse modification of habitat of such species which is determined by the Secretary . . . to be critical." Under the Services' joint regulations implementing the ESA, EPA is required to review its actions "at the earliest possible time" to determine whether the action may affect listed species or critical habitat. EPA must initiate consultation under Section 7 whenever its action "may affect" a listed species or critical habitat. The phrase "may affect" has been interpreted broadly to mean that "any possible effect, whether beneficial, benign, adverse, or of an undetermined character, triggers the formal consultation requirement." Because this proposed rule would prevent EPA from utilizing the best available science, EPA would be unable to sufficiently determine if one of its decisions, regulations, or policies would harm a threatened or endangered species or its critical habitat. EPA must at a minimum engage in a programmatic consultation with the services to determine if this action is likely to adversely affect endangered species.
While some of EPA's actions are exempt from the NEPA, this proposed rule is clearly not exempt. By EPA's own tortured logic, this proposal encompasses virtually every aspect of EPA's statutory mandate, and most of these still fall under the ambit of NEPA. Accordingly, the EPA must prepare an EIS for the proposed rule because the net effect of the changes to the existing regulations will mean more environmental harm due to delayed decisions, under-regulation of harmful pollutants, and the rejection of the best available science. By every definition, this will have a massive impact on the human environment and therefore requires review under NEPA.
NEPA requires every federal agency to prepare an EIS for all "major Federal actions significantly affecting the quality of the human environment." EPA may forego a NEPA analysis only if a categorical exclusion applies, and EPA has not indicated that it believes a categorical exclusion applies. If EPA does indeed believe that one applies, EPA must provide the public with notice and the opportunity for the public to provide comment. If EPA does not believe that a categorical exclusion applies, EPA must at the very least elect to first complete an Environmental Assessment (EA) to determine if an EIS is needed or complete a full EIS. It is highly likely, if not a certainty, that this proposal will have negative environmental impacts. A "reasonable person" reviewing this proposal could only conclude that the proposal is designed to weaken and hamstring EPA's ability to properly regulate harmful pollutants, resulting in significant harm to human health and environmental quality. Therefore, a finding of no significant impact from an EA would be highly suspect and would almost certainly be arbitrary and capricious. EPA must prepare a complete EIS on this proposed rule and provide a reasonable period to provide public comment consistent with the requirements of NEPA.
Response: An ESA section 7(a)(2) consultation is not required for this rule because it does not affect listed species or critical habitat. Further, contrary to the commenter's assertion, the rule does not prevent EPA from using the best available science. The rule is one of internal agency procedure that describes the consideration the agency will give to studies based on data availability and the factors EPA uses to evaluate studies that may be used for regulatory purposes and ISI. 
See response to Comment 20-1294 regarding NEPA applicability. 
1.3.9 Other Environmental Statutes (NPRM Comments)
Comment [37-32]: Commenter (0046) writes that EPA must also submit its proposal to the Science Advisory Board pursuant to the requirements of 42 U.S.C. §4365 (c)(1) (the Environmental Research Development Demonstration Authorization Act), which requires the Administrator to submit to the SAB any "proposed criteria document, standard, limitation, or regulation, together with relevant scientific and technical information in the possession of the (EPA) ... on which the proposed action is based" at the time it provides that proposal to another agency of the government for formal review. The SAB is then to review and comment on the proposal, which the Administrator is to consider, although the Administrator is not required to obtain SAB approval for any final action. See H. Rep. No. 95-722 (95th Cong. 1st Sess. (1977) (Conference Report).
Response: EPA sought SAB review of aspects of the proposed rule and received the SAB's comments on April 24, 2020. See https://yosemite.epa.gov/sab/sabproduct.nsf/2DB3986BB8390B308525855800630FCB/$File/EPA-SAB-20-005.pdf.
Comment [993-4238]: Commenter (1008) states that they are in favor of all federal agencies promoting public access to research results, including to the data supporting research conclusions that inform regulatory decisions. Commenter further appreciates the previous Administration's memorandums and orders that require federal agencies to make their data publicly accessible (e.g. Increasing Access to the Results of Federally Funded Scientific Research" - OSTP Memo Feb 22, 2013, "Open Data Policy-Managing Information as an Asset" - OMB Memo M-13-13). However, no agency (including the EPA), in its push towards research transparency and public access to research data, should compromise or disregard its Congressional mandated mission.
Response: EPA intends to implement this rule consistent with its mission to protect human health and the environment. Implementation of the final rule will not result in EPA compromising or disregarding its mission. 
Comment [1320-4331]: Commenter (1335) notes that there may be articles in the EPA legislation, such as those cited in the proposed Rule, which exhort the EPA to use the best science available to reach regulatory conclusions, but the overarching intent and thrust of the Environmental Protection Act is to actually protect the environment. It would be extremely unwise to weaken the ability of the EPA to protect the environment to the best of its ability, to protect Americans from pollution and climate change (whatever the cause). In fact, any effort to thwart that central intent of the Act by the Agency must be in its core illegal.
Response: The Agency has no knowledge of the Environmental Protection Act referenced by the commenter. The final rule does not rely on or interpret the substantive environmental statutes that EPA cited in the 2018 and 2020 proposals. EPA's implementation of this rule will not weaken the Agency's ability to protect human health and the environment. The EPA is issuing this final rule to help strengthen the transparency of the dose-response data underlying certain EPA actions. As EPA states in the preamble, "codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the public's access to the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety."
Comment [1412-4472]: Commenter (1427) writes that the proposal asserts existing statutes and regulations governing the application of science in regulatory processes burden EPA, constrain its ability in the fulfillment of its mission where it must forthrightly (transparently) interface with the public. EPAs proposed remedies include weakening its duty to comply with legal, i.e., statutory and regulatory, and ethical obligations; defining new sciences unique to EPA administration; and assigning new authorities to non-scientist administrators to referee determinations that science applied to EPA regulations is valid and credible. Such remedies should be rejected.
Response: EPA disagrees with the commenter's assertion that the statutes and implementing regulations constrain EPA in the manner described by the commenter. EPA states clearly in § 30.3, that were there are conflicts between this rule and the substantive provisions of the statutes and regulations the Agency administers, this rule will yield. EPA is relying on its housekeeping authority to promulgate this purely procedural rule. It is well within the authority of the Administrator to develop procedures that govern the internal affairs of the Agency. See the response to Comment 31-135. The rule includes reasonable factors for the Administrator to consider when deciding whether to exempt a study from the rule requirements. 
Comment [2322-157]: Commenter (2337) urges EPA not to finalize the Proposed Rule. The Proposed Rule is inconsistent with EPA's statutory mandate under various environmental laws to use the best available scientific data to protect public health and welfare and the environment.
Response: EPA disagrees that the final rule is inconsistent with its statutory mandates. EPA intends to continue relying on the high-quality studies to support its significant regulatory decisions and influential scientific information. See response to Comments 1320-4331 and 1412-4472.
Comment [2322-159]: Commenter (2337) expresses that the Proposed Rule requires that underlying data supporting scientific studies concerning the environmental or health impacts of pollutants be made publicly available for independent validation in order for EPA to use such data in making regulatory decisions. The Proposed Rule further provides that EPA will conduct independent peer review on all data it uses to justify its regulatory decisions. While commenter supports EPA's goal of basing its rulemaking on the strongest scientific evidence possible and fostering transparency, they are concerned that the Proposed Rule unduly limits the scientific data EPA can consider and conflicts with EPA's statutory obligations.
Response: The final rule does not prevent the agency from considering any particular studies. Nor does the rule authorize violations of the statutes and regulations. EPA will not categorically exclude any studies from consideration when promulgating significant regulatory decisions or developing influential scientific information. EPA has explained that this procedural rule has limitations on its scope in terms of conflicts with other substantive requirements and that, in situations where this rule applies, EPA will give greater consideration to those studies where the underlying dose-response data are available for independent validation. That means that the agency will consider all relevant studies when taking covered actions. 
The final rule does not require EPA to conduct independent peer review of all the studies it uses to justify regulatory decisions. Section 30.6 provides, in pertinent part, that the "EPA shall conduct independent peer review consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review and the exemptions described therein. The EPA will evaluate whether or not to conduct additional peer review of individual studies identified as pivotal science if the studies have already undergone peer review." If adequate peer review has occurred under those guidelines, additional peer review may not be warranted. 
Comment [2322-161]: Commenter (2337) writes that this repeated statutory requirement that EPA use the "latest," "best available," and "peer-reviewed" science conflicts with EPA's proposal to use only publicly available scientific data. None of the environmental statutes restrict reliable scientific evidence to studies where all underlying data has been made publicly available. Indeed, there are many reasons why data cannot, or cannot easily, be made public, including privacy and individual confidentiality concerns, particularly when related to personal health information. All high quality scientific research, regardless of the public availability of its raw data, should equally be used by EPA in establishing regulatory standards. Were EPA to reject otherwise reliable epidemiological or similar health evidence because underlying data could not be made available without violating privacy or confidentiality, EPA would not be using "the latest scientific knowledge" or "the best available peer reviewed science" in violation of its statutory obligations. Further, EPA would be ignoring data that would ensure its ability to meet its obligations to establish standards protective of public health, with "an adequate margin of safety."
Response: The final rule does not restrict EPA to using only publicly available scientific data. Additionally, the final rule does not categorically exclude any relevant studies from consideration. EPA will evaluate and weight studies determined to be pivotal science, using the criteria in § 30.5, which will allow all high-quality studies to be factored into the agency's development of significant regulatory decisions and influential scientific information. Also see the response to Comments 1320-4331, 1412-4472 and 2322-159.
Comment [2322-205]: Commenter (2337) contends that the Proposed Rule's provisions allowing data to be made public in controlled processes (section 30.5) and EPA to grant exemptions to the rule's requirements (section 30.9) do not resolve the rule's inconsistencies with environmental statutes. Where all underlying data cannot be made publicly available, the Proposed Rule requires the Administrator to make a case-by-case determination whether to exempt the data from disclosure requirements. This exemption will not ensure that EPA considers the latest or best available data, lacks consistent standards, and in any event will likely be difficult and time consuming to administer.
Response: EPA states clearly in § 30.3, that were there are conflicts between this rule and the substantive provisions of the statutes and regulations the Agency administers, this rule will yield. The final rule does not categorically exclude any relevant studies from consideration. See response to Comments 2322-159 and 2322-161.
EPA disagrees that the exemption provision lacks consistent standards. The final rule includes reasonable factors for the Administrator to consider when deciding whether to exempt a study from the rule requirements. The additional procedures required by this rule only apply to pivotal science, which is a subset of the total number of studies that the EPA would evaluate. Given the current robust process for identifying and reviewing scientific studies and documentation that are existing Agency practice, as well as that the determination of dose-response data availability is limited to pivotal science underlying significant regulatory actions and influential scientific information, the EPA disagrees that implementation will be difficult or time-consuming.
Comment [2412-206]: Commenter (2427) writes that EPA's proposed rule that claims to strengthen transparency in science supporting regulations, would in fact prevent the Agency using the best available science  -  and therefore would keep EPA from complying with environmental laws.
Response: See response to Comments 2322-159 and 2322-161.
Comment [2772-2255]: Commenter (2787) argues that the proposed rule suffers from significant procedural flaws including lack of supporting evidence, insufficient detail in the proposal itself, and conflicts with EPA's statutory obligations.
Response: EPA disagrees that the final rule suffers from significant procedural flaws. See responses to comments in RTC section 1.5.
Comment [3117-104]: Commenter (3132) asserts that the proposal is void ab initio because the EPA has violated the National Environmental Policy Act. This is because the EPA is preparing the proposed rule without first preparing the necessary Environmental Impact Statement. See generally, NE_PA. With a few strictly limited exceptions, NEPA's Environmental Assessment and EIS requirements apply to EPA action. 40 CFR §6:204. Obviously, none of the exceptions apply here and any rational Environmental Assessment would find an EIS necessary. See generally CFR §§6:200-210. 
Response: NEPA does not apply to this rulemaking because the rule is not a "major Federal action significantly affecting the quality of the human environment." NEPA section 102(2)(c); 42 USC § 4332(2)(c). The rule is one of internal agency procedure that describes the consideration the agency will give to studies based on data availability and the factors EPA uses to evaluate studies that may be used for significant regulatory actions and influential scientific information and, as such, will not affect the quality of the human environment.
Comment [3118-3141]: Commenter (3133) suggests that the proposal is void ab initlo because the EPA has violated the NEPA. This is because the EPA is preparing the proposed rule without first preparing the necessary Environmental Impact Statement. NEPA. With a few strictly limited exceptions, NEPA's Environmental Assessment and EIS requirements apply to EPA actions. 40 CFR §6:204. Obviously, none of the exceptions apply to the proposal and any rational EA would find an EIS necessary. See generally 40 CFR §§6:200-210.
Response: See response to Comment 3117-104.
Comment [4831-2080]: Commenter (4842) states that there is no basis in existing bedrock environmental laws that authorizes EPA to limit science considered in rulemaking processes. EPA cites several foundational environmental laws as a justification for this proposal. Nowhere in the cited statutes is there any basis for demanding access to raw data, nor does this relate sensibly to any definition of best available science. Rather, this proposal undermines the use of best available science called for in environmental statutes including the Clean Air Act. Further, there is no basis for politically appointed administrators to choose which science will be considered and which may not be.
Response: The final rule does not rely on or interpret the substantive environmental statutes that EPA cited in the 2018 and 2020 proposals. Also see response to Comments 1412-4472 and 2322-205.
Comment [4832-283]: Commenter (4843) contends that this mandate directly undermines the agency's mission to protect health and human safety by purposefully creating more bureaucratic red tape to potentially prevent peer-reviewed scientific studies from having an effect on policy, without limiting the exposure of study participants. It works to undermine the scientific community as a whole by limiting the scope, type, character, and data from having any effect on policy unless researchers follow an arbitrarily limited state mandated and controlled scientific study format.
It is up to the agency to hire qualified people to read and interpret the most up to date science, to judge its value, and make recommendations based off the broader research as a whole to define and quantify potential hazards in the environment. This is a part of the EPA's core mandate and any decision that hinders it will directly increase the risk to everyone that the agency is sworn to protect. The mandate of the agency supersedes the current political climate.
Response: EPA disagrees that the final rule will undermine the Agency's mission. See the response to Comments 1320-4331 and 2322-205.
Comment [4868-4076]: Commenter (4879) suggests that given that there is no non-frivolous argument that any categorical NEPA exclusion could apply (see commenter's attached joint comment dated May 24, 2018), an EIS that fully discloses the environmental impact of the proposal is required by NEPA.
Response: See response to Comment 3117-104.
Comment [4873-2103]: Commenter (4884) asserts that EPA governance is not meant to be a profitable enterprise. It is meant to serve the public interests in good health and environmental quality. The Clean Air, Pure Waters, Toxic Substances Control, Resource Recovery, and Hazardous Waste laws all direct EPA to use the best science to achieve these ends without regard to cost. Why should EPA consider clean air cost free? Dirty air imposes poor health that the public must suffer and pay for. Dirty air soils clothes and buildings. Acidic deposition dissolves infrastructure, kills biota, and diminishes visibility. These impairments are costs to the public and the planet that cradles our lives and lively hood.
Response: Nothing in the final rule requires EPA to consider costs in a manner inconsistent with statutory requirements. 
Comment [5153-4079]: Commenter (5164) writes that their letter adds to the abundant evidence that the NEPA requires the preparation of a proper EIS before the regulation can be adopted.
Response: See response to Comment 3117-104.
Comment [5170-382]: Commenter (5181) notes that the Administration is currently engaged in a multi-year effort to streamline NEPA and recommends that the proposed rule not undermine that initiative either by excluding information that does not meet the requirements or by requiring more time for the required transparency standards to be met.
Response: Nothing in the final rule is related to the Administration's efforts to streamline NEPA. This is a rule of internal agency procedure that applies only to EPA.
Comment [5410-4088]: Commenter (5421) states that EPA's success and the success of the existing regulations in handling RCRA problems with current scientific studies (studies that EPA would be compelled to ignore under the proposed regulation) provides further evidence that NEPA requires EPA to prepare a thorough EIS before it adopts the proposed regulation. Adopting the regulation without an EIS would violate NEPA.
Response: EPA disagrees that NEPA applies to this rulemaking. See response to Comments 3117-104, 2322-159 and 2322-161. 
Comment [5865-695]: Commenter (6111) contends that the proposed rule would prohibit the continued and future use of these studies by EPA thereby obstructing EPA's statutory duty to consider the "best," "reasonably" available information in its decision-making processes. The resulting information vacuum would occur for no other reason than that the underlying human subject data is private and cannot be publicly disseminated.
Response: See response to Comments 1320-4331, 2322-157, 2322-159, 2322-161, 2322-205 and 5410-4088.
Comment [5985-4090]: Commenter (5995) suggests that even assuming arguendo that a non-frivolous scientific argument in favor of the proposed regulation could be made, the NEPA requires the preparation of a proper EIS before the proposed regulation can be adopted.
As the agency necessarily concedes, EPA has responsibility to prepare its own NEPA documents for compliance . https://www.epa.gov/nepa Significantly, EPA regulations also expressly recognize that NEPA compliance is necessary if [t]he proposed action is known or expected to cause significant public controversy about a potential environmental impact of the proposed action. 40 C.F.R. Sect. 6.204(b)(8), https://www.ecfr.gov/cgi-bin/text-idx?SID=e62b352cfd49343307deaef7f6cdc0db&mc=true&node=se40.1.6_1204&rgn=div8 (stating that, in such circumstances, a categorical NEPA exclusion is not applicable). In the present case, EPA must prepare a thorough EIS before it adopts the proposed regulation.
Response: EPA disagrees that NEPA applies to this rulemaking. See response to Comment 3117-104.
Comment [6093-543]: Commenter (6103) writes that commenters with legal expertise may note that there is no basis in the cited statutes for demanding access to raw data.
Response: Nothing in the final rule requires EPA to demand access to raw data. The rule does not require EPA to obtain dose-response data from researchers nor does it require researchers to submit dose-response data to the Agency. EPA has explained that the Agency will give greater consideration to studies where the underlying dose-response data are available for independent validation. Also note that the Agency is not relying on any of the environmental statutes cited in the proposal as legal authority for the final rule.
Comment [6097-3146]: Commenter (6107) writes that by purporting to limit EPAs consideration of important data, the proposed rule would cripple EPA efforts to comply with NEPA.
Response: EPA disagrees that NEPA applies to this rulemaking. Also see response to Comments 2322-159 and 3117-104. 
Comment [6099-563]: Commenter (6109) notes that EPA states "the best available science must serve as the foundation of EPA's regulatory actions," citing Executive Order No. 13563. EPA could have also cited multiple laws that already mandate EPA's use of "best available science," including the TSCA and SDWA. According to the U.S. 10th Circuit Court of Appeals decision in Ecology Ctr., Inc. v. U.S. Forest Service., (451 F.3d 1183, 1194 (10thCir. 2006)), "best available science" means that agencies "should seek out and consider all existing scientific evidence relevant to the decision" and "cannot ignore existing data." EPA is proposing precisely the opposite and providing for itself a mechanism by which the agency could ignore existing data and turn a blind eye to existing scientific evidence.
Response: See response to Comments 2322-159 and 2322-161. 
Comment [6102-480]: Commenter (6112) writes that the objectives are consistent with EPA's authority under numerous statutes that emphasize the need for sound science and public participation in regulatory decisions.
Response: EPA appreciates your comment in support of the final rule.
Comment [6103-516]: Commenter (6113) states that by forcing EPA to ignore such private (and appropriately non-public) data, the proposed anti-environment rule is fiendishly calculated to block the EPA staff from considering the best data available. This scheme has at least three devastating flaws.
First, the proposed rule will nonsensically bar EPA from considering the best scientific data available disabling EPA rulemaking. This rule would violate the agency's common sense and statutory duty to consider the sound scientific data.
Response: See response to Comments 2322-159 and 2322-161.
Comment [6105-520]: Commenter (6115) writes that the objectives are consistent with EPA's authority under numerous statutes that emphasize the need for sound science and public participation in regulatory decisions.
Response: EPA appreciates your comment in support of the rulemaking.
Comment [6107-4015]: Commenter (6117) provides that the EPA has already embarked on a well-thought-out plan to require and promote research transparency, as described in the EPA's 2016 Plan to Increase Access to Results of EPA-Funded Scientific Research ("Plan"). Although the preamble to the proposed regulation cites and "applies concepts and lessons learned" from the Plan, there are some concerning differences.
First, the Plan indicates that it does not apply to "scientific research data collected before its implementation." In contrast, the preamble and the proposed regulation suggest that the regulation may apply retrospectively:
      ...EPA seeks comments on how the prospective or retrospective application of the provisions for dose response data and models or pivotal regulatory science could inadvertently introduce bias regarding the timeliness and quality of the scientific information available.
The commenter does not recommend any retroactive application of this regulation. It is well established that agency rules cannot be applied retroactively unless Congress expressly granted the agency that power. There is no specific grant of power to apply this proposed regulation retroactively. Moreover, the two bills upon which this proposed regulation is modeled, the Honest and Open New EPA Science Treatment (HONEST) Act of 2017 (H.R. 1430  -  115th Congress, 2017-2018), and the Secret Science Reform Act of 2015 (H.R.1030  -  114th Congress 2015-2016), were not enacted.
Response: This rule does not change any existing EPA regulations. It only applies prospectively to significant regulatory actions and influential scientific information, based on dose-response data, that are developed after the effective date of the rule. However, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created.
Comment [6112-1605]: Commenter (6137) states that not only does the Proposed Rule contravene the statutes upon which EPA relies as statutory authority, but its limitations on science likewise conflict with core provisions of other environmental statutes. For example, the Food Quality Protection Act ("FQPA"), which regulates pesticide residue in conjunction with FIFRA, sets safety standards based on the consideration of all available data. Limiting the data available to conduct studies necessary to evaluate the harmful effects of pesticides runs counter to this mandate.
Specifically, the commenter notes that Congress overhauled our food safety laws when it unanimously passed the FQPA, amending both the Federal Food, Drug, and Cosmetics Act ("FFDCA") and FIFRA. The overhaul responded to a seminal 1993 National Academy of Sciences ("NAS") report criticizing EPA for treating children like "little adults" by failing to address the unique susceptibility of children to pesticide exposures based on the foods they eat, their play, metabolism, and sensitive stages of their development. NAS, Pesticides in the Diets of Infants and Children (1993). The NAS recommended that EPA revamp and strengthen its pesticide regulations to account for children's vulnerabilities, consumption patterns, and exposures. Because it would take time to fill gaps in knowledge, safeguards and methodologies, the NAS recommended that additional protection be afforded in the form of "uncertainty" or "safety factors." The NAS first described how EPA has regularly used uncertainty factors and then proposed an additional uncertainty factor for fetal developmental toxicity and where data are incomplete: "In the absence of data to the contrary, there should be a presumption of greater toxicity to infants and children. To validate this presumption, the sensitivity of mature and immature individuals should be studied systematically to expand the current limited data base on relative sensitivity." Id. 
The FQPA strengthened the food safety standard in several ways. First, under the FQPA, the EPA Administrator "may establish or leave in effect a tolerance for a pesticide chemical residue in or on food only if the Administrator determines that the tolerance is safe. The Administrator shall modify or revoke a tolerance if the Administrator determines it is not safe." 21 U.S.C. § 346a(b)(2)(A)(i). In other words, the absence of sufficient information to find a pesticide safe means it cannot be allowed in or on our food. 
Second, safe "means the Administrator has determined there is a reasonable certainty that no harm will result from aggregate exposure to the pesticide chemical residue, including all anticipated dietary exposures and all other exposures for which there is reliable information." Id. § 346a(b)(2)(A)(ii). The FQPA, therefore, requires that EPA conduct an assessment based on aggregation of all exposures to a pesticide whether from eating foods, drinking water with residues of the pesticide, or uses of the pesticide in and around the home or other places where people can be exposed. 21 U.S.C. § 346a(b)(2)(A)(ii), (C)(i)(I) & (ii). The FQPA also requires EPA to assess and protect against unsafe risks posed by cumulative exposures to pesticides that share a "common mechanism of toxicity," as is the case with pesticides in the organophosphate, carbamate, and pyrethroid families. See 21 U.S.C. § 346a(b)(2)(C)-(D). 
Third, EPA must make specific safety determinations for infants and children. Id. § 346a(b)(2)(C)(ii)(I) & (II). It must consider available information concerning "the special susceptibility of infants and children," including "neurological differences between infants and children and adults, and effects of in utero exposure to pesticide chemicals." Id. § 346a(b)(2)(C)(i)(II). EPA must also base its tolerance decisions on available information about food "consumption patterns among infants and children." Id. § 346a(b)(2)(C)(i)(I) & (III).
Fourth, EPA must account for children's sensitivities, scientific uncertainty, and gaps in available data. The statute requires that "an additional tenfold margin of safety for the pesticide chemical residue and other sources of exposure shall be applied for infants and children to take into account potential pre -and post-natal toxicity and completeness of the data with respect to exposure and toxicity to infants and children." Id. § 346a(b)(2)(C). EPA can depart from this requirement and use a different margin of safety "only if, on the basis of reliable data, such margin will be safe for infants and children." Id.; see Nw. Coal. for Alts. to Pesticides v. EPA, 544 F.3d 1043, 1046 (9th Cir. 2008) (reversing EPA's shrinkage of the safety factor in the absence of supporting data). 
As an over-arching mandate, the FQPA directs EPA to make its tolerance determinations based on its assessment of the pesticide's risk. 21 U.S.C. § 346a(b)(2)(C)(i). And throughout its mandates to assess the full effects of a pesticide, the FQPA directs EPA to base its risk assessment on "available information" about consumption patterns, special susceptibility of infants and children, and cumulative effects. Id. The FQPA also expressly directs EPA to consider the validity, completeness, and reliability of the available data from studies, the nature of the toxic effect, available information about the relationship between study results and human risk, and available information about aggregate and cumulative effects. Id. § 346a(b)(2)(C)(ii). 
The Proposed Rule upends these mandates by requiring EPA to put on blinders and ignore a huge and important subset of the available data. It collides squarely with the congressional direction to consider all available data and information in order to protect our food and children in particular. It also conflicts with the congressional mandate to afford greater protection to children and our food when gaps in data prevent a full quantitative assessment and establishment of a dose-response. It thus undermines EPA's ability to carry out its obligations under the FQPA.
Response: EPA disagrees that the final rule will undermine EPA's ability to comply with the FQPA. See response to Comments 1320-4331, 1412-4472, and 2322-159.
Comment [6112-4017]: Commenter (6137) notes that EPA also requested "comment on how the prospective or retrospective application of the provisions for dose response data and models or pivotal regulatory science could inadvertently introduce bias regarding the timeliness and quality of the scientific information available." 83 Fed. Reg. at 18,772. Like much of the rest of the Proposal, it is unclear what exactly EPA is asking or suggesting. Certainly, to the extent EPA uses the Proposed Rule to exclude important, valid scientific information, it will bias the quality of the scientific information available.
To the extent EPA suggests the Rule may be applied retroactively, there is simply no basis for doing so. It is well-established that an agency cannot apply a rule retroactively absent clear congressional intention for such application. E.g., Sierra Club v. Whitman, 285 F.3d 63, 68 (D.C. Cir. 2002) (referring to "unusual ability to implement rules retroactively"); Bowen v. Georgetown Univ. Hospital, 488 U.S. 204, 208 (1988) ("Retroactivity is not favored in the law. Thus, congressional enactments and administrative rules will not be construed to have retroactive effect unless their language requires this result. By the same principle, a statutory grant of legislative rulemaking authority will not, as a general matter, be understood to encompass the power to promulgate retroactive rules unless that power is conveyed by Congress in express terms." (citations omitted)). EPA has not identified any such congressional intention in any of the statutes at issue, and thus retroactive application of the Rule would be unlawful. 
Nor would the statutes upon which EPA relies support such an application. For example, as explained above, the Clean Air Act does not allow -- and, as the D.C. Circuit has held, certainly does not require, e.g., American Trucking, 283 F.3d at 372 -- applying the so-called transparency provisions of the Proposed Rule at all in rules subject to the procedural requirements of Clean Air Act § 307(d). Such rules include NAAQS. Further, far from suggesting that EPA could lawfully reopen long-settled NAAQS to apply a new and novel standard of review, the Clean Air Act requires EPA to review and revise air quality criteria and NAAQS at least every five years and to "promulgate such new standards as may be appropriate." 42 U.S.C. § 7409(d)(1) (emphasis added). This carefully chosen language confirms that Congress did not intend for EPA to apply rules like the Proposed Rule to undo existing NAAQS, but instead intended for regular reviews of scientific information to result in new NAAQS.
Even if EPA had statutory authority to apply the Proposed Rule retroactively, it could not do so rationally. See Sierra Club, 285 F.3d at 68 (if EPA had authority to implement a rule retroactively, "retroactivity must be `reasonable,'"). Indeed, retroactive application of the Proposed Rule would undo well-settled rules in a tremendously unfair and irrational way, and would lead to widespread confusion about countless environmental and public health policies and protections.
Response: See response to Comment 6107-4015.
Comment [6127-2240]: Commenter (6152) states that the proposal is also neither required nor authorized under any statute that authorizes the EPA. Commenter strongly urges EPA to withdraw this proposed rule.
Response: EPA disagrees that it lacks authority to promulgate this rule. See the response to comment 31-135.
Comment [6133-2173]: Commenter (6158) suggests that importantly, the critical actor under the best available science standard is the agency  --  the standard requires the use of the best science that is available to the agency, not necessarily available to the public. Excluding science that is available to the agency but not available to the public would thus be a violation of "best available science" provisions.
Under federal law, the agency must find a way to continue using the best available scientific data and models available to it, regardless of whether those data and models are publicly available. Excluding science whose raw data are not in the public domain runs counter to the "best available science" doctrine and is thus illegal.
Response: See response to Comments 2322-159 and 2322-161.
Comment [6152-654]: Commenter (6122) asserts that the proposed rule violates the Endangered Species Act (ESA) and the NEPA.
Section 7(a)(2) of the ESA requires that "each federal agency shall, in consultation with and with the assistance of the Secretary, insure that any action authorized, funded, or carried out by such agency is not likely to jeopardize the continued existence of any endangered species or threatened species or result in the destruction or adverse modification of habitat of such species which is determined by the Secretary . . . to be critical." Under the Services' joint regulations implementing the ESA, the EPA is required to review its actions "at the earliest possible time" to determine whether the action may affect listed species or critical habitat. The EPA must initiate consultation under Section 7 whenever its action "may affect" a listed species or critical habitat. The phrase "may affect" has been interpreted broadly to mean that "any possible effect, whether beneficial, benign, adverse, or of an undetermined character, triggers the formal consultation requirement." Because this proposed rule would prevent the EPA from utilizing the best available science, as discussed above, the EPA would be unable to sufficiently determine if one of its decisions, regulations, or policies would harm a threatened or endangered species, or their critical habitat. EPA must at a minimum engage in a programmatic consultation with the Services to determine if this action is likely to adversely affect endangered species.
While some of the EPA's actions are exempt from the National Environmental Policy Act (NEPA), this proposed rule is clearly not. By EPA's own tortured logic, this proposal encompasses virtually every aspect of EPA's statutory mandate, and most of these still fall under the ambit of NEPA. Accordingly, the EPA must prepare an Environmental Impact Statement ("EIS") for the proposed rule because the net effect of the changes to the existing regulations will mean more environmental harm due to delayed decisions, under-regulation of harmful pollutants, and the rejection of the best-available science. By every definition, this will have a massive impact on the human environment.
NEPA requires every federal agency to prepare an EIS for all "major Federal actions significantly affecting the quality of the human environment" and the Council for Environmental Quality's general NEPA regulations, which guide the EPA dictate whether or not an EIS must be prepared. The EPA may forego a NEPA analysis only if a categorical exclusion applies  --  the EPA has not indicated one way or another if it believes a categorical exclusion applies and if it does so, has not provided the public with notice of that belief or the opportunity to comment. If a categorical exclusion does not apply, the EPA must complete an Environmental Assessment (EA) or an EIS. To determine whether an EIS is required, the EPA may elect to prepare an EA first. If they do so, they must provide a period of notice and comment on the EA. It is highly likely, if not a certainty, that this proposal will have negative environmental impacts on the environment. A "reasonable person" reviewing this proposal could only conclude that the proposal is designed to weaken and hamstring the EPA's ability to properly regulate harmful pollutants, resulting in significant harm to human health and environmental quality, and that an EA/FONSI would be completely arbitrary and capricious. Accordingly, the EPA must prepare a complete EIS on this proposed rule and provide a reasonable period to provide public comment.
Response: An ESA section 7(a)(2) consultation is not required for this rule because it does not affect listed species or critical habitat. Further, contrary to the commenter's assertion, the rule does not prevent EPA from using the best available science. The rule is one of internal agency procedure that describes the consideration the agency will give to studies based on data availability and the factors EPA uses to evaluate studies that may be used for regulatory purposes and influential scientific information. 
EPA disagrees that this rulemaking is subject to NEPA. See response to Comment 3117-104.
Comment [6152-3730]: Commenter (6122) notes that EPA's proposal represents the most significant rejections of science in the agency's nearly 50-year history. If finalized, the rule would allow political appointees to reject the best-available science for virtually any reason and delay taking critical, life-saving actions in order to benefit special interests and corporate polluters. Simply put, EPA is proposing to formally adopt the mindset of flat-earthers and climate deniers, adopting the worst tactics of the tobacco industry, all in an effort to reward the Trump administration's political benefactors. If finalized, this proposal will kill Americans, more people will go to hospitals because of pollution-induced illness, it will degrade our environment, and it will cause wildlife to go extinct. This profoundly senseless proposal must be withdrawn in its entirety.
The scientific process has always provided transparency through the peer review process and other measures designed to ensure the accuracy of scientific studies. In contrast, the proposed rule will prevent EPA from relying on the best available science because, as the EPA is well aware, underlying data are sometimes unavailable and not essential to peer review. This does not diminish the credibility, veracity, or importance of a particular study or paper. Deliberately ignoring science in order to tip the scales in favor of regulatory inaction violates every law that the EPA is charged with administering. The EPA's proposed rule adds insult to injury by allowing the Administrator to cherry pick which studies to consider based entirely on his or her whims, including studies that are favorable to industry  --  or even authored by industry  --  where underlying data will inevitably be withheld on CBI grounds, while disallowing the public health studies that have long been the bedrock of EPA decision-making.
EPA may only act within the constraints of the authority delegated to it from Congress. There is no statutory authority that supports the proposed rule. For example, with regard to regulations under the CAA, the proposed rule refers to Sections 103 and 301(a). However, neither of these sections contains any language even remotely supporting this proposal  -  which is presumably why the proposed rule does not cite any specific statutory language. Agencies are not free to take actions unmoored from their statutory mandates. Rather, as the Courts have explained, "EPA is a federal agency  -  a creature of statute. It has no constitutional or common law existence or authority, but only those authorities conferred upon it by Congress." Here, since EPA's mandates under the CAA and all the other cited statutes plainly require that the agency consider the best available science, even where underlying data may not be publicly available, it could not be clearer that EPA may not proceed with its proposed regulation.
Response: EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, EPA does not rely on, interpret or apply provisions of a particular statute or statutes that it administers. EPA will undertake such efforts in forthcoming substantive actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. Also see response to Comments 31-135, 2322-159, 2322-161, 2322-205, 1320-4331, 1412-4472
Comment [6155-865]: Commenter (6125) writes that the policy justification for the proposal is inadequate. The proposal would apparently change the workings of all of the EPA's cited regulatory statutes, making key parts of regulatory decisions under each of them in advance. Accordingly, it must be justified as an exercise of regulatory authority under each such statute. 
It is the agency's job, not EPN or any other commenter's, to provide that justification. However, to show how far short the proposal falls, the next section of EPN's comments explores in detail the deficiencies in the justification of this proposal under the cited statutes, and then examines in more detail three of the agency's most important regulatory mandates - the ambient air quality standards provisions of the CAA, the drinking water standards provisions of the SDWA, and the general chemical regulatory provisions of the TSCA.
Response: EPA disagrees that the final rule lacks justification. EPA described in the preamble of both the 2018 and 2020 proposals that the purpose of the rule is to increase the transparency of the science underlying significant regulatory actions and influential scientific information. The final rule would not result in the Agency making key parts of regulatory decisions in advance of a rulemaking. The studies considered by the Agency will be included in the rulemaking docket pursuant to § 30.4. For pivotal science, the weight assigned to the studies under § 30.5 and the rationale for the weighting will also be included in the docket. This will allow the public to comment on how the Agency evaluated the studies underlying the significant regulatory action before a rule is promulgated. 
Comment [6155-2475]: Commenter (6125) contends that there is neither a legal basis nor a need for this rule. It would require that EPA violate explicit statutory provisions and would undermine environmental protection and critical health protections the public has come to rely on. It unlawfully shifts the basis for deciding what science to use in rulemaking away from the statutory goals of scientific reliability and environmental protection to so-called transparency, a term not used in the relevant EPA statutory provisions.
Response: See response to Comments 31-135, 1320-4331, and 1412-4472 
Comment [6163-1118]: Commenter (6133) writes that the Proposal conflicts with the statutes that EPA administers.
The Proposal unlawfully restricts EPA's consideration and use of "dose response data and models that underlie" what the Proposal calls "pivotal regulatory science." 83 Fed. Reg. at 18,770/2. The Proposal goes on to state: 
      Pivotal regulatory science' is the studies, models, and analyses that drive the magnitude of the benefit-cost calculation, the level of a standard, or point-of-departure from which a reference value is calculated. In other words, they are critical to the calculation of a final regulatory standard or level, or to the quantified costs, benefits, risks and other impacts on which a final regulation is based.
Id. By restricting EPA's implementation of its federal organic statutes and the Administrative Procedures Act in this fashion, and by defining "pivotal regulatory science" in this manner, the Proposal violates federal laws. The Proposal does so by requiring EPA to implement federal laws based on the Proposal's criteria and conception of "pivotal regulatory science," rather than on the congressional criteria and requirements in federal statutes that contradict, disallow, or fail to include those criteria and concepts in the Proposal.
Response: See response to Comments 1320-4331, 1412-4472 and 2322-159.
Comment [6163-1143]: Commenter (6133) writes that the Food Quality Protection Act (also known as the Food, Drug, and Cosmetics Act or FFDCA) governs pesticide tolerances. Section 408 of the FFDCA requires EPA to set tolerances, which are maximum residue limits, for pesticide residues on foods. In setting tolerances, EPA must find that the tolerance is "safe." 21 U.S.C. § 346a. Safe is defined as meaning that there is a "reasonable certainty that no harm will result from aggregate exposure to the pesticide residue." Id. § 346a(b)(2)(a)(ii). To make this finding, EPA considers, among other things: the toxicity of the pesticide and its break-down products, aggregate exposure to the pesticide in foods and from other sources of exposure, and any special risks posed to infants and children. Id. § 346a(b). For threshold effects, EPA is required to add an additional tenfold margin of safety to protect infants and children, unless the administrator finds based on reliable data that a different safety factor will ensure the pesticide is safe. Id. § 346a(b)(2)(C)(ii). The statute contains specific provisions regarding the type and availability of data that must be considered. Id. § 346a(b)(2)(D), (E), (F). 
The Proposal does not cite to the FFDCA, and apparently EPA never considered whether the Proposal is consistent with the law. It is not. First, the Act defines "safe" to mean that "there is a reasonable certainty that no harm will result from aggregate exposure to the pesticide chemical residue, including all anticipated dietary exposures and all other exposures for which there is reliable information." Id. § 346a(b)(2)(A)(ii). As part of this determination, EPA must "ensure that there is a reasonable certainty that no harm will result to infants and children from aggregate exposure." Id. § 346a(b)(2)(C). EPA cannot do this if it excludes relevant studies solely because the underlying data cannot be made public. 
The FFDCA specifically speaks to how threshold and non-threshold effects shall be considered. Id. § 346a(b)(2)(B). The Proposal cannot override the specific Congressional mandates in the FFDCA for how to conduct a tolerance assessment. In determining whether there is a reasonable certainty of no harm to infants and children, EPA must consider "available information" on consumption patterns among infants and children, special susceptibility of infants and children (including for example neurological and in utero effects), cumulative effects on infants and children. Id. § 346a(b)(2)(C). Likewise, the Act specifies numerous scientific factors that must be considered in evaluating safety, including considering "available data" on these factors. Id. § 346a(b)(2)(D). The Proposal plainly contradicts these mandates. Obviously, published, peer-reviewed literature is "available" and must be considered. As with studies considered under other statutes, EPA fails to explain the arbitrariness of excluding published peer-reviewed studies while allowing industry studies considered CBI to be considered. 
Finally, the FFDCA contains certain procedural requirements for "establishing general procedures and requirements to implement this section." Id. § 346a(e). Yet EPA failed to cite the FFDCA -- either its substantive or procedural requirements -- at all in its Proposal. 
Response: EPA disagrees that the final rule will undermine EPA's ability to comply with the FQPA. See response to Comments 1320-4331, 1412-4472, and 2322-159.
Comment [6163-1146]: Commenter (6133) provides the following: 
      The Atomic Energy Act (AEA), 42 U.S.C. § 2011 et seq., is not a typical environmental law, as the original act established the Atomic Energy Commission (AEC) just after World War II to promote the "utilization of atomic energy for peaceful purposes to the maximum extent consistent with the common defense and security and with the health and safety of the public." The concern found in the final clause of its original organic act, "the health and safety of the public," has at no point disappeared in subsequent iterations of the act and this Proposal runs contrary to its clearly stated intent. 
      The AEC was abolished in the 1970s, and since that then, most of the functions of the AEA are carried out by the Nuclear Regulatory Commission and the U.S. Department of Energy. However, when EPA was formed in the early 1970s, it assumed the AEC's authority to issue generally applicable environmental radiation standards to protect the health and safety of the public. Other federal and state organizations must follow these standards when developing requirements for their areas of radiation protection. EPA also implements the Federal Radiation Council's authority under the AEA, developing guidance for federal and state agencies containing recommendations for their use in developing radiation protection requirements and working with states that have radiation protection programs. 
      There are several specific statutory requirements that EPA executes under the AEA, which states that "the purpose of this [Act is] to effectuate the policies set forth above by providing for  -  (d) a program to encourage widespread participation in the development and utilization of atomic energy for peaceful purposes to the maximum extent consistent with the common defense and security and with the health and safety of the public." 42 U.S.C. § 2013(d) (emphasis added). 
      The following regulations are health-based standards, and as we discuss supra section III.G., EPA bases its regulatory limits and nonregulatory guidelines for population exposures to low-level ionizing radiation on the linear no-threshold (LNT) dose-response model, which uses the premise that any radiation dose carries some risk, and that risk increases directly with dose. The viability of each of these longstanding health-based protections will be undercut by promulgation of a final rule that resembles this draft for the reasons discussed supra section III.G., and in direct conflict with the AEA's requirement that the utilization of atomic energy for peaceful purposes be "to the maximum extent consistent with the common defense and security and with the health and safety of the public."
      Environmental Radiation Protection Standards for Nuclear Power Operations (40 C.F.R. Part 190); these standards limit radiation releases and doses to the public from the normal operation (non-emergency) of nuclear power plants and other uranium fuel cycle facilities.
      Environmental Radiation Protection Standards for Management and Disposal of Spent Fuel, High Level and Transuranic Wastes (40 C.F.R. Part 191); this regulation sets environmental standards for the disposal of highly radioactive spent nuclear fuel and certain kinds of highly toxic and radioactive wastes produced from the nuclear weapons program that must ultimately be disposed of in a deep geologic repository.
      Health and Environmental Protection Standards for Health and Environmental Protection Standards for Uranium and Thorium Mill Tailings (40 C.F.R. Part 192); this regulation sets standards for the protection of the public health, safety, and the environment from radiological and non-radiological hazards associated with uranium and thorium ore processing, and disposal of associated wastes. In May of 2015, EPA proposed revisions to 40 C.F.R. 192 that would establish groundwater restoration and monitoring requirements at in-situ recovery facilities, and then in January 2017, EPA re-proposed those revisions. We await final agency action on the matter.
      Criteria for the Certification and Recertification of the Waste Isolation Pilot Plant's Compliance with the 40 C.F.R. Part 191 Disposal Regulations (40 C.F.R. 194); these criteria apply to the certification and recertification of compliance with the radioactive waste disposal standards at the Waste Isolation Pilot Plant (WIPP) in New Mexico, the world's only deep geologic repository, which is operated by the U.S. Department of Energy (DOE) for permanent disposal of transuranic waste from the nation's nuclear defense program.
      Public Health and Environmental Radiation Protection Standards for Yucca Mountain, Nevada (40 C.F.R. Part 197); these regulations, last promulgated in 2008 (after a Federal Appeals Court found an earlier version unlawful, see, e.g., Nuclear Energy Inst., Inc. v. EPA, 373 F.3d 1251 (D.C. Cir. 2004)), establish public health and environmental standards for storage and disposal of spent nuclear fuel at the proposed repository at Yucca Mountain, Nevada. The U.S. Nuclear Regulatory Commission would implement these regulations at Yucca Mountain if a repository were to be established there.
      As discussed above, the Clean Air Act requires EPA to regulate airborne emissions of hazardous air pollutants (HAPs) from a specific list of industrial sources called "source categories." Standards known as the "National Emission Standards for Hazardous Air Pollutants" (NESHAPs) dictate specific regulatory limits for source categories that emit radionuclides. In 40 C.F.R. Part 61: the National Emission Standards For Hazardous Air Pollutants, EPA sets health based standards in a number of settings, such as Subpart B: Radon Emissions from Underground Uranium Mines; Subpart H: Emissions of Radionuclides Other than Radon from Department of Energy Facilities; Subpart I: Radionuclide Emissions from Federal Facilities Other than Nuclear Regulatory Commission Licensees and Not Covered by Subpart H; Subpart K: Radionuclide Emissions from Elemental Phosphorus Plants; Q: Radon Emissions from Department of Energy Facilities; R: Radon Emissions from Phosphogypsum Stacks; Subpart T: Radon Emissions from the Disposal of Uranium Mill Tailings; and Subpart W: Radon Emissions from Operating Mill Tailings.
      And last, under the Safe Drinking Water Act (SDWA), discussed above, EPA sets health-based standards on the levels of certain radionuclides in drinking water. After much litigation, in 2000 EPA revised an outdated set of standards that had been in place since the late 1970s and set new monitoring provisions for community water systems (CWS). The current standards are: Combined radium 226/228 of 5 pCi/L; a gross alpha standard for all alphas of 15 pCi/L (not including radon and uranium); a combined standard of 4 mrem/year for beta emitters; and a the MCL for uranium at 30 ug/L. 
 
In short, the Proposal could seriously damage EPA's ability to administer the AEA and protect the public from radiation. Yet the Proposal fails to cite the statute at all.
Response: EPA disagrees that the final rule will damage the Agency's ability to administer the AEA or any other environmental statutes. See response to Comments 1320-4331, 1412-4472, and 2322-159.
Comment [6163-3475]: Commenter (6133) contends that the Proposal's retroactivity provisions are arbitrary and capricious.
In the Proposal, EPA states that the proposed regulation "is intended to apply prospectively." 83 Fed. Reg. at 18,771. However, a few pages later, the agency "solicits comments on how the prospective or retrospective application of the provisions for dose response data and models or pivotal regulatory science could inadvertently introduce bias regarding the timeliness and quality of the scientific information available." Also, the Proposal states that "for regulatory programs . . . in which future significant regulatory actions may be based on the administrative record from previous reviews . . . , EPA seeks comment on the manner in which this proposed rule should apply to that previous record." Id. at 18,772. 
In short, despite its assertion that the rulemaking is "intended" to apply prospectively, the Proposal contemplates prohibiting EPA -- or will prohibit EPA --  from relying on studies generated prior to rulemakings that fail to meet the Proposal's ill-defined criteria for "publicly available data." This approach is arbitrary and capricious, runs counter to the specific language of many statutes the agency is tasked with administering, and would destroy the agency's ability to promulgate health-based standards to protect the American public using the best available science. 
The Proposal ignores an entire body of case law that has considered and roundly rejected both retroactivity in rulemakings and limiting data that underlies rulemakings to "publicly available data." In so doing, the Proposal is arbitrary and capricious, and should be rejected. The Supreme Court strongly disfavors retroactive application of rules. The Court has stated that: 
      Retroactivity is not favored in the law. Thus, congressional enactments and administrative rules will not be construed to have retroactive effect unless their language requires this result. [] By the same principle, a statutory grant of legislative rulemaking authority will not, as a general matter, be understood to encompass the power to promulgate retroactive rules unless that power is conveyed by Congress in express terms. [] Even where some substantial justification for retroactive rulemaking is presented, courts should be reluctant to find such authority absent an express statutory grant.
Bowen v. Georgetown Univ. Hosp., 488 U.S. 204, 208 - 09 (1988) (internal citations omitted). Notably, the Proposal does not identify a single provision in a single statute that EPA administers, or any other federal law, that requires or even authorizes any final rule based on the Proposal to have retroactive effect. See generally 83 Fed. Reg. at 18,768 - 74. There has been no power conveyed by Congress in express terms to promulgate retroactive rules related to any element of the Proposal; it is unsurprising that the Proposal does not and cannot identify any express or even implied grant of authority. See Bowen, 488 U.S. at 208 - 09. 
The Proposal claims prospective application, while nonetheless noting that in some circumstances EPA may desire to apply the rule retroactively. 83 Fed. Reg. at 18,771. This, too, is unlawful and fails to meet the high burden in the Supreme Court's Bowen decision and its progeny concerning retroactive application of agency rules. The suggestion in the Proposal, for example, that EPA may invoke the Proposal's approach to review all prior health and scientific studies underlying the NAAQS is illegitimate, arbitrary and capricious, and contrary to caselaw. Bowen and its progeny do not permit agency rules to have retroactive effect to disallow health studies and regulatory science generated prior to, or relied upon by EPA prior to, adoption of any final rule based on the Proposal. This caselaw does not entertain any such exception and accepting any such exception for these circumstances would circumvent the holdings and reasoning of this case law. 
Response: See response to Comments 1412-4472, 2322-159, 2322-161 and 6107-4015.
Comment [6340-1647]: Commenter (6362) writes that implementation of the rule should be statute-specific.
EPA requested comment on the effect this proposed rule may have on individual EPA programs. Each of the federal environmental statutes referenced by EPA as a source for its authority to propose this rule, was enacted and designed to achieve a specific environmental goal and purpose (e.g., TSCA regulates new and existing chemicals, CAA controls air pollution on a national level, and SDWA regulates public drinking water supplies across the nation). Each statute confers its unique authority upon the agency, requiring agency review according to different scientific standards; each has its own regulations designed to effectuate the specific corresponding program's mission; and, in many cases, each statute relies on different and variable scientific disciplines. As such, ACC believes that this rule, while applicable to all the statutes identified, should be implemented by regulations specific to the objectives and scientific disciplines of each statute. ACC believes that just as the FOIA, which is overseen by the US DOJ, is implemented by each agency with specific and separate regulations relevant to the requirements of each statute, this policy rule should be implemented by each EPA program office charged with implementing a given statute in a manner consistent with the authorities granted and requirements unique to that statute.
Response: EPA appreciates your comment. In this rule, EPA does not rely on, interpret or apply provisions of a particular statute or statutes that it administers. EPA will undertake such efforts in forthcoming substantive actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule.
Comment [6894-3611]: Commenter (6915) writes that because the proposed rule would run afoul of these provisions and potentially others, EPA's citation to general rulemaking authorities such as CAA § 301(a), 42 U.S.C. § 7601(a), and CWA § 501, 33 U.S.C. § 1361, is unavailing. Such general provisions of rulemaking authority cannot override more specific statutory directives. Global Van Lines, 714 F.2d at 1293-97. Nor can EPA's reliance on 5 U.S.C. § 301 in the notice extending the comment period save its ultra vires proposal. 83 Fed. Reg. at 24,256. Known as the "housekeeping statute," 5 U.S.C. § 301 is "simply a grant of authority to the agency to regulate its own affairs," not a general, independent basis for deviating from a specific statutory directive or limiting the scope of other statutes. See Chrysler Corp. v. Brown, 441 U.S. 281, 308-12 (1979).
Thus, as a general matter, EPA's obligation is clear: it must base its decisions on such criteria as the latest scientific knowledge, the best available, peer-reviewed science, and/or generally accepted scientific principles or laboratory tests. No statute suggests that EPA, in setting standards, can reject scientific evidence that meets those criteria solely because the underlying data are not public or because the evidence is based on models that otherwise follow long-accepted scientific guidelines. In short, EPA lacks sufficient legal authority to either adopt or implement the proposed rule, and its proposed action conflicts with the statutes it must follow.
Response: See response to Comment 31-135 on EPA's authority to promulgate this rule. Also see response to Comments 1412-4472, 2322-159 and 2322-161.
Comment [6894-3624]: Commenter (6915) writes that the proposal provides no analysis of its environmental impacts and fails to explain how EPA has addressed the requirements of the NEPA, 42 U.S.C. § 4321 et seq.
Response: EPA disagrees that this rulemaking is subject to NEPA. See response to Comment 3117-104.
Comment [6906-3976]: Commenter (6927) agrees that this proposed rule is consistent with the Administrative Procedures Act (APA) as well as programmatic statutes that EPA administers to disclose to the public the bases for agency rules and to rationally execute and adequately explain agency actions.
Response: EPA appreciates your comment in support of the rulemaking. 
Comment [7775-4127]: Commenter (8178) objects to the proposal to limit the use of scientific evidence in rulemaking proceedings.
In general, before approving the proposed regulation, NEPA requires EPA to prepare a thorough EIS for the project. In a discussion entitled: What is included in an EIS?, EPA outlines the EIS requirements. https://www.epa.gov/nepa/national-environmental-policy-act-review-process#EIS The requirements include: Alternatives: Consideration of a reasonable range of alternatives that can accomplish the purpose and need of the proposed action.
Although the documents that EPA presented in support of the proposed regulation fall far short of what NEPA requires in many ways, the most obvious deficit may be the EPA's failure to meaningfully discuss reasonable alternatives to the proposed action. In the present case, as a consequence of Anti-Environment regulation's extraordinary breadth, NEPA requires EPA to prepare an EIS which reviews and discusses the many reasonable (and often patently superior) alternatives to the proposed regulatory action.
While courts sometimes differ on the necessary scope of alternatives, it is clear that even a minimal EIS should discuss several alternatives. The proposed regulation lists 8 different statutes (i.e., CAA, CWA, SDWA, RCRA, CERCLA, EPCRTKA, FIFRA, and TSCA) it will impact. See proposed regulation Federal Register, Vol. 83, No. 83 pg. 18769. Clearly, a reasonable discussion of alternatives would compare the proposed actions impacts with an alternative that impacts only 1 statute. Another alternative would apply to two of the listed statutes. Put simply each permutation of these statutes less than 8 statutes ought to be fully discussed in the EIS as a reasonable alternative. The EPA only has itself to blame for the complexity of the requisite EIS because it alone proposed a regulation that would affect 8 important statutes in one regulation.
Response: EPA disagrees that this rulemaking is subject to NEPA. See response to Comment 3117-104.
Comment [7876-4130]: Commenter (8279) argues that in addition to being bad for the environment, SDWA and TSCA lists mark the starting points for the required EIS that EPA must prepare, as required by NEPA, before the regulation is approved.
Response: EPA disagrees that this rulemaking is subject to NEPA. See response to Comment 3117-104.
Comment [8750-1427]: Commenter notes that the Proposal acknowledges that "[t]he best available science must serve as the foundation of EPA's regulatory actions." But it then requires EPA to systematically ignore the best available science when it regulates to protect human health and welfare. This is counter to EPA's statutory mandates to use "best available science," and the proposal is a transparent attempt not to strengthen science, but rather to censor science that is inconvenient to the current Administration's political goals.
Since EPA was established nearly half a century ago, the Agency and its leadership --  under Administrations of both parties -- have recognized the central role that rigorous science plays in fulfilling the Agency's mission of protecting human health and the environment. EPA's obligation to consider the best available science is not only a policy commitment that flows from the Agency's mission; it is a legal obligation enshrined in many of the fundamental public health and environmental statutes that EPA is charged with administering. The agency has established an array of mechanisms over the last five decades -- including "rigorous review" by its scientific advisory boards "that goes beyond the typical journal peer review procedures" -- to ensure that the Agency's decisions are grounded in the best available science.
Response: See response to Comments 2322-159, 1320-4331 and 1412-4472.
Comment [8750-1435]: Commenter (9227) writes that EPA's statutory authorities generally require the agency to consider all available data when undertaking significant rulemakings.
As just noted, EPA's statutory authorities mandate a variety of requirements for what scientific information EPA must consider in rulemaking. These statutes are discussed in detail, infra at Section I.B.3. To take one example that appears in numerous statutes, including TSCA, CAA, SDWA, and the Endangered Species Act, Congress has often required agencies to act on the "best available science." For an agency to comply with this obligation, the agency must at least consider all available scientific information. "Best" means "of the most excellent, effective, or desirable type or quality." "Available" means "able to be used or obtained." And "science" means "the intellectual and practical activity encompassing the systematic study of the structure and behavior of the physical and natural world through observation and experiment." Assessing which science is "best" requires consideration of the overall quality of the science, and the public availability of underlying data is, at best, one of many aspects that should inform that assessment of overall quality.
An agency "cannot ignore available. . . information." Numerous courts have indicated that a plaintiff or petitioner can establish a violation of the "best available science" requirement by "point[ing] to any scientific evidence that the agency failed to consider." "The best available data requirement. . . prohibits [an agency] from disregarding available scientific evidence that is in some way better than the evidence [it] relies on.'" "An agency does. . . have an obligation to deal with newly acquired evidence in some reasonable fashion." EPA's proposal will result in EPA precluding itself from considering certain studies that are "available," thus violating the requirement that EPA rely on the best available science.
In addition, the requirement that agencies use "best available" science or information often means that the agency must act even if the available science or information is imperfect.
"Even if the available scientific and commercial data were quite inconclusive, [the agency] may -- indeed must -- still rely on it" when the agency has a duty to act. "[W]here the information is not readily available, we cannot insist on perfection." Just as the Courts have recognized that they cannot expect perfection, agencies cannot choose to ignore certain studies or sources of information based solely on whether the data is publicly available -- especially where the validity of those studies has been established using techniques that do not rely on public availability of underlying data.
EPA cannot reasonably elevate the interest in public availability of all underlying information above all other factors in assessing the "best available science." Textually, EPA's approach is unlawful.
Response: See response to Comments 1412-4472, 2322-159 and 2322-161.
Comment [8750-1450]: Commenter (9227) suggests that even the Proposal appears to concede that studies for which data is not publicly available could constitute the "best available science" that EPA is statutorily required to consider. The proposed exemption provision in section 30.9 makes it clear that EPA does not consider a study to be invalid or unsuitable for EPA's consideration based only on the public unavailability of underlying data or models. Specifically section 30.9 would give the Administrator discretion to authorize consideration of a scientific study where "[i]t is not feasible to ensure that all dose response data and models underlying pivotal regulatory science is publicly available." Of course, EPA could not have intended for proposed section 30.9 to provide the Administrator with discretion to take a study that is not "best available science" into consideration when promulgating a rulemaking. If the Administrator has discretion to allow consideration of a study for which it is infeasible to make the study's underlying data and models publicly available, then it obviously is not necessary for such underlying data and models to be publicly available for a scientific study to constitute "best available science." Yet, unless the Administrator elects to exercise his discretion under proposed section 30.9 and find that it is "infeasible" to make a study's underlying data and models publicly available, proposed section 30.5 broadly prohibits EPA from relying on the study in support of "significant regulatory actions."
Moreover, while proposed section 30.5's prohibition would apply to "pivotal regulatory science" used for "significant regulatory actions," the proposed rule says nothing to prohibit EPA's reliance on these studies for other agency purposes, such as in permitting, enforcement, or regulatory actions that do not qualify as "significant." Thus, EPA clearly does not believe that a study cannot be "best available science" based solely on the fact that underlying data and models are not publicly available.
In sum, if finalized, EPA's proposed rule would restrict EPA's ability to consider "best available science" when undertaking significant rulemakings, contrary to the numerous statutory directives discussed in detail below.
Response: The Administrator's exemption provision in §30.7 is needed because there are conditions under which compliance with the requirements of the final rule might be impracticable, such as the age of the study. EPA finalized the exemption provision with reasonable factors for the Administrator to consider when granting an exemption. In addition, the final rule requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. 
Also see responses to Comments 2322-161, 2322-159 and 2322-161.
Comment [8750-1812]: Commenter (9227) asserts that EPA failed to submit the proposal to the SAB or to consult with the scientific and technical community.
There is no indication that EPA consulted with the scientific and technical community --  or even its own Science Advisory Board -- before proposing to require that the underlying data and models be made publicly available for all pivotal regulatory science regardless of ethical, feasibility, or confidentiality constraints. As detailed in a June 28, 2018 letter from the chair of the SAB, the SAB learned of the rule only through a press event, Federal Register notice, and news articles. The letter further explained that the proposed rule "was not identified as a major action in either of the Spring 2017 or Fall 2017 semi-annual Regulatory Agendas," and that SAB members "had no information regarding the timeline for finalizing the rule . . . ." The letter also points out that "the precise design of the proposed rule appears to have been developed without a public process for soliciting input specifically from the scientific community," even though the proposed rule raises important scientific questions.
Not surprisingly, the SAB concluded in its May 31, 2018 meeting that the Proposal merits SAB review because it "deals with issues of scientific practice and proposes constraints to the use of scientific studies in particular contexts." Moreover, the SAB chair's June 28 letter raises a number of questions that echo the concerns we have detailed in our comments, including the feasibility of providing access to data and methods for already-completed studies; "legitimate confidentiality and privacy interests" that would counsel against providing "complete public access"; the costs and effort associated with implementing the Proposal; the relationship between the Proposal and previous EPA efforts to encourage transparency; and the need to consider "the multiple existing methods to assess the validity of prior epidemiologic studies" that "do not provide public access to data and analytic methods."
EPA's failure to consult with the SAB is contrary to statute and to EPA's well established practice. EPA must submit its Proposal to the SAB pursuant to the requirements of 42 U.S.C. § 4365(c)(1) (the Environmental Research Development Demonstration Authorization Act or "ERDAA"), which requires the Administrator to submit to the SAB any proposed criteria document, standard, limitation, or regulation, together with relevant scientific and technical information in the possession of the (EPA) on which the proposed action is based at the time it provides that proposal to another agency of the government for formal review. The SAB must then review and comment on the proposal. While the Administrator need not receive the SAB's final approval, the Administrator must consider the SAB's advice and comments.
As the SAB chair's letter notes, EPA's "usual process" is to inform the SAB about the publication of the agency's semi-annual regulatory agenda and provide descriptions of actions that are contained in the agenda, including "available information regarding the science that is informing these agency actions." That procedure was not followed here. In its evident zeal in the name of purported "transparency," EPA has ignored major statutory and regulatory requirements that provide actual transparency to the CAA's scientific review process. Should EPA decide to move forward with this Proposal, it must first allow the SAB to complete its review and take into account the SAB's recommendations in any final rule.
Response: EPA disagrees that it violated the Environmental Research, Development, and Demonstration Authorization Act (ERDDAA). EPA did seek and obtain SAB review of aspects of the proposed rule. See https://yosemite.epa.gov/sab/sabproduct.nsf/LookupWebProjectsCurrentBOARD/2DB3986BB8390B308525855800630FCB/$File/EPA-SAB-20-005.pdf. 
EPA submitted the proposed rule to OIRA for review pursuant to Executive Order 12,866. EPA did not submit the proposal "to any other Federal agency for formal review and comment" so, in according to applicable caselaw, SAB review was not triggered. See Coalition for Responsible Regulation, Inc. v. EPA, 684 F.3d 102, 124 (D.C. Cir. 2012) (EPA's obligation under ERDDAA wasn't triggered because it wasn't clear that EPA provided the endangerment finding to any other Federal agency for formal review and comment). 
Comment [8750-4447]: Commenter (9227) contends that the proposal violates these statutory commands by requiring EPA to ignore science when undertaking significant rulemakings.
In direct violation of statutory requirements to consider, for example, "any other information available" or "the latest scientific knowledge [that is] useful" or "best available science," the Proposal would prohibit EPA from considering relevant and high quality science whenever the underlying data for a study is not publicly available. Through the Proposal, EPA unlawfully tries to engraft an additional statutory requirement onto each of these statutes, requiring that to be considered a study's underlying data must be publicly available. For EPA's Proposal to succeed, EPA must demonstrate that a study cannot be "other information available to the Administrator" or the "latest scientific knowledge useful in indicating" health or welfare effects or the "best available science," or any of a number of other statutory formulations if the underlying data is not publicly available. EPA's Proposal fails to do so, and it could not do so.
As explained infra at Section II.A.1, there are many reasons that underlying study data may not be available that have no bearing on the quality or validity of the study. These include legal restrictions or concerns about privacy (especially with respect to studies involving human subjects), confidentiality, CBI, or national security. Further, if this requirement were applied retroactively to existing studies, it may no longer be possible to make underlying data and models publicly available. EPA acknowledges these impediments in proposed section 30.9, which provides the Administrator with discretion -- but not an obligation -- to allow the agency to consider a study for which underlying data or models are not publicly available if he determines that public disclosure is infeasible. But where the Administrator fails to exercise his discretion to grant an exemption pursuant to proposed section 30.9, or where data or models are unavailable for reasons that do not satisfy the infeasibility standard, proposed section 30.5 would prohibit EPA from considering such studies, regardless of whether they meet the statutory criteria for consideration.
The only way that this prohibition could comport with EPA's statutory obligations is if a study for which underlying data is not available cannot be, for example, "other information available" or "the latest scientific knowledge [that is] useful" or "best available science" -- i.e., if the public unavailability of a study's underlying dose response data and models makes the study ineligible to meet these criteria, regardless of whether the study has been peer reviewed, is based on rigorous methodologies, or has been published in a leading journal, and regardless of the reason for the public unavailability. EPA makes no such demonstration -- nor could it. There is simply no support for such a proposition; to the contrary, all of the evidence shows that studies may be "best available science," and certainly "other information available" regardless of whether the data underlying them is publicly available.
Response: See response to Comments 2322-159, 2322-161, 8750-1450 and 1412-4472. 
Comment [8796-4133]: Commenter (8817) states that the adverse environmental impacts may be horrific and require full disclosure in the NEPA required EIS.
EPA disagrees that this rulemaking is subject to NEPA. See response to Comment 3117-104.
Comment [9078-3552]: Commenter (6194) contends that the proposal is inconsistent with EPA's statutory obligations and conflicts with other applicable federal requirements.
The Proposal would prevent EPA from relying on the best available information in discharging its duties. EPA is charged with developing regulations that provide societal benefits by reducing harm to human health and the environment from the presence of chemicals in air, soil, drinking water, food, and consumer products. The Proposal implicates research that is central to making such determinations; creating an obstacle to the consideration of such science is contrary to both primary statutory directives and requirements to conduct cost benefit analyses. Likewise, EPA fails to address how it could implement a regulation that is inconsistent with federal requirements and standards governing the use of science across agencies. 
A. Statutory Requirements to Develop Health Based Standards and Conduct Cost-Benefit Analyses EPA acknowledges that it must use the "best available science" in all of its regulatory actions. 
This directive is embodied in multiple statutes implemented by EPA and is reflected in other environmental statutes, further illustrating Congress' conceptualization of "best" available science. Broadly speaking, there are three basic types of statutory requirements that the EPA and other agencies consider scientific information. First, there are requirements that the agency consider the "best available science." Second, there are requirements that the agency consider particular factors, including scientific factors. Third, there are requirements that the agency balance economic costs or technological feasibility against public health, environmental, or other benefits that must be scientifically assessed. The precise terminology regarding the required use of science may, as illustrated below, vary across statutes, but what these provisions have in common are requirements for EPA and other agencies to use scientific information that is "best," not simply adequate, and "available." 
 
Table 1: Examples of Statutory Directives regarding Use of Science by EPA and other Agencies
 SDWA [42 U.S.C § 300g1(b)(3)(A)]: EPA must use "[t]he best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices," and "[d]ata collected by accepted methods or best available methods (if the reliability of the method and the nature of the excision justifies use of the data)." 
 [42 U.S.C. § 300g1(b)(1)(b)(ii)]: EPA must use "best available public health information" in determining whether to regulate contaminants.
 CWA [33 U.S.C. § 1321(a)(27)]: Some provisions dealing with oil and hazardous substances on the Gulf Coast require "best available science." 
 TSCA [15 U.S.C. § 2625(h)]: "In carrying out sections 2603, 2604, and 2605 of this title, to the extent that the Administrator makes a decision based on science, the Administrator shall use scientific information, technical procedures, measures, methods, protocols, methodologies, or models, employed in a manner consistent with the best available science".
 Magnuson -  Stevens Fishery Conservation and Management Act [16 U.S.C. § 1851(a)(2)]: "Conservation and management measures shall be based upon the best scientific information available."
 Endangered Species Act [16 U.S.C. § 1536(a)(2)]: In consultation to ensure no jeopardy to endangered or threatened species, "each agency shall use the best scientific and commercial data available." 
 [16 U.S.C. § 1536(c)(1)]: Advice that an endangered or threatened species may be present in a location should be "based on the best scientific and commercial data available."
 [16 U.S.C. § 1536(h)(2)(b)(i)16 U.S.C. § 1533(b)(1)(A)]: Permanent exceptions shall be given to prevent extinction of species "based on the best scientific and commercial data available." The Secretary must determine whether a species is a threatened or endangered based upon "the best scientific and commercial data available."
 Marine Mammal Protection Act [16 U.S.C. § 1371(a)(4)(C)]: "If the Secretary determines, using the best scientific information available, that certain forms of deterrence have a significant adverse effect on marine mammals, the Secretary may prohibit such deterrent methods, after notice and opportunity for public comment, through regulation under this chapter." 
 [16 U.S.C. § 1371(a)(3)(A)]: The decision to waive the ban on taking must be based on the "best scientific evidence available and in consultation with the Marine Mammal Commission."
None of these statutory provisions link the concept of the "best available science" to the availability of underlying raw data, nor can such a limitation be read into the statutes. Where Congress wanted to direct EPA's decision regarding what science to consider or how -- it did so. When Congress wanted to link "best available science" to data being publicly available -- it did so. Thus, if Congress had wanted to put a blanket limitation on EPA's consideration of science for which underlying data is not publicly available, it clearly knew how to do so -- and it chose not to.
As EPA has previously explained, the agency would be unable to fulfill statutory mandates if it could not consider studies that follow federal laws and ethical standards regarding the privacy of patient and individual data: 
      If EPA and other governmental agencies could not rely on published studies without conducting an independent analysis of the enormous volume of raw data underlying them, then much plainly relevant scientific information would become unavailable to EPA for use in setting standards to protect public health and the environment. . . . [S]uch data are often the property of scientific investigators and are often not readily available because of the proprietary interests of the investigators or because of arrangements made to maintain confidentiality regarding personal health status and lifestyle information of individuals included in such data. Without provisions of confidentiality, the possibility of conducting such studies could be severely compromised.
Examples of EPA statutory mandates that could be impeded by the Proposal include (i) the development of standards to control emissions from burning hazardous waste fuels under the Resource Conservation and Recovery Act ("RCRA") "as may be necessary to protect human health and the environment," 42 U.S.C. § 6924(q)(1); and (ii) determinations of whether hazardous sites need to be placed on the Comprehensive Environmental Response, Compensation, and Liability Act's ("CERCLA") National Priorities List. With respect to the latter, CERCLA provides that, if a health assessment indicates that a release or threatened release may pose a serious threat to human health, the Administrator of the ATSDR shall "notify the Administrator of EPA who shall promptly evaluate such release or threatened release in accordance with the hazard ranking system . . . to determine whether the site shall be placed on the National Priorities List." 42 U.S.C. § 9604 (i)(6)(H). As referenced in footnote 41, health assessments provided by ATSDR to EPA on a scheduled basis are exempt from peer review as a prerequisite to submission. 
The Proposal would also cut EPA off from information needed to complete the cost-benefit analysis of regulations required under certain statutes and government-wide requirements.
Not only would the Proposal impede EPA's fulfillment of its statutory duties, it would also be contrary to judicial precedent, which holds that "best available science" includes "all existing scientific evidence relevant to the [agency] decision [in question]." Ecology Ctr., Inc. v. U.S. Forest Serv., 451 F.3d 1183, 1194 n.4 (10th Cir. 2006) (quoting Heartwood, Inc. v. U.S. Forest Serv., 380 F.3d 428, 436 (8th Cir. 2004)). While agencies need not produce new evidence, they must seek out information and "`cannot ignore existing data.'" Id. (emphasis added). Even if an agency is anticipating the arrival of better evidence, it must act on the basis of the evidence it currently has. Many of the fundamental public health studies on which EPA has based key rules and standards under the statutes are studies for which the raw data was not or could not have been released. Nonetheless these studies have been examined and validated, as will future studies. Arbitrarily cutting off consideration of a category of science is inconsistent with EPA's statutory obligations.
Response: This rule describes how the EPA will determine the consideration to afford pivotal science of the EPA's final significant regulatory actions and influential scientific information based on the availability of the underlying dose-response data and other applicable factors. The rule applies only to EPA and will not impact standards governing the use of science across other agencies. See response to Comments 1320-4331, 2322-159, 2322-161, 2322-205, and 1412-4472.
Comment [9217-3564]: Commenter (9225) writes that the proposed rule is not authorized by any authority delegated to EPA by Congress and is contrary to a number of statutes under EPA's authority. This includes, but is not limited to, the CAA; CWA; TSCA; Lautenberg Chemical Safety Act; SDWA; FIFRA; and more. 
Response: See response to Comments 31-135 and 1412-4472.
1.4 Evidence Act (SNPRM Comments)
Comment [49-214]: Commenter (12718) states that the Bipartisan Policy Center (BPC) dialogue squarely placed future open data and open science activities at EPA within the context of the Foundations for Evidence-Based Policymaking Act of 2018 (P.L. 115-435; Evidence Act), which EPA neglects to reference, cite, or align in the SNPRM. Instead EPA relies on OMB M-13-13, rather than new legal requirements signed into law during the current administration. The absence of incorporating the Evidence Act, which fully applies to EPA, in the course of the proposed regulatory action is a misstep and major omission. Excluding the bipartisan Evidence Act throughout the SNPRM also raises questions about the motivation and intent of EPA's current proposal, not to mention concerns about the fidelity EPA is employing in implementing current law expectations under the Evidence Act.
Response: The Agency did not cite the Evidence Act in either proposal but does acknowledge it in the final rule preamble. Nothing in the final rule conflicts with or contradicts the Act. The final rule furthers the goal of the Act by increasing transparency of the science that underlies significant regulatory actions and ISI. Further, EPA's Plan to Increase Access to Results of EPA Funded Scientific Research established procedures for providing public access to studies, funded in whole or in part by EPA, and their underlying data and models. 
Comment [49-216]: Commenter (12718) suggests that the final EPA regulatory action could still calibrate to the Evidence Act's new legal authority, while also addressing issues identified from across the scientific community about the SNPRM. A report released by BPC in early 2020 from our dialogues with EPA stakeholder-representatives offered the following suggestions to promote meaningful transparency at the agency: 
 Strengthen EPA's Learning Culture
 Improve EPA's Data Government and Management
 Enhance EPA's Policy Analysis and Evaluation Functions
 Bolster Public Trust and Enrich Communication
In short, the project highlighted that being serious about meaningful transparency required multiple threads of activities. It suggested that a single action to encourage data availability would be insufficient at really supporting transparency and accountability as envisioned and unanimously agreed to by the U.S. Commission on Evidence-Based Policymaking. Below I highlight several options from this report that EPA could still incorporate in conjunction with SNPRM revisions to promote meaningful and real transparency for the Agency:
      #3: Explore New Models for Incorporating Expert and Stakeholder Feedback
      #7: Establish and Empower an Office of the Chief Data Officer
      #8: Establish and Prioritize and Agency-Wide Data Governance Board
      #9: Establish and Sufficiently Resource a Legally Recognized Statistical Unit
      #13: Establish or Identify a Partnership for a Secure Data Enclave
      #14: Establish an Independent and Centralized Evaluation Unit
      #16: Support International Environmental Systematic-Review Processes
      #18: Establish an Advisory Body for Evidence-Building Activities
Taken together, these options could create an infrastructure and ecosystem at EPA that would position the agency as a leader in the federal government and the world for promoting transparency. Additional details about each option and the implications for EPA are available in BPC's final options paper on the EPA dialogues. 
Response: The EPA does not intend for this final rule to serve as the agency action taken to comply with the Evidence Act. A workgroup has been established to develop implementation plans to address the requirements of the Act, which may cover the activities described in the comment.
1.5 Consistency with the Peer Review Processes Required Under Environmental Statue (NPRM Comments)
Comment [6163-1276]: Commenter (6133) states that in addition to addressing how and whether the agency will consider science, the Proposal also contains a seemingly unrelated provision regarding agency peer review. The Proposal, in § 30.7, reads: 
      What role does independent peer review in this section?
      EPA shall conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein. 
Because transparency in regulatory science includes addressing issues associated with assumptions used in models, EPA shall ask peer reviewers to articulate the strengths and weaknesses of EPA's justification for the assumptions applied and the implications of those assumptions for the results. (83 Fed. Reg. at 18,774.)
There is no statutory authority for EPA to "conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein." The federal statutes that EPA lists as putative authority for the Proposal provide no authority for proposed § 30.7. See 83 Fed. Reg. at 18,769/2 (citing CAA sections 103, 301(a), 42 U.S.C. 7403, 7601(a); CWA sections 104, 501, 33 U.S.C. 1254, 1361; SDWA sections 1442, 1450(a)(1), 42 U.S.C. 300j - 1, 300j -  9(a)(1); RCRA sections 2002(a)(1), 7009, 42 U.S.C. 6912(a)(1), 6979; CERCLA (as delegated to the Administrator via Executive Order 12580) sections 115, 311, 42 U.S.C. 9616, 9660; EPCRA section 328, 42 U.S.C. 11048; FIFRA sections 25(a)(1), 136r(a), 7 U.S.C. 136r(a), 136w; and TSCA, as amended, section 10, 15 U.S.C. 2609, and the APA). The claimed authorities that EPA lists do not mention peer review or even allude to the concept. Id. Neither the Proposal nor accompanying docket materials identify any provision of any federal statute that authorizes EPA to promulgate proposed § 30.7. Our own research revealed no provision of any federal statute that authorizes EPA to promulgate proposed § 30.7. 
When Congress writes federal statutes, Congress knows how to create legal authority for peer review, who shall conduct that peer review, what role, if any, that EPA or other parties will play, and how that peer review may be conducted. None of the provision in the law authorize EPA's Proposal in § 30.7. Instead, the statutes require that EPA use peer-reviewed science, regardless of whether it would meet EPA's definition of the term in § 30.7. And in cases where the law requires EPA to conduct the review, the statutes often spell out specifically how that should happen. 
Statute Provisions on Peer Review CAA § 7511b. Federal ozone measures: 
*************
CAA
§ 7511b. Federal ozone measures
(g) Ozone design value study The Administrator shall conduct a study of whether the methodology in use by the EPA as of November 15, 1990, for establishing a design value for ozone provides a reasonable indicator of the ozone air quality of ozone nonattainment areas. The Administrator shall obtain input from States, local subdivisions thereof, and others. The study shall be completed, and a report submitted to Congress not later than 3 years after November 15, 1990. The results of the study shall be subject to peer and public review before submitting it to Congress. (42 U.S.C. § 7511b (emphasis added)).
§ 7412. HAP 
(p) NUATRC (3) SAP the Board of Directors shall be advised by a SAP, the 13 members of which shall be appointed by the Board, and to include eminent members of the scientific and medical communities. The Panel membership may include scientists with relevant experience from the NIEHS, the CDC, the EPA, the NCI, and others, and the Panel shall conduct peer review and evaluate research results. The Panel shall assist the Board in developing the research agenda, reviewing proposals and applications, and advise on the awarding of research grants. (42 U.S.C. § 7412 (emphasis added)). 
CWA
§ 1321. Oil and hazardous substance liability 
(a) Definitions (27) the term "best available science" means science that-- (A) maximizes the quality, objectivity, and integrity of information, including statistical information; (B) uses peer-reviewed and publicly available data; and (C) clearly documents and communicates risks and uncertainties in the scientific basis for such projects; (33 U.S.C. § 1321 (emphasis added)).
SDWA
§ 300g-1. National drinking water regulations 
(b) Standards 
(3) Risk assessment, management, and communication
(A) Use of science in decision-making in carrying out this section, and, to the degree that an Agency action is based on science, the Administrator shall use--
(i) the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices; and(ii) data collected by accepted methods or best available methods (if the reliability of the method and the nature of the decision justifies use of the data).
(B) Public information
In carrying out this section, the Administrator shall ensure that the presentation of information on public health effects is comprehensive, informative, and understandable. The Administrator shall, in a document made available to the public in support of a regulation promulgated under this section, specify, to the extent practicable--
(i) each population addressed by any estimate of public health effects;
(ii) the expected risk or central estimate of risk for the specific populations;
(iii) each appropriate upper-bound or lower-bound estimate of risk;
(iv) each significant uncertainty identified in the process of the assessment of public health effects and studies that would assist in resolving the uncertainty;
(v) peer-reviewed studies known to the Administrator that support, are directly relevant to, or fail to support any estimate of public health effects and the methodology used to reconcile inconsistencies in the scientific data.
(12) Certain contaminants
(B) Sulfate
(i) Additional study Prior to promulgating a national primary drinking water regulation for sulfate, the Administrator and the Director of the CDC shall jointly conduct an additional study to establish a reliable dose-response relationship for the adverse human health effects that may result from exposure to sulfate in drinking water, including the health effects that may be experienced by groups within the general population (including infants and travelers) that are potentially at greater risk of adverse health effects as the result of such exposure. The study shall be conducted in consultation with interested States, shall be based on the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices, and shall be completed not later than 30 months after August 6, 1996. (42 U.S.C. § 300g-1 (emphasis added)).
§ 300j-2. Grants for State programs 
(d) NYC watershed protection program
(1) In general The Administrator is authorized to provide financial assistance to the State of New York for demonstration projects implemented as part of the watershed program for the protection and enhancement of the quality of source waters of the NYC water supply system, including projects that demonstrate, assess, or provide for comprehensive monitoring and surveillance and projects necessary to comply with the criteria for avoiding filtration contained in 40 C.F.R. 141.71. Demonstration projects which shall be eligible for financial assistance shall be certified to the Administrator by the State of New York as satisfying the purposes of this subsection. In certifying projects to the Administrator, the State of New York shall give priority to monitoring projects that have undergone peer review. (42 U.S.C. § 300j-2 (emphasis added)).
RCRA 
§ 6939a. Exposure information and health assessments
(b) Health assessments
(2) Whenever in the judgment of the Administrator, or the State (in the case of a State with an authorized program), a landfill or a surface impoundment poses a substantial potential risk to human health, due to the existence of releases of hazardous constituents, the magnitude of contamination with hazardous constituents which may be the result of a release, or the magnitude of the population exposed to such release or contamination, the Administrator or the State (with the concurrence of the Administrator) may request the Administrator of the ATSDR to conduct a health assessment in connection with such facility and take other appropriate action with respect to such risks as authorized by section 9604(b) and (i) of this title. If funds are provided in connection with such request the Administrator of such Agency shall conduct such health assessment.
(e) Periodic reports. The Administrator of such Agency shall issue periodic reports which include the results of all the assessments carried out under this section. Such assessments or other activities shall be reported after appropriate peer review. (42 U.S.C. § 6939a (emphasis added)).
CERCLA
§ 9604. Response authorities 
(i) ATSDR; establishment, functions, etc.
Any toxicological profile or revision thereof shall reflect the Administrator of ATSDR's assessment of all relevant toxicological testing which has been peer reviewed. The profiles required to be prepared under this paragraph for those hazardous substances listed under subparagraph (A) of paragraph (2) shall be completed, at a rate of no fewer than 25 per year, within 4 years after October 17, 1986. A profile required on a substance listed pursuant to subparagraph (B) of paragraph (2) shall be completed within 3 years after addition to the list. The profiles prepared under this paragraph shall be of those substances highest on the list of priorities under paragraph (2) for which profiles have not previously been prepared. Profiles required under this paragraph shall be revised and republished as necessary, but no less often than once every 3 years. Such profiles shall be provided to the States and made available to other interested parties.
(7) (A) Whenever in the judgment of the Administrator of ATSDR it is appropriate on the basis of the results of a health assessment, the Administrator of ATSDR shall conduct a pilot study of health effects for selected groups of exposed individuals in order to determine the desirability of conducting full scale epidemiological or other health studies of the entire exposed population. (B) Whenever in the judgment of the Administrator of ATSDR it is appropriate on the basis of the results of such pilot study or other study or health assessment, the Administrator of ATSDR shall conduct such full scale epidemiological or other health studies as may be necessary to determine the health effects on the population exposed to hazardous substances from a release or threatened release. If a significant excess of disease in a population is identified, the letter of transmittal of such study shall include an assessment of other risk factors, other than a release, that may, in the judgment of the peer review group, be associated with such disease, if such risk factors were not taken into account in the design or conduct of the study.
(13) All studies and results of research conducted under this subsection (other than health assessments) shall be reported or adopted only after appropriate peer review. Such peer review shall be completed, to the maximum extent practicable, within a period of 60 days. In the case of research conducted under the NTP, such peer review may be conducted by the BOSC. In the case of other research, such peer review shall be conducted by panels consisting of no less than three nor more than seven members, who shall be disinterested scientific experts selected for such purpose by the Administrator of ATSDR or the Administrator of EPA, as appropriate, on the basis of their reputation for scientific objectivity and the lack of institutional ties with any person involved in the conduct of the study or research under review. Support services for such panels shall be provided by the ATSDR, or by the EPA, as appropriate. (42 U.S.C. § 9604 (emphasis added)).
EPCRA
No mentions of peer review 
FIFRA
§ 136w. Authority of Administrator 
(e) Peer review The Administrator shall, by written procedures, provide for peer review with respect to the design, protocols, and conduct of major scientific studies conducted under this subchapter by the EPA or by any other Federal agency, any State or political subdivision thereof, or any institution or individual under grant, contract, or cooperative agreement from or with the EPA. In such procedures, the Administrator shall also provide for peer review, using the advisory panel established under subsection (d) of this section or appropriate experts appointed by the Administrator from a current list of nominees maintained by such panel, with respect to the results of any such scientific studies relied upon by the Administrator with respect to actions the Administrator may take relating to the change in classification, suspension, or cancellation of a pesticide. Whenever the Administrator determines that circumstances do not permit the peer review of the results of any such scientific study prior to the Administrator's exercising authority under section 136d(c) of this title to immediately suspend the registration of any pesticide to prevent an imminent hazard, the Administrator shall promptly thereafter provide for the conduct of peer review as provided in this sentence. The evaluations and relevant documentation constituting the peer review that relate to the proposed scientific studies and the results of the completed scientific studies shall be included in the submission for comment forwarded by the Administrator to the advisory panel as provided in subsection (d). As used in this subsection, the term "peer review" shall mean an independent evaluation by scientific experts, either within or outside the EPA, in the appropriate disciplines. (7 U.S.C. § 136w (emphasis added)).
§ 136w-8. Pesticide registration service fees 
(a) Definition of costs: In this section, the term "costs", when used with respect to review and decision-making pertaining to an application for which registration service fees are paid under this section, means-- (1) costs to the extent that -- 
(A) officers and employees provide direct support for the review and decision-making for covered pesticide applications, associated tolerances, and corresponding risk and benefits information and analyses; (B) persons and organizations under contract with the Administrator engage in the review of the applications, and corresponding risk and benefits information and assessments; and (C) advisory committees and other accredited persons or organizations, on the request of the Administrator, engage in the peer review of risk or benefits information associated with covered pesticide applications; (2) costs of management of information, and the acquisition, maintenance, and repair of computer and telecommunication resources (including software), used to support review of pesticide applications, associated tolerances, and corresponding risk and benefits information and analyses; and (3) costs of collecting registration service fees under subsections (b) and (c) and reporting, auditing, and accounting under this section. (7 U.S.C. § 136w-8 (emphasis added)).
TSCA
§ 2625. Administration 
(h) Scientific standards In carrying out sections 2603, 2604, and 2605 of this title, to the extent that the Administrator makes a decision based on science, the Administrator shall use scientific information, technical procedures, measures, methods, protocols, methodologies, or models, employed in a manner consistent with the best available science, and shall consider as applicable-- (1) the extent to which the scientific information, technical procedures, measures, methods, protocols, methodologies, or models employed to generate the information are reasonable for and consistent with the intended use of the information; (2) the extent to which the information is relevant for the Administrator's use in making a decision about a chemical substance or mixture; (3) the degree of clarity and completeness with which the data, assumptions, methods, quality assurance, and analyses employed to generate the information are documented; (4) the extent to which the variability and uncertainty in the information, or in the procedures, measures, methods, protocols, methodologies, or models, are evaluated and characterized; and (5) the extent of independent verification or peer review of the information or of the procedures, measures, methods, protocols, methodologies, or models. (15 U.S.C. § 2625 (emphasis added)).
§ 2617. Preemption 
(b) New statutes, criminal penalties, or administrative actions creating prohibitions or other restrictions (1) In general Except as provided in subsections (c), (d), (e), (f), and (g), beginning on the date on which the Administrator defines the scope of a risk evaluation for a chemical substance under section 2605(b)(4)(D) of this title and ending on the date on which the deadline established pursuant to section 2605(b)(4)(G) of this title for completion of the risk evaluation expires, or on the date on which the Administrator publishes the risk evaluation under section 2605(b)(4)(C) of this title, whichever is earlier, no State or political subdivision of a State may establish a statute, criminal penalty, or administrative action prohibiting or otherwise restricting the manufacture, processing, distribution in commerce, or use of such chemical substance that is a high-priority substance designated under section 2605(b)(1)(B)(i) of this title.
(f) Waivers (2) Required exemptions Upon application of a State or political subdivision of a State, the Administrator shall exempt from subsection (b) a statute or administrative action of a State or political subdivision of a State that relates to the effects of exposure to a chemical substance under the conditions of use if the Administrator determines that-- (A)(i) compliance with the proposed requirement of the State or political subdivision of the State would not unduly burden interstate commerce in the manufacture, processing, distribution in commerce, or use of a chemical substance; (ii) compliance with the proposed requirement of the State or political subdivision of the State would not cause a violation of any applicable Federal law, rule, or order; and (iii) the State or political subdivision of the State has a concern about the chemical substance or use of the chemical substance based in peer-reviewed science; or (B) no later than the date that is 18 months after the date on which the Administrator has initiated the prioritization process for a chemical substance under the rule promulgated pursuant to section 2605(b)(1)(A) of this title, or the date on which the Administrator publishes the scope of the risk evaluation for a chemical substance under section 2605(b)(4)(D) of this title, whichever is sooner, the State or political subdivision of the State has enacted a statute or proposed or finalized an administrative action intended to prohibit or otherwise restrict the manufacture, processing, distribution in commerce, or use of the chemical substance. (15 U.S.C. § 2617 (emphasis added)).
§ 2605. Prioritization, risk evaluation, and regulation of chemical substances and mixtures 
(b) Risk evaluations (E) Metals and metal compounds In identifying priorities for risk evaluation and conducting risk evaluations of metals and metal compounds, the Administrator shall use the Framework for Metals Risk Assessment of the Office of the Science Advisor, Risk Assessment Forum, and dated March 2007, or a successor document that addresses metals risk assessment and is peer reviewed by the SAB. (15 U.S.C. § 2605 (emphasis added)).
Under longstanding federal case law, when Congress authorizes an approach in one section of a statute using specific language but does not do the same in other sections of a statute, courts presume that Congress acted purposefully and did not mean to address or authorize that approach in those other statutory sections. See, e.g., Dean v. United States, 556 U.S. 568 (2009) ("It is generally presumed that Congress acts intentionally when including particular language in one section of a statute but not in another." (citing Russello v. United States, 464 U.S. 16, 23 (1983)). Not only are there no implied grants of authority to an agency in the other statutory sections, but the congressional decisions to authorize the approach elsewhere in the statute give even greater force to the conclusion that the agency has not been given authority where Congress did not use the same or similar authorizing language. 
The EPA approach proposed in § 30.7 is even more unlawful than would be the case, independently, under this case law. Proposed § 30.7 says "EPA shall conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein." 83 Fed. Reg. at 18,774/2 (emphasis added). EPA proposes to bind itself ("shall") to conduct a particular form of peer review on "all pivotal regulatory science" based on an unenforceable, non-binding OMB bulletin that has never been the subject of notice and comment rulemaking and that itself is not authorized by any federal law. To the contrary, treating the content of the unenforceable bulletin as a binding regulation would itself violate the federal statutes that EPA implements, because those statutes do not codify the bulletin, and EPA would be unlawfully codifying a mere policy preference. This EPA may not do. 
This Proposal is unlawful, arbitrary, and capricious, and an abuse of EPA discretion. To reiterate, the Proposal identifies no statutory authority for EPA to conduct independent peer review for any, much less all, "pivotal regulatory science" consistent with the dense content (and exceptions) of the OMB bulletin. The Proposal identifies no statutory authority for EPA to bind itself, and future administrations, to conduct peer review only in this fashion, unless and until future notice-and-comment rulemaking is undertaken. The Proposal identifies no suggestion in statutory language or legislative history that Congress intended EPA to conduct binding peer review consistent with this OMB bulletin, notwithstanding that Congress has known about this bulletin since 2005. EPA has simply made up proposed § 30.7 -- with its the link to the OMB bulletin, and the putative authority for the Proposal -- out of whole cloth. This EPA may not do. 
Response: None of the statutory provisions cited by the commenter prohibits EPA from conducting additional peer review of studies the agency evaluates as support for significant regulatory decisions and ISI. That said, the final rule provides that if EPA determines that studies have already undergone adequate peer review, EPA will not conduct de novo independent peer review. Adequate peer review is peer review that is consistent with both OMB's and EPA's peer review guidance. The intent of the rule is for EPA to go beyond current OMB/EPA guidance by recognizing that peer review that actually examined the underlying data will lead to even more confidence in the results of a peer reviewed study and that pivotal science that has had this deeper peer review will be given greater consideration in study quality weighting.
The final rule states in § 30.3 that "[i]n the event the procedures outlined in this rule conflict with statutes that EPA administers, or their implementing regulations, the statutes and regulations will control." Additionally, EPA intends to promulgate future science transparency rules under its substantive environmental statutes that may address peer review requirements consistent with those authorities.
Comment [7869-3376]: Commenter (8272) contends that EPA's regulatory process already involves peer review, solicitation of expert opinion, steps to ensure data quality, and public comment periods that allow stakeholders to raise questions or concerns and present additional evidence. If it had undertaken a similarly thorough process in developing this rule, it could have asked the National Academies of Sciences, Engineering, and Medicine (NASEM) to investigate and report on whether the current level of transparency has resulted in outcomes contrary to EPA's statutory mandates or executive orders; procedures and considerations related to transparency in the scientific community; and the potential results of applying the proposed criteria when regulating. 
Response: EPA agrees with the commenter that its regulatory processes provide for peer review, solicitation of expert opinion, steps to ensure data quality, and public comment periods. While EPA did not consult with NASEM on this rule, it did solicit peer review and expert opinion from the SAB and provided for public notice and comment.
Comment [8750-1449]: Commenter (9227) asserts that as EPA acknowledges in footnote 3 of the Proposal, in at least two instances the D.C. Circuit Court of Appeals has recognized that studies for which underlying data is not publicly available may constitute "best available science." The D.C. Circuit's decisions in these cases further demonstrate that the public unavailability of a study's underlying data does not render a study incapable of constituting "best available science" otherwise unworthy of EPA's consideration.
In American Trucking Associations v. EPA, the petitioner challenged EPA's reliance on scientific studies for which underlying data was not publicly available in deciding to strengthen the NAAQS for PM. The Court held that the CAA did not require EPA to make public underlying data where EPA relied on the study itself and not the raw data underlying the study. The Court agreed with EPA's position that requiring agencies to obtain and publicize the data underlying all studies on which they rely "would be impractical and unnecessary." Importantly, the Court concluded that:
If EPA and other governmental agencies could not rely on published studies without conducting an independent analysis of the enormous volume of raw data underlying them, then much plainly relevant scientific information would become unavailable to EPA for use in setting standards to protect public health and the environment. . . . Such data are often the property of scientific investigators and are often not readily available because of. . .proprietary interests. . . or because of [confidentiality] arrangements [with study participants].
The court accordingly recognized that ignoring relevant scientific information simply because the underlying data is not available would violate EPA's obligations to consider "best available science." Coalition of Battery Recyclers Association v. EPA involved another challenge to EPA's reliance on a scientific study for which the underlying data was not publicly available. In that case, EPA had relied upon the study in question to determine the "concentration-response relationship between blood lead levels and IQ changes." The D.C. Circuit again upheld EPA's reliance on studies without making the underlying data publicly available and explained, "raw data often is unavailable due to proprietary interests of a study's scientific investigators or confidentiality agreements with study participants." Likewise, in City of Waukesha v. EPA the D.C. Circuit concluded that agency peer review satisfies the requirement to use best, peer reviewed science and supporting studies.
Response: While EPA recognizes that courts have held that the agency can rely on studies where the underlying data and models are not publicly available, EPA finds that public confidence in the regulatory actions taken by EPA will be enhanced by increased transparency and the ability of the public to conduct independent validation of the studies' findings and conclusions.
Comment [8750-1629]: Commenter (9227) contends that there is broad agreement in the scientific literature, reflected in EPA's own guidance, that a "weight of the evidence" approach is an optimal way to analyze and synthesize an array of scientific information in a decision-making context. This approach, which is described in more detail below, calls for scientific assessments to be based on a broad array of studies -- reflecting multiple lines of inquiry, where appropriate -- each of which is carefully weighted based on various indicia of credibility. This careful and rigorous process is incompatible with the requirements of the Proposal, which would bar EPA from considering even highly credible, persuasive studies based solely on whether the underlying data is available. Yet the Proposal never acknowledges the conflict between its requirements and EPA's proven practices for scientific assessments, and never provides any good reasons for this change of course. One prominent example of this "weight of the evidence" approach is contained in EPA's Preamble to the ISA. The ISA are pollutant-specific reports that EPA produces as the scientific basis for establishing and updating EPA's NAAQS, which establish health-based standards for critical air pollutants. The ISA are intended to implement the CAA's directive to "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of identifiable effects on public health and welfare which may be expected from the presence of [a] pollutant in the ambient air." These are some of the most consequential scientific evaluations that EPA performs, in terms of the health, environmental, and economic impacts of the resulting standards, and they must withstand the highest level of technical and legal scrutiny. Thus, EPA uses the very best and most defensible scientific methods to produce them, which are described in the Preamble to the ISA.
The Preamble to the ISA is an "overview document outlining the basic steps and criteria used in developing the Integrated Science Assessments," which EPA references as a companion document to each ISA. As EPA explains, the "Preamble describes the process of searching the literature, selecting studies for consideration, evaluating study quality, synthesizing and integrating the evidence, and characterizing the evidence for public health and welfare impacts of criteria air pollutants." It also "describes the five-level causal framework for evaluating weight of evidence and drawing scientific conclusions and causal judgments." Central to this scientific assessment process is the understanding that evidence from all types of studies, such as animal studies, human observational studies (cohort, time series), controlled chamber studies, and exposure assessments, among others, must be evaluated and incorporated into final determinations of effects. No single study alone drives the final determinations of causality; rather, the weight of evidence from several lines of inquiry is critical. This framework to evaluate all available science builds upon decades of accrued knowledge and thinking drawing from expertise across several disciplines, including evidence-based decision making.
The Preamble states: "In its evaluation and integration of the scientific evidence on health or welfare effects of criteria pollutants, the U.S. EPA determines the weight of evidence in support of causation and characterizes the strength of any resulting causal classification." The Preamble explains in further detail:
In the ISA, the U.S. EPA assesses the body of relevant literature, building upon evidence available during previous NAAQS reviews, to draw conclusions on the causal relationships between relevant pollutant exposures and health or environmental effects. ISAs use a five-level hierarchy that classifies the weight of evidence for causation. This weight-of-evidence evaluation is based on the integration of findings from various lines of evidence from across health and environmental effect disciplines that are integrated into a qualitative statement about the overall weight of the evidence and causality.
Similarly, section 26 of the TSCA requires that decisions made under sections 4, 5, or 6 of the law must adhere to certain scientific standards including use of best available science and a weight of the scientific evidence approach. In its final regulation, Procedures for Chemical Risk Evaluation Under the Amended TSCA, EPA defines weight of scientific evidence as:
Weight of scientific evidence means a systematic review method, applied in a manner suited to the nature of the evidence or decision, that uses a pre-established protocol to comprehensively, objectively, transparently, and consistently, identify and evaluate each stream of evidence, including strengths, limitations, and relevance of each study and to integrate evidence as necessary and appropriate based upon strengths, limitations, and relevance.
Systematic review in turn requires a full review of the body of scientific evidence available, where study quality is evaluated largely according to methodological design and not the degree to which underlying data are publicly available. EPA's Proposal contravenes TSCA's requirements to apply a weight of the scientific evidence approach, as defined by the agency, by instating a process that, among other things, conflicts with applying a systematic review approach in the evaluation of chemicals under TSCA.
The Proposal's approach of preemptively barring studies based on the unavailability of data cannot be reconciled with EPA's detailed policies for scientific assessment.
Response: The final rule does not categorically exclude studies from consideration if the underlying data and models are not publicly in a manner to allow independent validation. The final rule describes how EPA will consider studies underlying significant regulatory actions and ISI, based on data and model availability and study quality factors. EPA will continue utilizing systematic review and weight of the evidence processes to evaluate scientific studies.
1.6 Administrative Procedures Act (APA)
1.6.1 APA (NPRM and SNPRM Comments).
1.6.1.1 APA (NPRM Comments)
1.6.1.1.1 Was the Proposed Rule Consistent with the APA?
Comment [2157-149]: Commenter (2171) suggests that the intent of this rule is ill-considered, and its potential for implementation is severely limited. EPA asked for input in the public notice that would have been better addressed through an Advanced NPRM. The record provided with the docket for this proposed rule is nonexistent, and EPA has failed to meet many of the basic requirements in preparing rule proposals for public notice and comment. The rule will likely be determined to be arbitrary and capricious and is certain to be challenged legally. The commenter states that the EPA should withdraw this proposed rule immediately.
Response: This rulemaking establishes internal EPA procedures for considering the dose-response data underlying pivotal science to be used in EPA's significant regulatory decisions. Because this is a procedural rulemaking that exclusively governs EPA and no other party outside of the agency, it is not subject to the APA's notice and comment requirements. See 5 U.S.C. § 553(b) (exempting "rules of agency organization, procedure, or practice" from § 553 notice and comment requirements) (emphasis added). Nevertheless, and consistent with § 553(b), the proposal, supplemental proposal, and other relevant rulemaking notices were published in the Federal Register and included: "(1) a statement of the time, place, and nature of public rule making proceedings; (2) reference to the legal authority under which the rule is proposed; and (3) either the terms or substance of the proposed rule or a description of the subjects and issues involved." Id. Taken together, these documents provided enough detail and rationale to support this action and for the public to meaningfully comment on the rulemaking.
Moreover, EPA made relevant technical and supporting documents available to the public in the rulemaking docket, and those documents are consistent with the stated rationale for the rule. Finally, in addition to meeting the notice requirements of § 553(b), both the proposal and supplemental proposal were subject to public comment. The proposal received nearly 130,000 comments and the supplemental proposal received over 70,000 comments. 
Comment [6906-3975]: Commenter (6927) agrees that this proposed rule is consistent with the APA as well as programmatic statutes that EPA administers to disclose to the public the bases for agency rules and to rationally execute and adequately explain agency actions.
Response: The environmental statutes EPA administers do indeed require that EPA provide the bases for decision making. This rule advances that transparency by giving greater consideration to pivotal science that is available for independent validation.
Comment [3117-103]: Commenter (3132) argues that the proposal violates the APA in many ways. (See, e.g., May 7, 2018 Comment Letter of Attorneys General for seven states and the District of Columbia at page 1 and the May 7, 2018 Comment Letter of the Environmental Protection Network (EPN) (included in Docket for this rulemaking)). 
For example, the commenter contends that the proposal is both impermissibly misleading and ambiguous. APA section 553 requires the EPA to provide a concise general statement for the proposal. In contrast, the proposal obfuscates rather than informs. Rather than concede that the proposal would be a dramatic departure from past EPA practices, the proposal tries to downplay court and agency decisions that support the use of the data banned under the proposal. See Proposal, FR 18769 note 3. The vagueness and misleading nature of statements regarding the proposal thwart the fundamental APA goal of keeping the public informed.
Response: See response to Comment 2157-149.
Comment [4399-3922]: Commenter (4410) states that the promulgation of this proposed rule would set a dangerous and potentially life-threatening precedent regarding the use of health-based data, modeling, and research in regulatory decision-making. As proposed, the rule is arbitrary, capricious, unethical, and intellectually dishonest. The commenter suggests that the EPA should immediately announce that it is withdrawing this proposal.
Response: See response to Comment 2157-149. EPA notes that this rule is not being finalized as proposed.
Comment [4577-332]: Commenter (4588) states that the aim of the EPA's proposed rule is to make "data and models underlying the science publicly available in a manner sufficient for validation and analysis." The commenter strongly supports the open exchange of information to ensure the validity of and to build upon research. However, the commenter believes the proposed rule is unnecessary, makes erroneous assumptions about the scientific process, and would unadvisedly limit the kind of scientific research that should be considered in decisions affecting human health and our environment. Moreover, this proposed policy change would unnecessarily, and likely unlawfully, lead the EPA away from its mandate -- in its mission and under various environmental statutes -- to use the best scientific evidence in assessing environmental risks, and its obligation under the APA to refrain from arbitrary decision making. The practical effect of this rule would be to limit EPA's ability to protect public health and meet its statutory obligations as well as its overall mission.
The commenter provides that a Congressional bill similar in intent to the EPA's proposed rule  -  the HONEST Act  -  was offered and defeated last year. Members of Congress had serious and valid concerns about the negative impact the bill would have by significantly reducing the amount of scientific evidence that EPA can consider when adopting regulatory standards to protect public health and the environment. Given the requirements of the APA and federal environmental statutes, the agency cannot do by regulation what Congress declined to do through legislation.
Response: EPA disagrees that this rulemaking will compromise its ability to carry out its mandate to protect human health and the environment and meet its statutory obligations. By contrast, and as discussed in the final rule preamble at III.A, EPA is taking this procedural action to help strengthen the transparency of the data and models underlying EPA's significant regulatory actions and influential scientific information, which will further advance EPA's mission. 
As explained in III.E, the final rule adopts a weighting approach to considering data availability and not the categorical exclusion presented as an option in the proposal and supplemental proposal. This procedural rule does not interpret or apply provisions of the statutes that it administers. EPA has explained that this rule has limitations on its scope in terms of conflicts with other substantive requirements and that, in situations where this rule applies, EPA will give greater consideration to those studies where the underlying data and models are available for independent validation. If implementing this procedural rule will result in a conflict with an environmental statute that EPA administers, then this procedural rule will yield.
Moreover, EPA plans to issue statute-specific transparency regulations or programmatic regulations. See the final rule preamble at III.A. EPA will conduct any necessary conflict analyses as appropriate when EPA develops those subsequent rulemakings. 
Regarding the HONEST Act, see response to Comment 1370-4426.
Comment [4829-2022]: Commenter (4840) asserts that under the provisions of the APA, the EPA administrator does not have the authority to refuse to consider any comment submitted to the agency. If she or he thinks it is not valid, inaccurate, or inapplicable, she or he must explain why. Under the APA, submissions, including scientific studies, cannot arbitrarily or capriciously be discarded because the underlying data are not provided. When I was OSHA Administrator, we wanted to protect the integrity of the science used in setting regulations, so we explored asking for conflict-of-interest disclosures, similar to those requested by every leading scientific journal. Our legal experts determined that we could request this disclosure, but we could not reject submissions that failed to include them. This is a comparable situation  -  rejecting submitted studies because the underlying data are not available is prohibited under the APA.
Response: As explained in the final rule preamble at III.E, the final rule adopts a weighting approach to considering data availability and not the categorical exclusion presented as an option in the proposal and supplemental proposal. 
This procedural rule does not interpret or apply provisions of the statutes that it administers. EPA has explained that this procedural rule has limitations on its scope in terms of conflicts with other substantive requirements and that, in situations where this rule applies, EPA will give greater consideration to those studies where the underlying data and models are available for independent validation. If implementing this procedural rule will result in a conflict with an environmental statute that EPA administers, then this procedural rule will yield.
Comment [4838-4075]: Commenter (4879) contends that first, the proposal is too vague as written to provide the public with a meaningful opportunity to comment.
The Proposed Rule lacks credible specificity and is overly vague in its terms and scope. Under the APA, EPA is required to articulate the specifics of its proposed rulemakings in a manner that provides a valid opportunity for public comment.
In this proposal, however, the commenter states that the EPA solicits comment across a long list of topic areas but fails to provide the Agency's own "sufficient detail and rationale" [APA § 553(b )(3)] on the solicited comment areas. The commenter asserts that they are left in the position of speculating on EPA's views and on those of other commenters that would presumably shape EPA's final rule. It is well settled law that this approach fails to provide adequate notice for informed public comment.
Response: See response to Comment 2157-149.
Comment [5011-2648]: Commenter (5022) contends that the proposal will not meet its purported scientific goals and will instead undermine the scientific basis for decision-making by the agency. Since its inception, EPA has developed rules with demonstrable efficacy in protecting the public by relying in large part upon the kinds of data that EPA would now preclude from consideration. Some of EPA's greatest public health accomplishments, such as eliminating lead in gasoline and classifying secondhand smoke as a cause of cancer (which led to banning smoking from indoor public places), were based on the kinds of data that would be discarded under the proposal. Such data are widely used in rulemaking proceedings by other U.S. government agencies and around the world. 
Response: See response to comment 4577-332.
Comment [5153-4078]: Commenter (5164) provides that in an April 23, 2018 letter submitted by 985 scientists and technical experts, the scientists and technical experts say:
      As scientists and technical experts, we urge you to cease any plans to restrict the types of science that the EPA can use in regulatory decision-making. EPA can only adequately protect our air and water and keep us safe from harmful chemicals if it takes full advantage of the wealth of scientific research that is available to the agency.
The commenter notes that an electronic copy of the letter is attached to their letter and that the letter can also be found at: https://www.eenews.net/assets/2018/04/27/document_gw_01.pdf
By itself, the commenter states that the attached letter constitutes legally sufficient evidence to compel the conclusion that the proposed rule must be rejected as bad for the environment. In addition, the letter provides more than sufficient evidence that adoption of the proposed rule without careful and public consideration of the letter constitutes arbitrary and capricious action violating the APA.
Response: See response to Comment 4829-2022.
Comment [5985-4089]: Commenter (5995) states that both individually and as editors, the editors of InternationalMosaic.com object to the proposal to limit the use of scientific evidence in rulemaking proceedings. The proposal has generated a maelstrom of critical comments. See, e.g., https://www.eenews.net/assets/2018/04/27/document_gw_01.pdf (letter of 985 scientists and experts); Indeed, on August 2, 2018, at 11:00 a.m., PST, the www.regulations.gov site reported that 219,818 public comments had been logged in. https://www.regulations.gov/docket?D=EPA-HQ-OA-2018-0259
The commenter adds that, as explained in their previous comments, there is a wealth of scientific evidence that compels the conclusion that the proposed rule should be rejected as bad for the environment. In addition, the commenter claims that the adoption of the proposed rule would constitute arbitrary and capricious action violating the APA.
Response: See response to Comment 4829-2022.
Comment [6099-568]: Commenter (6109) states that the proposed rule is likely to run afoul of the APA. The commenter provides that the APA requires agencies to consider and respond to all information presented to it pursuant to a rule making. In an April 24, 2018 letter to Administrator Pruitt, members of the U.S. Senate Environment and Public Works Committee write, "Were a comment that contained scientific information that the proposed new policy would exclude from consideration to be submitted as part of a rulemaking, the APA would require that you consider it, setting up a direct conflict between the APA and the proposed new policy."
Response: See response to Comment 4829-2022.
Comment [6106-3183]: Commenter (6116) notes that while EPA states that the new rule would be applied prospectively, it notes that the rule could impact prior records for prior, older rules that are subject to recurring updates and reviews, such as the NAAQS. The new rule does not point to any basis in law or fact to question the validity and rationality of prior EPA rulemakings, such as NAAQS, and thus, it would be improper and contrary to the APA for EPA to reopen prior regulations for review based on the specious, unsupported claim that there were fundamental, substantive issues due to alleged lack of transparency in past scientific studies. Without specific evidence, particular findings, and robust legal analysis regarding individualized federal agency decisions, it is wholly unorthodox and legally suspect for EPA to issue a new regulation based on blanket assertions of a generalized, non-specific nature.
Response: The rule does not apply retrospectively to EPA regulations.
Comment [6112-1641]: Commenter (6137) states that not only is EPA's Proposed Rule unlawful both substantively and procedurally, but it is also impermissibly arbitrary. Indeed, according to the commenter, the Proposed Rule is the epitome of arbitrary rulemaking: in crafting the rule, EPA relie[s] on factors which Congress has not intended it to consider, entirely fail[s] to consider an important aspect of the problem, offer[s] an explanation for its decision that runs counter to the evidence before the agency, or is so implausible that it could not be ascribed to a difference in view or the product of agency expertise.
State Farm, 463 U.S. at 43 (defining "arbitrary and capricious" agency action). EPA wholly fails to "examine the relevant data and articulate a satisfactory explanation for its action including a `rational connection between the facts found and the choice made.'" Id. (quoting Burlington Truck Lines v. United States, 371 U.S. 156, 168 (1962)). To the contrary, as described below, EPA has mischaracterized its prior policy, relied on authorities that undermine its reasoning and conclusions, failed to adequately account for the Proposed Rule's costs and benefits, granted itself unfettered discretion to decide whether and how the Rule will apply, and created an unjustified inequity between what science can be used to support a decision not to regulate dangerous chemicals as compared to what can be used to support a decision to regulate. For all of these reasons, the Proposed Rule cannot stand. 
Response: The differential is based on whether a regulatory action as significant under Executive Order. 12866. The designation of a rule as a significant regulatory action has been a long-standing practice by the Federal Government to identify those regulations that have the potential to have the greatest impact on the American public.
Comment [6112-2497]: Commenter (6137) states that under the terms of the Proposed Rule, EPA must ensure that underlying data is publicly available only "[w]hen promulgating significant regulatory actions." 83 Fed. Reg. at 18,773 (proposed to be codified as 40 C.F.R. § 30.5). But as defined in the Proposed Rule, a significant regulatory action refers only to the promulgation of a new rule or regulation. Accordingly, as written, the Proposed Rule arguably allows EPA to utilize non-public studies or studies that rely on non-public data to justify a decision not to regulate at all. This creates a lopsided and inequitable playing field whereby EPA can rely upon studies that it prohibits others to use when it decides not to issue a new regulation.
An example highlights the dangerousness of this inequity. Under TSCA, many provisions involve go/no-go decisions about whether to promulgate regulations at all. Thus, upon consideration of whether to regulate a chemical, EPA could arguably utilize non-public studies or studies that rely on non-public data to justify a decision not to regulate. For example, EPA could use non-public data to justify a finding that a chemical does not pose an unreasonable risk to health or the environment under Section 2605 and thus does not require regulation. Pursuant to the definitions in the Proposed Rule, such a decision does not qualify as a significant regulatory action, because a finding of no unreasonable risk does not trigger any new rule or regulation. By contrast, EPA would not be able to rely on non-public studies or studies that rely on non-public data to justify a decision to regulate a chemical. 
The Proposed Rule thus creates a regulatory regime in which EPA can consider science based on non-public data if it shows that a chemical is not harmful (or minimizes the harmful effects of a chemical) and supports not regulating the chemical, but cannot consider science that shows that a chemical has harmful effects if it is similarly based on non-public data but supports a decision to regulate the chemical. EPA has not, and indeed cannot, provide any explanation or basis for this differential treatment. For this reason, too, the Proposed Rule is arbitrary and capricious and cannot stand. 
Equally problematic, the Proposed Rule could lead to absurd results. For example, when determining whether a chemical poses an unreasonable risk of harm under TSCA, EPA has to consider all relevant scientific studies. If EPA finds that a chemical poses an unreasonable risk and that it must issue a regulation on that basis, the Proposed Rule mandates exclusion of studies based on non-public data from consideration, even if those same studies provide the basis for the unreasonable risk finding. Absent these particular studies, the remaining research may show no unreasonable risk of harm, obviating the need for regulation, and in turn, eliminating the Proposed Rule's requirement that EPA not consider the excluded studies. Instead, when determining no unreasonable risk of harm, EPA must consider the excluded studies because such a finding does not result in a significant regulatory action and thus the Proposed Rule's ban on non-public data would not apply. But because the excluded study demonstrates that the chemical is unreasonably risky, EPA could not go forward with an unreasonable-risk determination. This could create an endless cycle of review and re-review with certain studies included, then excluded, and then included again ad infinitum. In such situations, application of the Proposed Rule creates a catch-22 in which EPA cannot make any determination that complies with both the APA's requirement of reasoned decision-making and the requirements of the Proposed Rule. 
The commenter contends that these flaws would plague any decision made under any provision of the statute in which EPA is required to choose between issuing or not issuing a regulation and, therefore, renders the Proposed Rule arbitrary and capricious. See e.g. 15 U.S.C. § 2603(a) (EPA must require testing if it finds that a chemical "may present an unreasonable risk")
Response: See response to Comment 4829-2022.
Comment [6112-2867]: Commenter (6137) states that the EPA's Proposed Rule requires the Agency to "give explicit consideration to high quality studies that explore . . . various threshold models across the dose or exposure range[.]" 83 Fed. Reg. at 18,774 (proposed 40 CFR § 30.6). A "threshold" is a dose or exposure "below which effects do not occur or are extremely unlikely." A "threshold model"  -  a type of "non-linear" model -  is a dose-response model in which there is no response below the threshold dose or exposure. According to the commenter, the EPA's proposed prioritization of threshold models is arbitrary, for at least three reasons.
First, EPA provides no evidence to justify the proposed requirement that it consider threshold models in dose-response assessments. EPA baldly asserts that "there is growing empirical evidence of non-linearity [i.e., a threshold] in the concentration-response function for specific pollutants and health effects[,]" but provides not a single example or citation for this unsubstantiated claim. Id. at 18,770. By contrast, EPA has found strong empirical evidence of no-threshold concentration-response functions for lead and reduced IQ, and particulate matter and increased mortality, following extensive reviews of the relevant literature to inform the Agency's national ambient air quality standards. EPA's failure to provide any support for its claim is telling.
Second, the proposed requirement to consider threshold models ignores essential science, namely the approach to dose-response assessment recommended by the NAS in Science and Decisions: Advancing Risk Assessment. Historically, researchers applied no-threshold dose-response models to carcinogens and threshold models to non-carcinogens. But in Science and Decisions, NAS determined that, due to the variability in susceptibility and exposures to other chemicals (background exposures) within populations, the effects of non-carcinogens may lack thresholds in populations even when these effects have thresholds in certain individuals who are less susceptible and/or have lower background exposures. In other words, thresholds vary by individual; for some, effects have high thresholds, while for others, effects have practically no threshold due to increased susceptibility or background exposures, and this latter group may develop disease when a population is exposed, even at very low levels. Based on these findings, NAS concluded that no-threshold models should be applied to both carcinogens and non-carcinogens unless reliable data affirmatively support a threshold model based on detailed assessments of mode of action (how a chemical causes disease), susceptibility, and background exposures. EPA's Proposed Rule entirely ignores Science and Decisions, its recommended approach to dose-response assessment, and the need for a detailed assessment before concluding that a threshold model is appropriate.
The commenter states that "explicit consideration" of "studies that explore . . . various threshold models" required under the Proposed Rule does not qualify as a "detailed assessment" of the type recommended by NAS. While NAS recommends consideration of the available data on mode of action, susceptibility, and background exposure before deciding whether dose-response models are appropriate, the Proposed Rule requires consideration of threshold models regardless of this data. EPA provides no justification for ignoring NAS' researched approach to dose-response data.
Third, EPA's proposed requirement also disregards the Agency's own Guidelines for Carcinogen Risk Assessment, which expressly state that, in cancer risk assessments, no threshold should be assumed unless the mode of action is known and the chemical does not cause cancer by inducing DNA mutations that initiate tumor development. Like Science and Decisions, the Cancer Guidelines state that it is necessary to assess mode of action to determine whether a threshold or no-threshold model is appropriate. EPA has historically followed this approach to ensure that its cancer risk assessments are adequately health protective. The Proposed Rule breaks from this prior practice and instead disregards science that the Cancer Guidelines say should be considered. EPA's reversal of longstanding policy is unjustified and arbitrary and should not be permitted.
Response: This rule is not being finalized as proposed. This rule does not.apply to dose-response models.
Comment [6112-3062]: Commenter (6137) argues that the Proposed Rule is arbitrary and capricious because its standardless provisions give EPA unfettered discretion in deciding whether and how the Rule applies. It has long been held that regulations must contain "narrow, objective, and definite standards to guide the [decision-making] authority,' . . . thereby to guard against the danger of arbitrary action." United States v. Abney, 534 F.2d 984, 986 (D.C. Cir. 1976) (quoting Shuttlesworth v. City of Birmingham, Ala., 394 U.S. 147, 149 (1969)). "When administrators provide a framework for principled decision-making" by "articulat[ing] the standards and principles that govern their discretionary decisions in as much detail as possible," "the result will be to . . . enhanc[e] the integrity of the administrative process." Envtl. Def. Fund, Inc. v. Ruckelshaus, 439 F.2d 584, 598 (D.C. Cir. 1971). But where a regulation is "wholly silent as to what factors the agency is to consider in granting exceptions . . . [a]gency discretion is unfettered," and the regulation is "arbitrary, capricious and contrary to law." Nat. Res. Def. Council, Inc. v. EPA, 863 F.2d 1420, 1432 (9th Cir. 1988).
The commenter states that the EPA's Proposed Rule lacks the requisite standards and principles that are the hallmark of lawful agency decision-making. The vague definitions proposed in 40 C.F.R. § 30.2 invite limitless agency discretion to decide when the Rule's requirements apply. For example, the Proposed Rule defines "regulatory science" as "scientific information . . . that provide the basis for EPA final significant regulatory decisions." 83 Fed. Reg. at 18,773. But this definition lacks any discernible meaning. There is no standard or definition for determining when scientific information does or does not "provide the basis" for a regulatory decision, nor are there any standards or definitions for what subset of "regulatory decisions" should be deemed "significant." Likewise, the Proposed Rule defines "pivotal regulatory science" as studies that "drive the requirements and/or quantitative analysis" of EPA's action, id., but this too lacks any understandable meaning. There is no standard governing what science does or does not "drive" the regulatory action. The lack of regulatory standards here means that EPA staff will impermissibly determine applicability of the rule "based upon their own unwritten personal standards." White v. Roughton, 530 F.2d 750, 754 (7th Cir. 1976).
In contrast to the definitions of proposed section 30.2  -  which lack any meaning or standards at all  -  proposed section 30.9 provides a veritable grab bag of reasons the Administrator may use in his discretion to decide when not to apply the Rule. It allows the Administrator to grant a case-by-case exemption to the Rule's requirements if the Administrator determines it is not "feasible" to ensure that data may be made publicly available "in a manner sufficient for independent validation, in a fashion that is consistent with law, protects privacy, confidentiality, confidential business information, and is sensitive to national and homeland security." 83 Fed. Reg. at 18,774. It also authorizes the Administrator to exempt application of the Rule if it is not "feasible" to conduct peer review for the multiple reasons outlined in OMB guidance. Id. Here, the many vague and discretionary "exception[s] . . . threaten[] to swallow the rule," Nat. Res. Def. Council, Inc., 863 F.2d at 1432, empowering the Administrator to pick and choose the preferred scientific studies he wants considered or excluded from consideration in any given rulemaking procedure with no clear standards or principles to keep this discretion in check. 
Proposed sections 30.2 and 30.9 thus give EPA unfettered discretion to be a case-by-case arbiter of scientific information, without notice-and-comment, any oversight, or any stated standards or principles limiting this discretion. These provisions allow for arbitrary application of the Rule, rendering the Proposed Rule arbitrary and thus unlawful.
Response: EPA disagrees with the commenter's contention that there are not standards or definitions for what subset of "regulatory decisions" should be deemed "significant." EPA has defined significant regulatory actions as those determined to be "significant regulatory actions" by the OMB pursuant to Executive Order 12866. That determination is made for every rule EPA proposes and finalizes. The criteria for the determination are listed in the Executive Order.
As to "drive," as EPA states in the preamble in Section III.E, those studies that are integral to characterizing dose-response relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information
EPA notes that the Administrator exemption is now contained in section 30.7 of the final rule. In granting an exemption to the final rule's procedural requirements, the Administrator must apply specifically enumerated standards and provide a justification for the Administrator's decision. The final rule outlines the standards to be applied and imposes a process for documenting an exemption decision precisely to ensure that section 30.7 will not be applied in an arbitrary or capricious manner. 
Comment [6112-4452]: Commenter (6137) states that in the preamble to the Proposed Rule, EPA states  -  with no evidence or justification  -  that "the benefits of this proposed rule justify the costs." 83 Fed. Reg. at 18,772. This conclusory statement is the very definition of arbitrary and capricious. State Farm, 463 U.S. at 42-43. EPA provides no evidence of what it considered to reach this statement, much less why its conclusion is rational. Indeed, EPA provides no information at all about what the "benefits" or "costs" of the Proposed Rule are, or what it evaluated to reach its conclusion. There is simply nothing in the record to support EPA's conclusory statement.
Despite the lack of evidence in the regulatory docket, as these and other comments make clear, the costs of this Proposed Rule are far-reaching and substantial, while EPA has failed to identify any alleged benefits at all, other than vague and unsupported references to improved transparency. EPA's Proposal to undermine, exclude, and ignore important and relevant health science in rulemaking proceedings will have serious implications on public health, privacy, the environment, and the judicial review process, and thus the costs will be significant. EPA cannot ignore these harms in its cost-benefit analysis, even if the costs are not readily quantifiable. See, e.g., Michigan v. EPA, 135 S. Ct. 2699, 2707 (2015) (costs include "harms that regulation might do to human health or the environment"); Executive Order 12,866, 58 Fed. Reg. 51,735, 51,735 (Oct. 4, 1993) (it is "essential to consider" the "qualitative measures of costs and benefits that are difficult to quantify"); see also Food Labeling: Nutrition Labeling of Standard Menu Items in restaurants and Similar Retail Food Establishments; Extension of Compliance Date; Request for Comments, 82 Fed. Reg. 20,825, 20,828 (May 4, 2017) (acknowledging that delaying a nutrition labeling requirement would lead to millions of dollars in lost health benefits). Yet the commenter states that there is no evidence that EPA evaluated any of these adverse impacts, quantitatively or qualitatively. Moreover, EPA itself estimated that the economic cost of this type of restriction on science would cost "considerably more" than $250 million.
Response: This rule exclusively governs internal EPA procedures and does not regulate any entity outside of the agency. The final rule preamble discusses potential costs and benefits to EPA at Section III.J.
Comment [6116-1119]: Commenter (6141) urges EPA to adopt clear, even-handed, and effective rules that are consistent with the objectives and principles enunciated in the APA as well as well-established federal administrative policies for good government. Notably, APA establishes rules for how federal agencies may develop and issue regulations. These rules include explicit requirements for all federal agencies to ensure that the public has meaningful opportunity to comment on proposed rules or other binding regulatory obligations. In effect, the APA rules direct EPA to disclose to the public the legal and technical bases for its binding regulatory actions and to rationally execute and adequately explain EPA's rationale for those final actions.
Response: See response to Comment 2157-149.
Comment [6119-1361]: Commenter (6144) states that, while EPA does not cite the APA as a source of its authority, this proposal would be violating the law because the agency would be arbitrarily and capriciously selecting which information to use in decision-making. Furthermore, under the APA, agencies are required to consider and respond to all information presented in the docket. If EPA chooses to ignore a portion of this evidence because it cannot access or publicize the dose response data, models, and codes, such action would be on its face arbitrary and capricious.
Response: See responses to Comments 2157-149 and 4829-2022.
Comment [6152-3754]: Commenter (6122) notes that the response from mentioned journals correctly pointed out that the proposed rule limits the science that could be used in decision-making and that peer reviewed science that has already been scrutinized by the scientific community should be used in making life or death decisions on environmental regulations: "it does not strengthen policies based on scientific evidence to limit the scientific evidence that can inform them: rather, it is paramount that the full suite of relevant science vetted through peer review, which includes more rigorous features, inform the landscape of decision making."
The commenters state that ignoring peer-reviewed studies because they are unable to publish data or because it is not reproducible is arbitrary and capricious. Many public health studies  --  including those focusing on acute environmental impacts  --  are not reproducible because reproducing them would require intentional, unethical exposure of toxins on human beings, other living beings, or the environment. This is not a limitation, nor does it discount the importance and reliability of those studies. By deliberate design, the proposed rule would have EPA ignore some of the most rigorous, long-term, peer-reviewed studies conducted by leading independent researchers. For example, a recent study found that for children younger than age two even extremely short exposure to soot particles, like those created by burning fossil fuels such as coal oil and natural gas, increases the kinds of respiratory infections that are a leading cause of sickness and death. This is the kind of study the EPA would ignore because undisclosed data would include the identities and health histories of the nearly 150,000 people in the study. Furthermore, HIPAA correctly makes it illegal for the researchers to release this kind of personal health information.
Response: See response to Comment 4829-2022.
Comment [6155-645]: Commenter (6125) states that EPA programs do not differ among each other in how they treat the desirability of making raw data available in the consideration of relevant science and research. If some EPA programs need the changes that EPA has proposed, then they all do. Indeed, much of the justification in EPA's preamble reads as though the proposal would in fact apply to all EPA regulatory decisions. 
The specific discussion from the proposal begins:
      The best available science must serve as the foundation of EPA's regulatory actions. Enhancing the transparency and validity of the scientific information relied upon by EPA strengthens the integrity of EPA's regulatory actions and its obligation to ensure the Agency is not arbitrary in its conclusions. By better informing the public, the Agency in (sic) enhancing the public's ability to understand and meaningfully participate in the regulatory process. In applying the best available science to its regulatory decision-making, EPA must comply with federal transparency and data integrity laws and must also ensure that its decision-making is marked by independence, objectivity, transparency, clarity, and reproducibility. 83 Fed. Reg. at 18769 [footnotes omitted]
One would think from this language that the proposal applies to all EPA regulatory activities under all of the statutes listed in the proposal. And indeed, to the extent that EPA's purported concerns have validity, the proposal should so apply. But it does not. In fact, EPA states that it applies only to "major rules" and, by a serendipitous and unexplained coincidence, does not apply to many types of decisions where industry is seeking a benefit from the government. Specifically, it does not apply to air or water permits for major industrial facilities, to standards for hazardous waste cleanups, to registration or reregistration of pesticides, or the setting of tolerances for pesticide residues on food, or to the approval for market of new chemicals. Such differential application is itself arbitrary. See American Trucking Ass'ns v. EPA, 175 F. 3d 1027, 1052-53 (agency arbitrarily applied different criteria for considering certain studies). Any defensible EPA proposal would have to explain why the need for transparency is so compelling that it could support a drastic change in the decision-making standards for, "major regulatory actions," and yet so weak that it does not apply at all to a host of other EPA decisions, many of which benefit industry directly and that collectively are probably far more important for protection of human health and the environment.
Response: This rule applies to significant regulatory actions, not simply to the subset of significant regulatory actions that are major rules. EPA plans to issue statute-specific transparency regulations or programmatic regulations. See the final rule preamble at Section III.A. The designation of a rule as a significant regulatory action has been a long-standing practice by the Federal Government to identify those regulations that have the potential to have the greatest impact on the American public.
Comment [6155-648]: Commenter (6125) states that the proposal on its face does not apply to other federal agencies or to states and tribes, (or for that matter, even to EPA outside of major rulemaking). But it does not acknowledge that this would mean two different federal definitions of science or address the potential confusion this would create. 
In some cases, sister federal agencies have a formal role in EPA decision-making. Several provisions of the CWA require that EPA act "after consultation with appropriate Federal and State agencies and other interested persons." E.g. CWA 304(a), 33 USC 1314(a) . See also CWA 104 (c) (providing for cooperation with HHS on research on the harmful effects of pollutants on health or welfare); SDWA 1412(d) (in proposing and promulgating regulations, EPA "shall consult with the Secretary"); CERCLA 104(i)(1),(2),(3),(10) (creating ATSDR with representatives from many federal agencies, including EPA reporting to the Surgeon General, to examine, summarize, and interpret "available toxicological information and epidemiologic evaluations" of " hazardous substances"). Because the proposal does not acknowledge the existence of any such provisions, it does not explain how any such consultation will be affected. If EPA receives comments from agencies during rulemaking based on the best available science, would EPA examine the data underlying the comments to decide whether to disregard those comments? Would disregarding them result in a decision that is arbitrary for disregarding relevant information in the record? Or will it consider them, effectively overriding the terms of this proposal when it comes to science from other agencies? EPA has not attempted to explain how the different federal definitions of science would work, or even identified it as a source of potential problems for EPA or other agencies. According to the commenter, this is another instance of inadequate notice under the APA.
Response: The differential is due to a determination under Executive Order 12866. The designation of a rule as a significant regulatory action has been a long-standing practice by the Federal Government to identify those regulations that have the potential to have the greatest impact on the American public.
Regarding compliance with the APA, see response to Comment 2157-149. As discussed in the final rule preamble at Section III.A, EPA plans to issue statute-specific transparency regulations or programmatic regulations. EPA will address any statutory conflicts and potential implementation issues as appropriate when EPA develops these subsequent rulemakings. 
Comment [6155-649]: Commenter (6125) states that the EPA's proposal fails all of the general administrative law requirements to provide adequate notice and opportunity for informed public comment. Indeed, this proposal ostensibly to support "transparency" in science is shrouded in so many vague generalities and so few specifics that any reader going beyond the introductory paragraph will learn next to nothing about the rule. As explained above, the proposal sends contradictory signals even about its substance--whether it is mandatory or hortatory, and whether a study that can be replicated but where the raw data is not available can be used. Compare the preamble summary (EPA "should" ensure that the data underlying its rulemaking is publicly available) and 30.4 (EPA "shall" identify the studies it is relying on and "should" make all such studies available to the public to the extent practicable) with 30.1 (rule "directs" EPA to ensure public availability) and 30.8 (authorizing exemptions). Compare 30.5 (information in a study is publicly available where it includes enough information to enable the public to "replicate" its findings) with Preamble note 3 (rule would "preclude" use of two studies that have been replicated). Assuming that such an incoherent, ambiguous, and self-contradictory proposal would even be possible to implement, it falls pathetically short of stating with the required detail and specificity the substance and basis of the agency action. 
EPA proposes to adopt rules restricting its use of science under at least eight statutes. Yet, the entire support for that radical step is set forth in a proposal of 21 paragraphs (and 24 footnotes), 8 of the paragraphs of which are filled with thirty questions posed to commenters rather than a clear explanation of the purpose, legality, or implications of the proposal. The requested comments cover a wide array of topics, including such basic questions as how the proposal "can best be implemented" and the legal bases for EPA's authority to issue the proposal. Providing answers to these questions is the responsibility of the agency, not the commenting public. 
Ordinarily, an agency should fully understand how it proposes to answer those questions in order both to understand its rule, and to meet its legal obligation "to disclose to the public the bases for agency rules and to rationally execute and adequately explain agency actions." That explanation should be included in the preamble, which can serve as a guide to interpreting the regulation. The commenter notes that the proposal does not do that and fails also to discuss how it will actually work under each of the eight statutes. It lists research and general rulemaking authorities under eight statutes but does not even identify any statutory requirements for science it might affect, much less the likely effects of those proposed changes to programs under the eight statutes. See Connecticut Light & Power Co. v. Nuclear Regulatory Commission, 673 F.2d 525, 530 (D.C.Cir.) (NPRM should provide an accurate picture of the agency's reasoning so that interested parties may comment meaningfully upon the agency's proposed rule). See Billington v. Underwood, 613 F.2d 91, 94 (5th Cir.1980) ("Such a statement must be sufficiently specific for it to enable an applicant to prepare rebuttal evidence to introduce at his hearing appearance.") In that context, in which the principles are similar to those in rulemaking, the purposes of adequate notice, including having the opportunity to prepare for the hearing and rebut the agency's allegations, require that a statement of reasons include a brief description of the event including when it occurred, who was involved, and what provision was violated. See id., see also Edgecomb, 824 F.Supp. at 314; Driver, 713 N.W.2d at 673."); Owner-Operator Independent Drivers Ass'n, Inc. v. Federal Motor Carrier Safety Admin., 494 F.3d 188, 209 (D.C Cir. 2007). It is certainly true that a notice can be "too general to be adequate." Small Refiner Lead Phase - Down Task Force, 705 F.2d at 549.
Response: See responses to Comments 2157-147 and 4829-2022.
Comment [6155-668]: Commenter (6125) contends that there is neither a legal basis nor a need for this rule. It would require that EPA violate explicit statutory provisions and would undermine environmental protection and critical health protections the public has come to rely on. It unlawfully shifts the basis for deciding what science to use in rulemaking away from the statutory goals of scientific reliability and environmental protection to so-called transparency, a term not used in the relevant EPA statutory provisions. The proposal is brief, evasive, superficial and ambiguous, and provides far too little information to meet the legal requirement that the public must be alerted to its substance and basis in order to understand sufficiently to provide public feedback and ultimately to create a workable legal framework the public must be fully alerted to its substance and basis. In other words, according to the commenter, the proposal is unintelligible, unlawful, and unworkable. The commenter requests that EPA withdraw it.
Response: See responses to Comments 2157-147 and 4829-2022.
Comment [6155-958]: Commenter (6125) states that if commenters on a future proposed rule submit studies barred by this per se rule, will EPA refuse to consider them? Two other questions: What if the studies are submitted by a federal agency, either consulting or commenting on a rule, or a state, or a tribe? What would be the legal basis, under the relevant statute, for refusing to do so? If EPA will consider them at this stage of rule development, what is the argument for not considering them when framing EPA's proposal of that same rule? To provide adequate notice, an agency must of course disclose all underlying supporting data on which it relies. See e.g. Chamber of Commerce v. SEC, 443 F. 3d at 899; CAA Section 307 (d)(3) (A) and (B).
Here, EPA has included only an undifferentiated data dump of citations without identifying passages of particular relevance (e.g. footnotes. 9-12 [of comment letter]), or explaining references to policies of other federal agencies (e.g. 83 FR 18770). As explained above and in Appendix D [of the commenter's letter] most of these sources are either mischaracterized or provide no support for the proposal. But in any case, such undifferentiated references do not provide adequate notice of the agency's thinking or intentions. Jackson v. Des Moines Mun. Housing Agency, 2008 WL 10707693, at *4 (S.D.Iowa, 2008) states this clearly in the context of the necessity of giving individuals adequate information to prepare rebuttal evidence against the government ("This is incorrect, however, because even a citation that includes a statement of the general language of a regulation is not sufficient to provide adequate notice. `Agency notice must describe the range of alternatives being considered with reasonable specificity[;][o]therwise, interested parties will not know what to comment on 'Small Refiner Lead Phase - Down Task Force, 705 F.2d at 549" ). The commenter asserts that the proposal does not identify any alternatives it may be considering, much less discuss them.
Response: See responses to Comments 2157-147 and 4829-2022.
Comment [6155-2153]: Commenter (6125) states that instead of acknowledging and discussing the implications of having two definitions of science, one of them applicable only to major rulemaking, the proposal simply takes the major rule limitation as a given, offering no explanation to support it and then asks the public to comment on the scope of the rule, giving no indication of what approach EPA plans to take [or of whether the new definition of science should be applied more universally]. The rule effectively proposes two different definitions of science for purposes of agency decision-making, one for major rules and the other for everything else; in other words there would be two different EPA definitions of science for actions addressing the same subject, the risk from the conduct, under the same statute. According to the commenter, this is the very definition of arbitrary. 
Response: The differential is due to a determination under Executive Order 12866. The designation of a rule as a significant regulatory action has been a long-standing practice by the Federal Government to identify those regulations that have the potential to have the greatest impact on the American public.
Comment [6155-2485]: Commenter (6125) states that instead of acknowledging and discussing the implications of having two definitions of science, one of them applicable only to major rulemaking, the proposal simply takes the major rule limitation as a given, offering no explanation to support it and then asks the public to comment on the scope of the rule, giving no indication of what approach EPA plans to take [or of whether the new definition of science should be applied more universally]. The commenter suggests that the rule effectively proposes two different definitions of science for purposes of agency decision-making, one for major rules and the other for everything else; in other words there would be two different EPA definitions of science for actions addressing the same subject, the risk from the conduct, under the same statute. This is the very definition of arbitrary. 
Response: The differential is due to a determination under Executive Order 12866. The designation of a rule as a significant regulatory action has been a long-standing practice by the Federal Government to identify those regulations that have the potential to have the greatest impact on the American public. Further, this rule applies to significant regulatory actions, not only the subset of significant regulatory actions that are designated as major rules.
Comment [6155-2602]: Commenter (6125) suggests that since no reasonable argument can be made that the proposal is legally required, the only possible theory upon which the rule itself and its categorical nature might be justified is as an exercise of the agency's discretionary authority to implement particular statutory provisions.
The commenter provides that their comments explore the defects of EPA's proposal that are probably common to all the statutory provisions it would affect. The commenter states that they do this with the significant handicap of being forced to comment on a very spare, uninformative proposal rather than the more detailed proposal that the law requires, particularly when a proposal would make such far-reaching changes as this one envisions.
We begin by addressing the relevant legal principles.
EPA has completely failed to follow well-established principles establishing the legal need for reasoned agency decision-making generally, and in particular for any change to a prior position.
To comply with these requirements, an agency must, among other things:
 Explain why the agency is rejecting policy judgments or factual determinations underlying the prior rule or position.
 Provide a "reasoned explanation" for changing course;
 Demonstrate that the new policy is itself consistent with the governing statute;
 Ensure that the new policy is itself supported by the record, "based on consideration of the relevant factors," and supported with "rational connection[s] between the facts found and the choice made"; and
 Consider relevant alternatives reflected in the prior rule's record, in particular, alternatives most consistent with statutory goals and purposes, and explain why the agency is not adopting them in the new rule.
The commenter contends that the proposal does not comply with any of these requirements.
Response: See responses to Comment 2157-149 and 4829-2022.
Comment [6155-3040]: Commenter (6125) states that given the total lack of justification for EPA's affirmative proposal, it is of no legal significance that EPA proposes to grant itself the power to waive its per se bar. What EPA needs to justify are the cases in which the policy applies, not those cases where the exemption will be used to avoid it. Otherwise, studies might be subject to an illegal bar on use unless EPA saw fit to grant them an exemption that rested in its discretion. 
It is well established that existence of a waiver or exemption mechanism cannot be used to justify a provision otherwise beyond an agency's legal authority. Dimension Financial Corp. v. Board of Governors of Federal Reserve System, 744 F.2d 1402, 1410 (10th Cir. 1984) ("The possible exception to the initial impact of Regulation Y (Part 225.21(B)(4)) contains requirements with no objective standard and thus unbounded agency discretion. This is a device to meet objections to the new regulation and cannot cure the exercise of powers denied by Congress or not provided for by Congress. Public Utilities Comm. of Calif. v. United States, 355 U.S. 534; 78 S.Ct. 446, 2 L.Ed.2d 470; In re Surface Mining Regulation Litigation, 627 F.2d 1346 (D.C.Cir.)") 
EPA moreover has failed to explain or seek comment on how it will administer the exemption provision. The provision itself gives EPA discretion by providing that it "may" allow exemptions in any case where "it is not feasible to ensure that all dose response data and models underlying pivotal regulatory science is publicly available in a manner sufficient for independent validation, in a fashion that is consistent with law, protects privacy, confidentiality, confidential business information, and is sensitive to national and homeland security," or where peer review is not feasible. Proposed section 30.9, 83 FR at 18774. Of course, if it were feasible in either case, there would be no need for an exemption. The proposal provides no standards or criteria for picking and choosing among exemption requests. That lack of standards or criteria is particularly
troubling because it could lead to arbitrary decision-making in a politically charged climate.
EPA has asked for comments on the general subject of when exemptions should be available but has provided no suggestions of its own on that subject. With no standards or criteria in the exemption discussion, there is evidently nothing to prevent the exemptions from being granted and withheld arbitrarily. Highly relevant and reliable studies for which data cannot be shared could still be excluded, no matter how authoritative the science might be, and even if it has been subject to peer review, multiple replications, a third-party reanalysis, and been approved for use in regulatory decisions and analyses by EPA's external science advisors.
Response: See responses to Comments 4829-2022 and 6112-4452.
Comment [6155-4298]: Commenter (6125) writes that given the basic purpose of EPA's statute -- to authorize regulations to protect the public health and the environment -- one would expect EPA to consider those impacts before adopting a per se rule that could affect such decisions. At a minimum, EPA should identify the key studies that its proposal would bar that had been used to support regulations in the past and evaluate the impact of their loss on those regulations and thereby on human health and the environment. 
Indeed, reasoned decision making requires an agency to identify and address the impact of its proposed actions on its ability to perform its Congressionally mandated functions, State Farm, 463 U.S. at 43, because the likely impacts of the proposal are a critical issue in deciding whether to proceed with the rule. Nevertheless, the proposal outright fails to consider any impacts at all  --  literally none at all! Failure to consider potential environmental impacts in a proposal that fundamentally affects each of EPA's core statutory responsibilities is fatally arbitrary. See Sierra Club v. Costle, 657 F.2d 298, 326 (D.C. Cir. 1981) ("[W]e can think of no sensible interpretation of the statutory words `best technological system' which would not incorporate the amount of air pollution as a relevant factor.") See also N.L.R.B. v. Creative Food Design Ltd., 852 F.2d 1295, 1306 (D.C. Cir. 1988) (noting that agencies "may not ignore relevant factors" in making decisions); Small Refiner Lead Phase-Down Task Force v. U.S.E.P.A., 705 F.2d 506, 521 (DC. Cir. 1983) ("Arbitrary and capricious" review generally refers to the requirement that an agency "must consider all the relevant factors" and reach a "reasonable" conclusion; ​Sierra Club v. Costle,657 F.2d at 323."); Environmental Defense Fund, Inc. v. Costle, 657 F.2d 275, 283 (DC Cir. 1981) ("Thus, we must be assured that the agency action was "based on a consideration of the relevant factors," Citizens to Preserve Overton Park, Inc. v. Volpe, supra, 401 U.S. at 416, 91 S.Ct. at 823; and that "the agency has exercised a reasoned discretion, with reasons that do not deviate from or ignore the ascertainable legislative intent," Ethyl Corp. v. EPA, supra, 541 F.2d at 35-36, quoting Greater Boston Television Corp. v. F.C.C., 444 F.2d 841, 850 (1970), cert. denied, 403 U.S. 923, 91 S.Ct. 2229, 29 L.Ed.2d 701 (1971).)"
Moreover, as in CMA v. EPA, 217 F. 3d 861, 865-66 (D.C. Cir 2000), EPA's assertion of potential benefits without demonstrating their existence or providing any quantitative comparison of their value as compared to costs and disbenefits to public health is yet another clear indicator of arbitrariness and lack of support in the evidence before the agency. Id. (citing State Farm, 463 U.S. at 43).
The proposal provides no supporting analysis for the following implausible assertion:
      EPA believes the benefits of this proposed rule justify the costs. The benefits of EPA ensuring that dose response data and models underlying pivotal regulatory science are publicly available in a manner sufficient for independent validation are that it will improve the data and scientific quality of the Agency's actions and facilitate expanded data sharing and exploration of key data sets; this is consistent with the conclusions of the National Academies. 
      
      83 FR 18772. (emphasis added)
Not only is there no support provided for the assertion that the proposal will "​improve the data and scientific quality of the agency's actions," but for all the reasons discussed above, it is simply not true, or perhaps more aptly, transparently false. The remaining section of the regulatory "analysis" is a particularly egregious sleight of hand:
      The proposed rule directs EPA to make all reasonable efforts to explore methodologies, technologies, and institutional arrangements for making dose response models and data underlying pivotal regulatory science used in significant regulatory decisions available to the public in a manner sufficient for independent validation, consistent with law and protection of privacy, confidentiality, and national and homeland security. However, it does not compel the Agency to make that information available where it concludes after all such reasonable efforts that doing so in way that complies with the law and appropriate protections is not possible.
      (emphasis added) Id. 
While it is not flatly untrue that the proposal "does not compel" EPA to make the information available, it is worse than disingenuous not to mention also that unless the information is publicly available it cannot be used as a basis for the rulemaking decision unless EPA grants an exemption. See at 18770 n. 7.
EPA regulations have relied on assessing many thousands of health-related studies of pollutants done in the last five decades, including epidemiological, human clinical, and animal toxicology studies. The proposal would exclude many such studies solely because the underlying data are not available, often because of legal or ethical reasons. Even for controlled human and animal studies, where subject data might ethically be released, the underlying data may no longer be accessible years after publication. Without the information provided in excluded research studies it would no longer be possible to assess the full weight of the available scientific evidence  -  a key guiding principle for judging the scientific integrity of the decision-making process. Some of the most useful information regarding health effects comes from real world(epidemiological) and laboratory (clinical) studies of human subjects, in which detailed information regarding health, lifestyle, medical status, location, and more about participants can be collected.
In fact, as far as we can judge given the ambiguity of EPA's proposal, the per se rule could call into question and potentially invalidate a vast range of currently existing EPA regulations. The proposal says nothing about what studies it will exclude, aside from those identified but not mentioned by name in footnote 3. Appendix C [of the commenter's letter] provides some examples that we were able to pull together in the time available.
Even though limited by time and resources, our assessment shows the magnitude of the issue and the absolute need for EPA to address the potential damages to public health and the environment before any final action. To issue a valid rule, the commenter asserts that the EPA would have to analyze this issue on its own, including what studies the proposal will exclude, present the results for public comment, and explain why the benefits of the per se rule justified the public health sacrifice and regulatory confusion that it would cause. 
Response: See responses to Comments 4829-2022 and 6112-4452.
Comment [6155-4391]: Commenter (6125) states that its cost impacts both on the agency and on reviewers; among other deficiencies, the proposal unhelpfully and tautologically states that costs will be low because EPA will "implement" the rule "in a manner that minimizes costs" (proposed section 30.8). There is no explanation of what specific types of costs may be incurred, much less how they will be minimized. According to the commenter, this type of vague and unsupported pronouncement does not provide an adequate basis for public comment; see Chamber of Commerce v. SEC, 443 F. 3d 890, 904-05 (D.C. Cir. 2006) (unsupported statement that costs of action are expected to be minimal does not provide adequate notice of agency's final actions relating to cost estimates).
Response: See response to Comment 6112-4452.
Comment [6163-1114]: Commenter (6133) states that as EPA is aware, when an agency drafts a proposed rule pursuant to congressionally delegated authority, the exercise of that authority is governed by the informal rulemaking procedures outlined in the APA, 5 U.S.C. § 553.5. EPA is required to provide the public with adequate notice of a proposed rule, followed by a meaningful opportunity to comment on the rule's content. 5 U.S.C. § 553 (b)-(c).
The requirement under § 553 to provide the public with adequate notice of a proposed rule is generally achieved through the publication of a NPRM in the Federal Register, and the APA requires that the notice of proposed rulemaking include "(1) the time, place, and nature of public rulemaking proceedings; (2) reference to the legal authority under which the rule is proposed; and (3) either the terms or substance of the proposed rule or a description of the subjects and issues involved." 5 U.S.C. § 553(b)1-3. Generally speaking, the notice requirement of § 553 is satisfied when the agency "affords interested persons a reasonable and meaningful opportunity to participate in the rulemaking process." Forester v. Consumer Prod. Safety Comm'n, 559 F.2d 774, 787 (D.C. Cir. 1977). 
The Proposal fails to reference any other legal authority to support its adoption. The agency claims its Proposal is "consistent with" APA provisions to ensure public participation in the rulemaking process, 83 Fed. Reg. at 18,769/2, but this faint "consistent with" falls far short of any legal authority for the Proposal, or even any claim of such authority. The APA provides no authority for the Proposal and, tellingly, EPA does not and cannot identify any authority therein. Even were this "consistent with" claim an attempt by EPA to claim any legal authority for the Proposal, the throw-away statement fails to provide sufficient notice for the public to comment on the proposed rule or any asserted legal authority in the APA.
Finally, the Proposal's solicitation of comment -- "on whether additional or alternative sources of authority are appropriate bases for this proposed regulation" -- does not and cannot itself provide any justification for EPA finalizing a rule based on additional or alternative sources of legal authority. This fails to provide sufficient notice for the public to comment on the proposed rule or any other possible legal authorities. For all these reasons, the commenter asserts that the EPA lacks any basis to finalize a rule invoking any other legal authorities to support its adoption.
Response: See response to Comment 2157-149. EPA cites Reorganization Plan No. 3 of 1970, 84 Stat. 2086 (July 9, 1970), which created the EPA, as authority for this internal procedural rule. See Section I.C of the final rule preamble. As discussed in the final rule preamble at Section I.C and elsewhere in the response to comments document, EPA's Reorganization Plan conferred EPA with authority to promulgate regulations to govern its internal affairs. This authority is equivalent to the Federal housekeeping authority provided to Executive departments in 5 U.S.C. § 301. 
Comment [6163-1174]: Commenter (6133) states that the Proposal states that it is consistent "with the principles underlying the APA and programmatic statutes that EPA administers to disclose to the public the bases for agency rules and to rationally execute and adequately explain agency actions." 83 Fed. Reg. at 18,769. In a footnote to this sentence the Proposal states: 
      EPA has the authority to establish policies governing its reliance on science in the administration of its regulatory functions. Historically, EPA has not consistently observed the policies underlying this Proposal, and courts have at times upheld EPA's use nonpublic data in support of its regulatory actions. See Coalition of Battery Recyclers Ass'n v. EPA, 604 F.3d 613, 623 (D.C. Cir. 2010); American Trucking Ass'ns v. EPA, 283 F.3d 355, 372 (D.C. Cir. 2002). EPA is proposing to exercise its discretionary authority to establish a policy that would preclude it from using such data in future regulatory actions.
83 Fed. Reg. at 18,769 n.3.
EPA recognizes the cited cases contradict the proposed rule but attempts to waive them away and asserting it has discretionary authority to do the opposite of what the D.C. Circuit decided. EPA's consideration of peer reviewed scientific studies that do not have public data is the norm, required by the APA and the programmatic statutes that EPA administers. See sections II, IV, & V. The proposed departure from this norm to preclude the use of such data, which the Proposal makes explicit in this footnote, is not within EPA's discretion and would violate the programmatic statutes. As explained above, nothing the Proposal provides EPA with authority to do so. The Proposal's citations to two cases that contradict its proposed actions does not support the unexplained assertion of authority. 
The court in American Trucking stated:
      More generally, we agree with EPA that requiring agencies to obtain and publicize the data underlying all studies on which they rely "would be impractical and unnecessary." Particulate Matter NAAQS, 62 Fed. Reg. at 38,689. As EPA persuasively stated in denying Petitioners' original request for the information:
      If EPA and other governmental agencies could not rely on published studies without conducting an independent analysis of the enormous volume of raw data underlying them, then much plainly relevant scientific information would become unavailable to EPA for use in setting standards to protect public health and the environment.... [S]uch data are often the property of scientific investigators and are often not readily available because of ... proprietary interests ... or because of [confidentiality] arrangements [with study participants].
Am. Trucking Associations, Inc., 283 F.3d at 372.
In Coalition of Battery Recyclers, the D.C. Circuit cited American Trucking, explaining that the court had "rejected the notion that EPA had improperly failed to obtain and make public data underlying studies on which it had relied during a NAAQS rulemaking, holding that `[t]he Clean Air Act imposes no such obligation' and that `requiring agencies to obtain and publicize the data underlying all studies on which they rely would be impractical and unnecessary.'" 604 F.3d at 623 (citations omitted). The court noted "that raw data often is unavailable due to proprietary interests of a study's scientific investigators or confidentiality agreements with study participants." Id. The Proposal at least concedes that D.C. Circuit law does not support its actions. Yet, according to the commenter, EPA does not explain how the Proposal is consistent with the principles underlying the APA and programmatic statutes that EPA administers. To the extent EPA believes this to be true, it should withdraw the Proposal and explain its belief.
Response: See responses to Comments 2157-149 and 4829-2022. As discussed in the final rule preamble at III.A, EPA plans to issue statute-specific transparency regulations or programmatic regulations. EPA will address any statutory conflicts as appropriate when EPA develops these subsequent rulemakings.
Comment [6163-2726]: Commenter (6133) asserts that the Proposal unlawfully restricts EPA's consideration and use of "dose response data and models that underlie" what the Proposal calls "pivotal regulatory science." 83 Fed. Reg. at 18,770/2. The Proposal goes on to state:
      Pivotal regulatory science' is the studies, models, and analyses that drive the magnitude of the benefit-cost calculation, the level of a standard, or point-of-departure from which a reference value is calculated. In other words, they are critical to the calculation of a final regulatory standard or level, or to the quantified costs, benefits, risks and other impacts on which a final regulation is based.
Id. By restricting EPA's implementation of its federal organic statutes and the APA in this fashion, and by defining "pivotal regulatory science" in this manner, the Proposal violates federal laws. The Proposal does so by requiring EPA to implement federal laws based on the Proposal's criteria and conception of "pivotal regulatory science," rather than on the congressional criteria and requirements in federal statutes that contradict, disallow, or fail to include those criteria and concepts in the Proposal.
Response: See response to Comment 6163-1174.
Comment [6163-2871]: Commenter (6133) provides that in proposed § 30.6, EPA proposes to "evaluate the appropriateness of using default assumptions, including assumptions of a linear, no-threshold dose response, on a case-by-case basis." 83 Fed. Reg. at 18,774. In proposed § 30.9, the Proposal grants the Administrator the ability to "grant an exemption to this subpart on a case-by-case basis if he or she determines that compliance is impracticable" for a number of enumerated reasons. The commenter contends that both of these provisions inject additional arbitrariness into the rule, in that they ensure that the Proposal may be applied unevenly -- for certain rulemakings, the "rules" of the Proposal can be discarded or ignored where desired. This, in addition to and with other sections of the Proposal, underscores that it is arbitrary and capricious and must be withdrawn.
Response: EPA notes that it has finalized changes to proposed sections 30.6 and 30.9 (sections 30.5 and 30.7 in the final rule, respectively) in response to public comments raising similar issues. While sections 30.5 and 30.7 of the final rule allow EPA to make certain decisions on a "case-by-case basis," both provisions require EPA to document and explain the basis and rationale for its decisions in the associated significant regulatory decision or influential scientific information. In addition, as finalized, section 30.7 governing the Administrator's ability to grant exemptions outlines the specific circumstances under which the Administrator may use that mechanism.
Comment [6163-2934]: Commenter (6133) states that in the Proposal, EPA professes concern with transparency, clarity, and independence; using the best available information; making sure that information is replicable and verifiable, and ensuring the public is able to participate meaningfully in the regulatory process. The Proposal says this will help EPA carry out its mission in a manner the public can trust and understand:
      The proposed regulation provides that, for the science pivotal to its significant regulatory actions, EPA will ensure that the data and models underlying the science is publicly available in a manner sufficient for validation and analysis.
83 Fed. Reg. at 18,769/1.
      The best available science must serve as the foundation of EPA's regulatory actions. Enhancing the transparency and validity of the scientific information relied upon by EPA strengthens the integrity of EPA's regulatory actions and its obligation to ensure the Agency is not arbitrary in its conclusions. By better informing the public, the Agency in enhancing the public's ability to understand and meaningfully participate in the regulatory process. 
Id. at 18,769/2.
      When EPA develops significant regulations using public resources, including regulations for which the public is likely to bear the cost of compliance, EPA should ensure that the data and models underlying scientific studies that are pivotal to the regulatory action are available to the public. This proposed rule is designed to increase transparency in the preparation, identification, and use of science in policymaking.
Id. at 18,769/3.
      Regulatory determinations based on science should describe and document any assumptions and methods used and should address variability and uncertainty.
Id. at 18,770/2.
      Pivotal regulatory science' is the studies, models, and analyses that drive the magnitude of the benefit-cost calculation, the level of a standard, or point-of-departure from which a reference value is calculated. In other words, they are critical to the calculation of a final regulatory standard or level, or to the quantified costs, benefits, risks and other impacts on which a final regulation is based. 
      Id.
      This [P]roposal will help ensure that EPA is pursuing its mission of protecting public health and the environment in a manner that the public can trust and understand.
Id. at 18,769/1.
In this section of our comments, we make the following points opposing the Proposal and supporting its withdrawal:
 First, the Proposal as written sweeps broadly to capture -- and thereby to prohibit EPA from considering -- studies, models, and analyses that are integral to the functioning of EPA regulatory programs, implementation of statutes like the CAA, and protection of public health and the environmental. It is both destructive and unlawful for EPA to refuse or fail to consider these additional studies, models, and analyses. We discuss numerous examples below.
 Second, to the extent that the Proposal does capture one or more of the studies, models, or analyses below, the Proposal would require EPA to conduct independent peer review of these materials before considering or using them, or before continuing to make them available for public use and awareness. See 83 Fed. Reg. at 18,774 (proposed § 30.7). This is objectionable and absurd. It is also unlawful for the same reasons that the Proposal is unlawful, as detailed in these comments and others.
 Third, to the extent that EPA disagrees that one or more of these studies, models, and analyses are captured by the Proposal, continuing to consider these materials while prohibiting EPA from considering other materials would be arbitrary and capricious. This is because these studies, models and analyses have the same hallmarks as "pivotal regulatory science" that the Proposal would exclude, as discussed in greater detail below. We emphasize that we do not believe EPA should or that EPA may fail to consider these other studies, models, or data, for the reasons set forth in these comments. Rather, our point is that continuing to consider these materials demonstrates additionally that the Proposal is arbitrary, capricious, and an abuse of discretion.
The Proposal states that "[t]he provisions of this subpart apply to dose response data and models underlying pivotal regulatory science that are used to justify significant regulatory decisions regardless of the source of funding or identity of the party conducting the regulatory science." 83 Fed. Reg. at 18,773/3 (proposed § 30.3). Next, the Proposal defines "dose response data and models" to mean:
      the data and models used to characterize the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and the magnitude of a predicted health or environmental impact. Such functions typically underlie pivotal regulatory science that drives the size of benefit-cost calculations, the level of a standard, and/or the points of departure from which reference values (reference doses or reference concentrations) are calculated.
Id. at 18,773/2 (proposed § 30.2). Then, the Proposals defines "pivotal regulatory science" to mean "the specific scientific studies or analyses that drive the requirements and/or quantitative analysis of EPA final significant regulatory decisions." Id. Finally, the Proposal defines "regulatory science" to mean "scientific information, including assessments, models, criteria documents, and regulatory impact analyses, that provide the basis for EPA final significant regulatory decisions." Id.
The Proposal either covers on its face, or appears to cover, examples of studies, models, and analyses that are integral to the functioning of EPA regulatory programs, implementation of statutes like the CAA, and protection of public health and the environment. The commenter states that it would be harmful, unlawful, arbitrary, and capricious, and an abuse of EPA's discretion to include these materials within the sweep of the Proposal's prohibitions.
Alternatively, according to the commenter, if EPA disagrees that the following examples are covered by the Proposal, then continuing to consider these materials that have the same hallmarks as the prohibited materials, and that raise the same issues and concerns that cause EPA to prohibit their consideration, demonstrates that the Proposal is arbitrary and capricious, biased, and internally inconsistent and contradictory. Moreover, the commenter states that in this case, the Proposal would suffer from fatal failures to explain why EPA may consider these materials, while the Proposal would prohibit EPA from considering other materials.
Response: See response to Comment 4829-2022. EPA notes that the rule requires any peer review to be conducted in accordance with applicable OMB guidance. EPA notes that the final rule does not apply to models and applies to only the subset of data that consists of dose-response data.
Comment [6163-2935]: Commenter (6133) states that dose response data and models is defined as "the data and models used to characterize the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and the magnitude of a predicted health or environmental impact. Such functions typically underlie pivotal regulatory science that drives the size of benefit-cost calculations, the level of a standard and/or the points of departure from which reference values (reference doses or reference calculations) are calculated." (§ 30.2). Despite being an important phrase repeated through the Proposal and the proposed text, this compound definition is vague and arbitrary. It also is circular -- the very terms being defined are used in the definition. It's unclear what data EPA is referring to in this phrase and definition. Moreover, it's unclear what EPA means by "[s]uch functions typically under pivotal regulatory science . . ." And the problems with "pivotal regulatory science" have already been discussed [in their comment letter]. As explained in section XII [of their comment letter], the definition does not adequately describe what the proposal covers. This definition, according to the commenter, along with the rest of the Proposal, is arbitrary and capricious and must be withdrawn.
Response: EPA has clarified the distinction between dose-response data and data. See Sections III.B, III. C and III.E of the final rule preamble. 
Comment [6163-3025]: Commenter (6133) states that in proposed § 30.6, EPA proposes to "evaluate the appropriateness of using default assumptions, including assumptions of a linear, no-threshold dose response, on a case-by-case basis." 83 Fed. Reg. at 18,774. In proposed § 30.9, the Proposal grants the Administrator the ability to "grant an exemption to this subpart on a case-by-case basis if he or she determines that compliance is impracticable" for a number of enumerated reasons. The commenter contends that soth of these provisions inject additional arbitrariness into the rule, in that they ensure that the Proposal may be applied unevenly -- for certain rulemakings, the "rules" of the Proposal can be discarded or ignored where desired. This, in addition to and with other sections of the Proposal, underscores that it is arbitrary and capricious and must be withdrawn.
Response: See response to Comment 6163-2871 
Comment [6163-3346]: Commenter (6133) states that the Proposal at section 30.3 states that:
      Except where explicitly stated otherwise, the provisions of this subpart do not apply to any other type of agency action, including individual party adjudications, enforcement activities, or permit proceedings.
83 Fed. Reg. at 18,773. 
This provision most clearly highlights one of the arbitrary and capricious advantages that industry stakeholders enjoy under the Proposal: it exempts from its censoring coverage EPA activities where industry is the primary party likely to submit confidential information that EPA may consider and rely upon. This, notwithstanding that the submitted information is not "publicly available in a manner sufficient for independent validation," while still being highly relevant and even integral to EPA's legal responsibilities. Id. at 18,768.
Permitting activities are one key example. For permitting actions taken under the CAA, RCRA, CWA, etc., the Proposal arbitrarily and capriciously allows EPA to continue to rely on highly relevant regulatory science and other information supplied by industry that is not "publicly available in a manner sufficient for independent validation." Id. at 18,771 - 73. A company seeking a permit or permit revision may submit regulatory science, confidential business information or other non-confidential information that is not "publicly available in a manner sufficient for independent validation." Id. EPA could consider non-peer reviewed, nontransparent industry science or information to conclude that a non-transparent industry model demonstrates no adverse air quality impact on a neighboring national park or wilderness area. This, despite the inputs and assumptions behind the model being unavailable to the public. An applicant could assert that there are safe exposure levels for PM2.5 or lead, and therefore EPA need not require any mitigation measures at concentrations below NAAQS levels in attainment areas. Industry applicants could rely upon hidden CBI to project no emissions increases for purposes of NSR permitting under the so-called "demand growth" exclusion, notwithstanding the unavailability of information critical to industry's claim and EPA's acceptance of that claim. Considering this and other non-transparent information, EPA could conclude that permits or permit revisions may be granted in situations where they should not lawfully be granted, notwithstanding that the non-transparent, unavailable information is scientifically erroneous and even absurd.
A second example is public information submitted during enforcement proceedings. The Proposal arbitrarily and capriciously allows EPA to continue to rely on highly relevant regulatory science and other information supplied to the agency by industry during enforcement proceedings, even when that information is not "publicly available in a manner sufficient for independent validation." Id. at 18,771 - 73. Consider, for example, a company that receives a notice of violation from EPA and meets with the agency to make the case that EPA and the DOJ should not file a complaint. The company may submit regulatory science, CBI, or other non-confidential information that is not "publicly available in a manner sufficient for independent validation." Id. at 18,768. EPA could consider non-peer reviewed, non-transparent, erroneous industry science to conclude that formaldehyde or asbestos are not carcinogens, or that PM2.5 or lead have safe exposure levels, or that CO2 does not endanger public health or welfare. Considering this and other non-transparent information, EPA could conclude that prosecution is not warranted, or that the information represents mitigating factors for penalties or injunctive relief, notwithstanding that the non-transparent, unavailable information is scientifically erroneous and even absurd.
The third case is public information submitted during individual party adjudications. Id. at 18,771 - 73. The Proposal arbitrarily and capriciously allows EPA to continue to rely on highly relevant regulatory science and other information supplied to the agency by industry during individual party adjudications, even when that information is not "publicly available in a manner sufficient for independent validation." Id. at 18,768. Consider, for example, a company facing an EPA order or applicability determination that qualifies as an adjudication under the APA or one of the federal statutes that the agency administers.
The company may submit regulatory science, CBI or other non-confidential information that is not "publicly available in a manner sufficient for independent validation." EPA could consider non-peer reviewed, non-transparent industry science to conclude that formaldehyde or asbestos are not carcinogens, or that PM2.5 or lead have safe exposure levels, or that CO2 does not endanger public health or welfare. Considering this and other non-transparent information during the individual party adjudication, EPA could conclude that adoption of the order is not warranted, or that agency regulations should be interpreted in a way that does not apply to that company's actions. Indeed, EPA could conclude, after considering the non-transparent, unavailable information, that the regulations should not apply in ways that would affect an entire industrial sector favorably, while harming the public meant to be protected by those regulations. Under proposed section 30.3, EPA could consider the non-transparent, unavailable information to reach these objectionable outcomes, notwithstanding that the information is scientifically erroneous and even absurd. The Proposal nowhere explains why it is valid and consistent with EPA's statutory authorities and responsibilities to consider information that is not "publicly available in a manner sufficient for independent validation" under the situations allowed in proposed section 30.3 (individual party adjudications, enforcement activities, or permit proceedings), while prohibiting EPA consideration of that information in situations covered by the Proposal's prohibitions. Indeed, it is striking that the Proposal does not even attempt any such explanation or justification. Id. at 18,771 - 73. This is undoubtedly because there is no coherent, lawful justification or explanation that the agency could muster; it is unsurprising that the Proposal cannot overcome this.
Indeed, it is a hallmark of the Proposal's inherent arbitrariness and capriciousness that the Proposal prohibits EPA from considering the identical regulatory science, studies, and information in some regulatory situations, while allowing EPA to consider the identical regulatory science, studies, and information in other regulatory situations -- based merely upon the type of situation, rather than any differences in availability, replicability, verifiability, or validation concerning the information. Proposed section 30.3 prohibits EPA from considering information that is not "publicly available in a manner sufficient for independent validation" 
102 during so-called "significant regulatory decisions," while prohibiting EPA from considering that identical regulatory science, studies, or information during "any other type of agency action, including individual party adjudications, enforcement activities, or permit proceedings." 83 Fed. Reg. at 18,768, 18,771. The Proposal does not and cannot explain or justify this differential treatment, so the Proposal does not even try.
Finally, proposed section 30.3 is unlawfully vague, open-ended, and arbitrary due to the capacious and unlimited way that EPA has drafted the exclusion from the Proposal's prohibitions. Section 30.3 indicates that "the provisions of this subpart do not apply to any other type of agency action." This grants EPA capacious and effectively unlimited discretion and authority to decide what "any other type of agency action" is and is not, without providing the public or regulated entities any criteria, understanding or advance notice as to how EPA will exercise that discretion and authority. That is the essence of arbitrary and capricious agency action. Indeed, the Proposal is structured in such a way that EPA will be exercising that discretion and authority -- to decide what "any other type of agency action" does and does not cover -- in secret, with no public input and no public awareness, concerning the situations in which EPA will and will not consider non-transparent, unavailable information. The commenter contends that, in addition to this being perversely ironic, considering the "transparency" title of the Proposal, this fact renders the Proposal even more arbitrary and capricious and unlawful.
Response: By focusing on significant regulatory actions and influential scientific information, EPA is applying this rule to the actions that have the potential to have the greatest impact on the American public. 
Comment [6163-3479]: Commenter (6133) states that the term "pivotal regulatory science" is perhaps the most vague, unexplained, and internally inconsistent term used in the Proposal. The term has no statutory basis in any statute cited by EPA, or otherwise. Beyond having no statutory underpinning, the meaning of the phrase is neither self-evident nor adequately defined in the Proposal.
EPA's choice to modify "regulatory science" with the adjective "pivotal" does nothing to clarify the scope of scientific studies and information encompassed by the Proposal. "Pivotal regulatory science" is defined within the regulation as "the specific scientific studies or analyses that drive the requirements and/or quantitative analysis of EPA final significant regulatory decisions." 83 Fed. Reg. at 18,773/3 (proposed § 30.2). This definition is as unclear and unsupported as the term itself.
The use of the phrase "drive the requirements" within the C.F.R. definition is particularly incoherent. What does "drive the requirements" mean? The Proposal nowhere says. Does the definition apply only to scientific studies that were outcome determinative? Does it encompass any scientific study that was considered in making the requirements? What about studies that were useful but not determinative? Something else entirely? Can more than one study be "pivotal" to the regulatory decision, or does the term "drive the requirements" imply that only one study could be "pivotal" to a given decision? Furthermore, are most of the studies used by EPA considered to "drive the requirements" or is this term limited in some fashion, unrevealed to the public? Will EPA "know it when it sees it," making it up as the agency goes along? 
According to the commenter, it is arbitrarily, vague, and unexplained under the Proposal which science would be considered "pivotal," and under what conditions. Because the term was created out of thin air to serve EPA's purposes and has no statutory grounding or intuitive meaning, this ambiguity-ridden definition is woefully inadequate. It is also arbitrary and capricious and an abuse of EPA's discretion. EPA is well aware of the insufficiency of the definition, as is evident in the agency's solicitation of comments on the definitions of "pivotal regulatory science" and "dose response data and models" within the Proposal. See 83 Fed. Reg. at 18,771.
Notably, the proposed CFR definition also differs substantially from a definition of "pivotal regulatory science" appearing earlier in the Proposal, which defines the term as "the studies, models, and analyses that drive the magnitude of the benefit-cost calculation, the level of a standard, or point-of departure from which a reference value is calculated. In other words, they are critical to the calculation of a final regulatory standard or level, or to the quantified costs, benefits, or risks and other impacts on which a final regulation is based." 83 Fed. Reg. at 18,770.
Next, it bears repeating that EPA does not and cannot identify any statutory basis -- in federal environmental statutes, the APA or otherwise -- to apply the Proposal's approach "to the calculation of a final regulatory standard or level, or to the quantified costs, benefits, or risks and other impacts on which a final regulation is based." 83 Fed. Reg. at 18,770. EPA simply makes this up. 
EPA's separate explanation here suffers from additional defects, namely an internal inconsistency, incoherency and unbounded reach that do not accord with the proposed C.F.R. definition. EPA's preambular explanation says that "pivotal regulatory science" is "critical to the calculation of a final regulatory standard or level, or to the quantified costs, benefits, or risks and other impacts on which a final regulation is based." 83 Fed. Reg. at 18,770 (emphasis added) The Proposal nowhere explains what these "other impacts" are. Nor does the Proposal limit or bound these "other impacts," nor link them to the sentence's incoherent notion of what is "critical" and what is not. Moreover, the preambular gloss is inconsistent with the proposed C.F.R. definition. The former says "pivotal regulatory science" is critical to hopelessly vague "other impacts" on which a final rule is based. Id. The proposed C.F.R. definition, by contrast, says "pivotal regulatory science" "drive[s] the requirements and/or quantitative analysis of EPA final significant regulatory decisions." 83 Fed. Reg. at 18,773/3 (proposed § 30.2). The Proposal does not square the contradictions between science that drives a final rule's requirements and science that is "critical" to "other impacts" in a final rule.
Furthermore, the commenter states that the EPA not only fails to provide a passable definition for its invented term, "pivotal regulatory science," the agency fails to provide its rationale for limiting the scope of the rule to so-called "pivotal regulatory science." Within the unlawful, arbitrary, and capricious worldview reflected in the Proposal, why is the "public availability of science and data in a manner sufficient for independent validation" any less important or necessary or justified when the science is not "pivotal" or "critical" to a regulatory decision? Why should not all science, studies, data, and information considered by EPA meet the standards for transparency, verifiability, independent validation, and trustworthiness that are the abiding concerns of the Proposal? Why is it not arbitrary and capricious for EPA to continue to consider science and data that are unavailable and insufficient for independent validation in areas outside the reach of the Proposal? EPA offers no explanation for this disparate treatment; the agency's reasoning, such as it is, is entirely conclusory.
By way of explanation for the limitation, EPA only suggests that the imposed standards "are of paramount importance when the government relies on science to inform its significant regulatory decisions." 83 Fed. Reg. at 18,769. This explanation is hopelessly circular and ultimately incoherent. For starters, EPA does not explain why it believes this explanation to be true. Next, the Proposal just substitutes the word, `paramount,' for the word, `critical,' that it substitutes for the word, `pivotal.' (The Proposal's drafters evidently were just flipping through a thesaurus.) This failure to thoroughly explain both the term "pivotal regulatory science" in a way that meaningfully defines the scope of the regulation, and the rationale behind limiting the application only to pivotal (critical, paramount) science, makes it impossible for interested parties to comment fully and meaningfully on the Proposal. The commenter suggests that, should EPA intend to finalize this unlawful proposal, EPA first must withdraw the Proposal, then issue a supplemental proposal with the necessary definitions and explanations. Better yet, the commenter suggests that EPA abandon the illegal and harmful proposal altogether.
Response: Regarding compliance with the APA, see response to Comment 2157-149. EPA notes that the APA does not require a proposal to be withdrawn before a supplemental proposal is issued. Response to comments on pivotal science are addressed in the final rule preamble (See Sections III.C and III.E) and in Chapter 6 of this document.
Comment [6168-4103]: Commenter (6178) states that the Agencies' proposed action in the NPRM  -  adopting a new standard for all scientific research to be relied on by the EPA going forward  -  is itself a substantive agency action, triggering certain requirements of the APA. Specifically, the requirement of the APA that agency action not be `arbitrary and capricious' as interpreted by the Supreme Court of the United States in Motor Vehicle Manufacturers' Association v. State Farm Mutual Automobile Insurance Co.(`State Farm').
Under Section 706 of the APA, a reviewing court can set aside agency action if the action is found to be "arbitrary, capricious, an abuse of discretion, or otherwise not in accordance with law." The Supreme Court provided clear guidance as to what constitutes an arbitrary and capricious agency action in State Farm where the Court stated: "Normally, an agency rule would be arbitrary and capricious if the agency has... relied on factors which Congress has not intended it to consider...."
As stated above, a plain reading of the Congressional Acts cited by the EPA as support for its proposed Transparency Rule demonstrates that transparency, as contemplated by the NPRM, is not a factor on which Congress intended the EPA to rely on when evaluating what scientific research it would consider when adopting regulations pursuant to the Acts. In fact, to do so would very likely contradict the stated Congressional intent that such research be conducted with a focus on cooperation and coordination with other like-minded institutions. For these reasons, the commenter asserts that the Transparency Rule must be considered arbitrary and capricious under the APA as it would require the EPA to `rely on factors which Congress has not intended it to consider'.
Response: See response to Comment 2157-149. This final rule builds upon and is consistent with existing federal government open data initiatives and supporting federal agency guidance. 
EPA notes that the final rule does not cite any of the statutes that it administers as authority for this rule. This procedural rule does not interpret or apply provisions of the statutes that it administers. EPA has explained that this procedural rule has limitations on its scope in terms of conflicts with other substantive requirements and that, in situations where this rule applies, EPA will give greater consideration to those studies where the underlying data and models are available for independent validation. If implementing this procedural rule will result in a conflict with an environmental statute that EPA administers, then this procedural rule will yield. As discussed in the final rule preamble at III.A, EPA plans to issue statute-specific transparency regulations or programmatic regulations. EPA will address any statutory conflicts as appropriate when EPA develops these subsequent rulemakings.
Comment [6336-2687]: Commenter (6358) states that the EPA's process for promulgating this rule represents a gross deviation from the agency's typical action development process. EPA failed to perform an appropriate regulatory impact analysis, despite indications that implementation costs could be substantial. EPA has not received sufficient input from the OIRA, which conducted a highly truncated review and does not appear to have met with outside groups during that process, or from its SAB, which states that the rule "deals with a myriad of scientific issues for which the Agency should seek expert advice from the Science Advisory Board." Taken together, these deviations from typical process leave EPA unable to consider its proposal accurately, and constitute an arbitrary and capricious approach to rulemaking.
Response: See response to Comment 2157-149. EPA's Action Development Process is not legally binding and is a purely internal, collaborative process. In any event, the supplemental proposal underwent OMB and interagency review and was thoroughly reviewed by the SAB. The final rule discusses the SAB's findings and the SAB's report is available in the rulemaking docket.
Comment [6339-3188]: Commenter (6361) suggests that while EPA states that the new rule would be applied prospectively, it notes that the rule could impact prior records for older rules that are subject to recurring updates and reviews. The new rule does not point to any basis in law or fact to question the validity and rationale of prior EPA rulemakings, and thus it would be improper and contrary to the APA for EPA to reopen prior regulations for review due to alleged lack of transparency in past scientific studies. Without specific evidence, particular findings, and legal analysis regarding individualized federal agency decisions, the commenter states that it is unorthodox and legally suspect for EPA to issue a new regulation based on blanket assertions of a generalized, non-specific nature.
Response: This rule does not apply retrospectively to EPA regulations.
Comment [6340-1690]: Commenter (4879) states that they strongly support EPA's demonstrated commitment in this proposal to build upon the principles underlying the APA, Executive Orders 12866, 13777, and 13783, and guidance of OMB.
Response: Thank you for the comment.
Comment [6425-2643]: Commenter (6447) states that the current proposal would require EPA to ignore some of the best available science, and to ignore certain information submitted during rulemaking. In particular, the proposal would make it difficult or impossible to use most epidemiological studies, where the source data are almost always comprised of sensitive personal information. The Agency does not the option of suppressing this information, and courts have held that EPA "cannot ignore existing data." In short, the commenter contends that the proposed action would be illegal, and would not survive a legal challenge.
Response: See response to Comment 4829-2022.
Comment [6428-1352]: Commenter (6450) states that the proposed rule is arbitrary and capricious for myriad reasons. EPA has failed to demonstrate the benefits of the proposal or how any such benefits would outweigh the associated costs. 
Response: Regarding cost-benefit analysis, see response to Comment 6112-4452.
Comment [6845-1604]: Commenter (6867) provides that the APA, 5 U.S.C. § 500 et seq., prohibits agency action, findings, and conclusions that are arbitrary, capricious, an abuse of discretion, or otherwise not in accordance with law. See 5 U.S.C. § 706(1). In view of the overwhelming evidence that (i) a very high percentage of published studies are not reproducible, (ii) pre-publication peer review is not sufficient to ensure that a study lacks major errors, and (iii) pre-publication peer review is not sufficient to ensure that the computational code underlying a study lacks major errors, we respectfully submit that the Proposed Rule is necessary and arguably required by the APA. Indeed, according to the commenter, an agency action based on a study without fully disclosed data, methodology, and computational code is arguably so lacking in reasoned foundation as to be arbitrary and capricious.
Response: The commenter has not supported any of these assertions, which they contend are supported with "overwhelming evidence" with any evidence.
Comment [6888-4166]: Commenter (6909) opposes the proposed rule because it will do harm to important public health safeguards and will also negatively impact researchers and the scientific endeavor generally. The commenter states that the proposed rule's attempt to exclude perfectly reliable scientific studies from EPA's consideration is a thinly veiled attempt to allow the EPA to ignore sound science at its discretion if that science does not conform with the current Administration's political agenda. The commenter adds that the proposed rule would be very damaging and would fail to fulfill EPA's obligations under the APA for numerous reasons. 
The commenter states that a ban on EPA considering relevant studies that have been vetted through peer review but whose underlying data cannot been made publicly available or which cannot be replicated will not strengthen EPA's decision-making process and its ultimate policy making, it will weaken them.
Response: See response to Comment 4829-2022.
Comment [6894-3595]: Commenter (6915) states that the EPA's proposal would violate the APA, 5 U.S.C. §501 et seq., both because it is arbitrary and capricious, and because it flouts the Act's important procedural requirements. EPA claims that the entire basis for the proposed rule is to ensure that the "pivotal regulatory science" underlying EPA regulations is transparent. But EPA ignores existing laws and policies that already do exactly that and which also take into account the need to protect medical data and other confidential information. This proposed rule would promote transparency in name only; in truth, it would mean that EPA's important decisions would no longer be informed by the latest, best available, and generally accepted science. Disturbingly, the proposed rule's only failsafe is the EPA Administrator's sole discretion to determine on a case-by-case basis that compliance is "impracticable" when making data publicly available is "not feasible." But the proposal provides no standards to govern the Administrator's exercise of discretion in determining "impracticability or "feasibility" -- a recipe for the very arbitrariness that the APA prohibits.
Response: See response to Comments 2157-149, 4829-2022, 6112-3062, and 6163-5030.
Comment [6895-3615]: Commenter (4879) states that the EPA's failure to solicit input from the SAB and other scientific groups is exacerbated by its failure to meet the fundamental legal requirements for a valid rulemaking proposal under the APA. The APA requires that "general notice of proposed rulemaking shall be published in the Federal Register," including the "terms or substance of the proposed rule." 5 U.S.C. §553(b). The straightforward purpose of this requirement is to give the affected public an opportunity to provide meaningfully informed comment on an agency's proposal. See Home Box Office, Inc. v. Fed. Commc'n Comm'n, 567 F.2d 9, 35-36 (D.C. Cir. 1977). But here, EPA's NPRM is vague as to the actual parameters of the proposed rule, is open-ended in terms of the alternatives under consideration, and fails to provide key information such as projected costs. Courts will not hesitate to strike down final rules based on proposals so lacking in specificity. See, e.g., Horsehead Res. Dev. Co. v. Browner, 16 F.3d 1246, 1268 (D.C. Cir. 1994) (noting that "general notice that a new standard will be adopted affords the parties scant opportunity for comment").
Far from meeting the requirement to "disclose in detail the thinking that has animated the form of a proposed rule and the data upon which that rule is based," Home Box Office, 567 F.2d at 35 - 36, the proposal at issue here creates far more questions than it answers. Most fundamentally, the proposal fails to provide a rationale for EPA to act contrary to accepted scientific practice, i.e., to preclude consideration of probative scientific information that has been subject to rigorous peer review for the sole reason that underlying data are confidential and therefore not publicly available. The proposal states that "EPA believes the benefits of this proposed rule justify the costs," 83 Fed. Reg. at 18,772, but fails to provide any specific information, quantification, or analysis as to what EPA believes are the proposed rule's purported benefits or expected costs, including the significant costs from the loss of probative information that the proposed rule would work to exclude. For example, Section 30.7 of the proposed rule could be read to require EPA to undertake very costly independent review of "pivotal" science on which it relies, but Section 30.8, entitled "How is EPA to account for cost under this subpart?" states only that EPA will "minimize costs." Id. at 18,774. The absence of data and analysis in support of EPA's cost-benefit conclusion deprives the public of a meaningful opportunity to evaluate the proposal and thus violates EPA's duty under the APA.
Response: See response to Comments 2157-149 and 6112-4452.
Comment [6894-3630]: Commenter (6915) asserts that the proposed rule disregards the APA's bedrock requirement that an agency's decision-making be based on a consideration of the relevant factors and data. See Motor Vehicle Mfrs., 463 U.S. at 42-43 (articulating standard and citing numerous cases). An agency's action is arbitrary and capricious not only if the agency "entirely fail[s] to consider an important aspect of the problem," but also if it "relie[s] on factors which Congress has not intended it to consider." Id. at 43. The proposed rule would call for EPA to do both. First, in excluding studies and models from its consideration based only on whether the underlying data are publicly available or have been subject to additional independent review by EPA, EPA would be excluding studies and models that Congress has instructed it to consider by requiring it to use, for example, the "best available science" or "latest scientific knowledge." Second, because none of the statutes EPA administers specify that, in setting standards, it shall consider whether the studies and models it uses have publicly available data or have been independently reviewed by EPA, EPA would be using factors that Congress did not intend it to rely on in deciding to exclude studies and models based on the proposed rule. See Am. Trucking, 283 F.3d at 372 (finding that the CAA does not require EPA to "obtain and publicize the data underlying the studies on which the Agency relies"). According to the commenter, EPA's failure to consider otherwise relevant studies and models that do not meet the proposed rule's requirements would therefore be arbitrary and capricious. See Motor Vehicle Mfrs., 463 U.S. at 42-43.
Response: See response to Comment 4829-2022.
Comment [6895-3232]: Commenter (6895) states that the Proposal claims to be "consistent with the principles underlying the Administrative Procedure Act". But, the Proposal does not clearly identify the authority under which it is promulgated, is so vague as to undermine the ability to meaningfully comment, and in its substance contradicts the APA requirement that agencies must consider all the relevant matter presented in a rulemaking record. 
i. The Proposal does not clearly identify the authority under which EPA has promulgated it.
As discussed above, the CAA does not authorize the Proposal, which is clearly aimed at limiting the studies EPA can rely on and consider in CAA rulemaking. Additionally, APA section 553(b)(2) requires a proposed rule to provide a "reference to the legal authority under which the rule is proposed." While the Proposal's preamble lists various statutes the Agency claims support its action, a closer look reveals that none of those sections actually authorizes EPA to disregard scientific studies. Certainly that is the case for the CAA sections listed by the Agency in the Proposal, including the general rulemaking authority under the Act which requires the evaluation of all material in the record. EPA's suggestion that it has "general authority" to make rules to comply with its obligations is similarly unavailing here, as the general authority does not exist to make rules that are outside EPA's statutory obligations, or in violation of them  -  and a rule that would enable EPA to disregard relevant studies in a CAA rulemaking record is clearly outside the Agency's authority. Nowhere does the Act permit EPA to limit its own access to relevant science. 
ii. The Proposal's terms are so vague as to fail to afford an opportunity for meaningful comment.
Significant issues are presented in the Proposal only in the broadest terms, despite an APA requirement that a proposed rule includes "the terms and substance of the proposed rule or a description of the subjects and issues involved," 5 U.S.C. § 553(b)(3), and with enough specificity "to permit interested parties to comment meaningfully." As noted supra, we assume (as we must, given no other information from the Agency), that the Proposal is aimed at precluding EPA's reliance on studies for which the data are not made public. That is the natural reading of the proposed regulatory language, which is the most choate information offered in the Proposal. But even that reading is not certain, given the very broad discretionary nature of the exemptions provided for in the rule text (see, e.g., 83 Fed. Reg. 18,774, proposed 40 C.F.R. § 30.9, giving the Administrator the ability to grant an exemption to the data publication requirement if that is "impracticable," which is undefined in the Proposal). As a result, it is unclear how this rule would apply to EPA decision making, and how it would be different than the Agency's current protocol for considering studies relevant to specific rulemaking proposals. 
As one example, EPA alludes to "transparency" multiple times, but without defining what that concept means, or explaining why it believes existing statutes, regulations, guidelines and protocols fail to meet it. While asserting an interest in "ensur[ing] that the data and models underlying science is [sic] publicly available in a manner sufficient for validation and analysis," there is no more specific statement explaining how that would be implemented or why releasing confidential health and business information is necessary. Nor is the Agency's rationale completely clear in the Proposal. When commenters are left to guess, that is not lawfully sufficient notice of the subjects and issues involved. 
iii. The Proposal's substance conflicts with the APA requirement that Agencies consider relevant matter presented in the rulemaking record.
Section 553(c) of the APA requires that when finalizing a proposed rule, and after taking comment, an agency must "consider[] the relevant matter presented" in the record. To the extent that the Proposal envisions authorizing EPA to ignore or disregard or not to consider some studies submitted during a rulemaking comment period, such Agency action would be taken in direct violation of this provision of the APA. Nor, according to the commenter, is there any basis in the APA for EPA to decide to exclude or prohibit the consideration of otherwise relevant matter because it is based on confidential information not available to the general public.
Response: See response to Comment 2157-149.
Comment [6895-4397]: Commenter (6916) states that an agency's decision is arbitrary and capricious where it fails to consider important aspects of the problem, is counter to the evidence before it, or relies on factors Congress has not intended it to consider. When the action rescinds, or represents a significant about face from, previous agency rules or longstanding practice, it must provide a reasoned basis for doing so. If finalized as proposed, EPA's attempt to cut off access to significant scientific results would fall short on all of these factors and would also lead to further arbitrary and capricious agency decision making. 
Response: EPA did not finalize the proposal as is. See responses to Comments 2157-149 and 4829-2022.
Comment [6895-4399]: Commenter (6916) asserts that the failure to adequately justify a changed agency position is one of the hallmarks of arbitrary and capricious decision making. The Agency furthermore must "provide a more detailed justification than would suffice for a new policy...when, for example, its [revised] policy rests upon factual findings that contradict those which underlay its prior policy; or when its prior policy has engendered serious reliance interests that must be taken into account. It would be arbitrary and capricious to ignore such matters."
EPA, in the Proposal, fails to engage with the reasoning or facts underlying the Agency's longstanding policy position, which respected the confidentiality of underlying data and accounted for all relevant science. Further, this position has engendered significant reliance interests. Many of the "dose response" studies the Agency singles out for attention have been relied on in decision making for many decades, and have been the basis for many other subsequent work, on which the Agency also has relied. The Agency seems not to understand this or have a plan for how to manage the fallout from a final rule that denies access to all of that body of work  -  whether in an exclusively forward-looking way, or retroactively. But, "[a]n agency cannot simply disregard contrary or inconvenient factual determinations that it made in the past."
iii. If finalized, the Proposal's requirement to disregard relevant studies would ensure further arbitrary decision making. 
Were this rule to be finalized as proposed, it would permit or even require the Agency to disregard certain studies placed in the record in future substantive rulemakings under the CAA, among other statutes. Not only would that render the future substantive rules unlawful under the Act's requirements, but it would also make them arbitrary and capricious, as "failure to consider the evidence proffered renders [a decision] arbitrary and capricious" and the Agency may not "hastily discount[]" relevant studies. The Agency seems completely unaware that its new policy, if finalized, could lead to future rulemaking actions that are unlawful, arbitrary, and capricious.
Response: See response to Comment 4829-2022.
Comment [6912-3753]: Commenter (6933) suggests that the NPRM be withdrawn because to the extent it can be understood what it requires -- through the haze of vague definitions and exceptions -- it both fails to address the issue which it purports to be concerned with, and is inconsistent with EPA's obligations to use best available science under both statutory requirements and caselaw. In essence, the rule identifies the goal of improving the quality and transparency of EPA's science and uses it as an excuse to adopt policies that actually reduce the quality, and likely the transparency, of EPA's scientific decision-making. According to the commenter, there is a fundamental mismatch between the problem identified and the solution adopted. Such an approach is plainly arbitrary and capricious and inconsistent both with the APA and with the environmental statutes under which EPA purports to act.
Response: See final rule preamble at Section III.A for a discussion of the match between the problem and the solution provided by this rule. See response to Comment 4829-2022
Comment [8750-1475]: Commenter (9227) states that in addition to violating the requirements of the various statutes that EPA administers or is subject to, the Proposal suffers from a total failure to consider important dimensions of the profound shift in policy that it implements. In the Proposal, EPA neglects to consider the many legitimate reasons why a study's underlying data may not be publicly available -- reasons that have nothing to do with the quality of the study -- and fails to offer solutions consistent with these legitimate limitations. EPA makes vague gestures to various guidelines and practices issued by other agencies and scientific organizations, none of which actually support the Proposal's radical position that EPA should exclude consideration of studies that rely upon confidential data. EPA does not even establish that there is a real problem that the Proposal would actually address: nowhere in the Proposal does EPA identify any prior agency action that has been called into serious question due to a failure to release study data. According to the commenter, the EPA's utter failure "to consider an important aspect of the problem" and to provide an explanation for the Proposal that is consistent with the evidence before the agency renders the Proposal wholly arbitrary and capricious. See Motor Vehicle Mfrs. Ass'n. v. State Farm Mut. Auto. Ins. Co., 463 U.S. 29, 43 (1983). Likewise, the commenter suggests that the EPA's failure to explain its 180-degree change in position from its former belief that the lack of publicly available data does not render a study inappropriate for consideration in regulating is a hallmark of arbitrary and capricious decision-making. FCC v. Fox Telev. Stations, Inc., 556 U.S. 502, 515-16 (2009).
Response: See the final rule preamble at Section III.A for an explanation of why a change in policy is needed. See response to Comment 4829-2022.
Comment [8750-1480]: Commenter (9227) states that there are multiple reasons why underlying data are not publicly available for all studies.
There are legal and ethical requirements that restrict making public the data underlying studies, including rules to shield private personal information, requirements to maintain confidential business information, situations where obtaining the necessary permissions to release data are logistically difficult or impossible, and situations in which researchers have made significant investments in developing datasets that they intend to continue to work with for future studies. Not all of these barriers can be overcome, nor can they be overcome in every case. While there are ways potentially to address some of them, they can be extremely costly and burdensome, and/or may harm the prospects for further research. Accordingly, while the scientific community has made efforts to make more data publicly available, to the best of our knowledge all of the policies adopted by government and academic journals recognize that data is not, and need not be, publicly available to evaluate their quality.
a) Strong legal and ethical requirements limit the release of data in human subjects studies.
Particularly with respect to human subjects, there are strong legal and ethical privacy and confidentiality protections, which researchers are bound to respect. In some cases, researchers would be subject to civil or criminal penalties for violations.
The environmental health dose response studies targeted by EPA's proposal are likely to include human population studies (or epidemiological studies). Often the best available epidemiological studies contain extensive and sensitive data on individuals, such as environmental exposures, medical history (such as infant reproductive developmental abnormalities, children's behavioral and development problems, heart attacks or dementia among the elderly), dates of birth, residential address, drug use, race, socio-economic status (income, education), status of subjects' marriages, employment history, etc. For example, air pollution studies commonly use residential address information to assign air pollution exposures and link them to health effects. Other studies focused on genetically susceptible populations may also be linked to genetic databases or contain information on key genetic mutations that are strongly predictive of serious health risks, such as risk of Alzheimer's disease, and are thus very sensitive.
To conduct these studies, investigators must obtain informed consent from the study participants to collect protected health information, and investigators must sign documents promising to protect the privacy of this individually identifiable health information. Absent complex, difficult, and costly de-identification and redaction techniques, these data simply cannot be released publicly. As discussed below in section II.A.2.b), in some cases such techniques are simply not applicable or still leave significant risk of breach of privacy.
Additional protections apply to specific types of human subject information. For example, medical records are subject to strict requirements governing the use and disclosure of such information under the HIPAA of 1996. HIPAA requires researchers to protect identifiable information, and it provides that such information may only be disclosed for research purposes with the written consent of the person providing the information.
Another limitation on public availability of data is the requirement under the Federal Policy for the Protection of Human Subjects (also known as the Common Rule) that for all federally funded studies involving human research subjects, researchers must first obtain IRB approval and informed consent from study participants.
An IRB reviews each human subjects research project to ensure that the specific research protocol protects individual rights. Participants must be notified about the degree to which the confidentiality of their records will be maintained, and must receive appropriate notification and give consent if study data is to be shared outside the research team. The IRB also considers risks to the participants and how use of the information obtained may adversely impact the rights and welfare of the subjects. Most institutions have committed to comply with the Common Rule for all of their research, even when it is not federally-funded.
For studies that had received IRB approval prior to finalization of this proposed rule, there may be no practical opportunity to make the data publicly available. Even for new studies going forward, it may be extremely difficult, require additional (often unavailable) funding for elaborate protective measures, or simply impossible to obtain IRB approval for protocols that would allow the data to be made publicly available.
EPA's own SAB voiced these concerns that EPA was discounting the challenges to making even limited releases of data, saying:
The proposed rule oversimplifies the argument that "concerns about access to confidential or private information can, in many case, be addressed through the application of solutions commonly in use across some parts of the Federal government." For studies already completed or underway, the participation of human subjects is undertaken according to terms approved by the cognizant IRB. These terms can vary from study to study. In some cases, the data cannot be released simply by redacting portions of it. For example, data may have been collected with an assurance to the participating individuals that their data would be kept confidential.
Some researchers might respond by choosing to work only on public administrative datasets, but this would harm rather than strengthen science quality by curtailing scientific inquiry. Thus, the effects of EPA's proposed approach would cause some researchers to choose not to pursue research with human subjects, stifling scientific discovery, while others would forgo compliance with EPA's regulatory requirements and have their research ignored by EPA. As a result, EPA's proposal would both discourage the development of best available science as well as EPA's use of it.
b) There are especially significant barriers to public release of underlying data and models from studies that have already been completed.
With respect to studies that have already been completed, there are additional formidable barriers to public release of underlying data and models. Particularly, with older studies, simply finding the data sets and determining ownership may be expensive or impossible. For older studies with human subjects, obtaining consent to release of data may be practically impossible, and the data may have been collected in ways that would make protecting privacy with release difficult or impossible.
For some studies, administrative issues related to the data could be the most difficult barrier to overcome in providing for public release. Larger and more costly studies are often performed by groups of researchers within a university, across multiple institutions, or across multiple individual companies. Over time, the data itself may become lost or misplaced, or it may become unclear who actually owns and controls access to the data. Academics move among institutions, companies merge and spin off, and the initial agreements were not always clear in the first instance. Obtaining consent from multiple institutional players takes extensive time and resources, at minimum, and simply may no longer be possible in some instances.
These problems are exacerbated with respect to human subject studies. Researchers are legally and ethically obliged either to protect the privacy of the individual study subjects or attain each subject's consent to share data. This can be impractical for older studies and virtually impossible for larger studies, and extremely burdensome. For example, the Harvard Six cities study was started in 1975 and had 8,111 participants. The ACS CPSII extended analysis by Krewski in 2009, which is central to PM2.5 NAAQS standards, was initiated in 1979 and encompassed data from 500,000 study participants who lived in 116 metropolitan areas. For these types of situations, tracking down participants (or where the participants have passed away, their family members) to get consent is simply not realistically possible.
Even in situations where investigators might theoretically be able to attain consent, it would require extensive financial and human resources, which are usually simply not available, especially to academic researchers or to EPA. EPA ignores this prohibitive constraint and makes no attempt to address it.
c) There are additional significant barriers to public release of data in some situations, even for prospective studies.
Even with respect to prospective application of EPA's proposal, providing for public release of underlying data and models is costly and resource intensive, creating a serious disincentive for researchers to meet EPA's proposed requirements. Investigators willing to make their study underlying data publicly available would still face the logistical hurdle of making the data and models available in a manner sufficient for independent validation by the public. In addition to the cost of thoughtful and effective deidentification or redaction of sensitive information, the proposed text would likely require researchers to prepare annotated manuals including precise detail as to what variables were collected, how information was collected, and the rationale for each step taken. Some manuals alone run into hundreds of pages. One press account noted the example of publicly available datasets from the National Center for Health Statistics (NCHS), which can come with 100-page manuals; researchers would need to hire additional staff to meet such requirements. Yet EPA fails even to recognize (much less propose any means to address) the cost to researchers in time and money, on top of the constraints on academic research already imposed by the very limited funding available for this type of work.
In addition, there are other barriers to public release of underlying data. Studies conducted on behalf of industry or with industry cooperation may contain confidential business information, the release of which could jeopardize a company's competitiveness.
Also, in some instances, researchers cannot make their data sets public without losing much of the value to the researcher of these laboriously and meticulously collected sets of information. Research, especially those studies that include large numbers of human subjects, are incredibly human and capital-intensive endeavors. Moreover researchers may base years of work and multiple papers on unique datasets they developed and hold, and many scientists build their careers on carefully harvesting information from single large studies for years to come. It is not only unreasonable, but also unfair, to expect academic scientists to turn over their intellectual property and research investments, forgoing potential earnings and career advancements. Moreover, EPA's myopic and inflexible approach to data access gives no consideration to data sharing arrangements between researchers and the agency that could be developed to support EPA's consideration and integration of research.
If scientists are forced to choose between giving away their hard-earned data or forgoing any regulatory impact, it will discourage scientists from engaging in critical science that is targeted to help prevent disease and disability in our population. It appears that in many cases, scientists will choose to retain their datasets, with a worst-of-both-worlds result -- EPA will be deprived of valid scientific information and the scientific community will be discouraged from contributing their critical expertise to policymaking. EPA's Proposal does not consider the real-world implications of forcing such choices on researchers.
The commenter argues that the agency's failure to consider or examine any of these legitimate reasons for not making data publicly available is arbitrary and capricious.
Response: See the final rule preamble at Section III.A and Chapter 2 of this document for an explanation of the need for greater transparency. See response to Comment 4829-2022. EPA notes that the final rule does not preclude EPA from considering studies based on data that is not publicly available. See § 30.5 of the final rule.
Comment [8750-1497]: Commenter (9227) states that in the proposal, the EPA blithely and irrationally ignores or assumes away real and significant issues, suggesting that existing mechanisms and techniques can be used to protect privacy and confidentiality while making underlying research data publicly available. In fact, according to the commenter, the evidence (including several of the sources that EPA cites) indicates that the potential mechanisms alluded to by EPA would only have the potential to address some of the barriers cited above, have serious limitations even for those, and are actually becoming less effective as it becomes easier to combine and manipulate public data sets.
Response: This rule does not require that dose-response data be made publicly available, nor does this rule categorically exclude any dose-response data based on the availability of the dose-response data. As discussed in Section III.E of the final rule preamble, EPA will give greater consideration to pivotal science if the underlying dose-response data is available, including through tiered or restricted access, for independent validation.
Comment [8750-1637]: Commenter (9227) states that the EPA begins by stating that the "proposed rule is consistent with the principles underlying the Administrative Procedure Act and programmatic statutes that EPA administers to disclose to the public the bases for agency rules and to rationally execute and adequately explain agency actions." While EPA is correct that it must disclose the basis and provide an adequate explanation for rulemaking (principles EPA manifestly fails to follow in this Proposal), it does not follow that these principles either require or support the quite specific notion that dose response data and models must be publicly available. Nor does EPA attempt to explain how these broadest of rulemaking principles support EPA's specific proposed approach here.
Response: This rule does not require that any data be made publicly available.
Comment [8750-1678]: Commenter (9227) states that the Proposal turns away from EPA's Scientific Integrity Policy, which stresses "a firm commitment to evidence," endorses use of "the best available science" and "[r]equire[s] reviews. . . regarding the content of a scientific product to be based only on scientific quality considerations." The Proposal, on the other hand, inhibits use of sound scientific information and evidence by arbitrarily excluding science for reasons unrelated to its quality. While the policy "[r]ecognizes the value of independent validation of scientific methods" and facilitating "the free flow of scientific information" by making information available "including access to data and non-proprietary models underlying Agency policy decisions," this is proposed as a flexible standard and an ideal to aspire to, not an absolute rule that takes priority over other competing interests -- such as use of the best scientific information. As discussed more in Section VII.C [of the commenter's letter], this Administration has blatantly violated key aspects of the policy by silencing scientists and the dissemination of scientific information, which this Proposal seems aimed at continuing, directly undoing "EPA's longstanding commitment to the timely and unfiltered dissemination of its scientific information  -  uncompromised by political or other interference" and goal to communicate scientific findings openly and actively to the public. By now placing "transparency" ahead of use of the best available science, aside from violating statutory requirements, EPA is changing its own policies and priorities and must justify this new position.
In footnote 2, EPA dubiously claims the Proposal is consistent with the Memorandum for the Heads of Executive Department and Agencies on Scientific Integrity (Mar. 9, 2009). Notably, the Memorandum specifies, "Except for information that is properly restricted from disclosure under procedures established in accordance with statute, regulation, Executive Order, or Presidential Memorandum, each agency should make available to the public the scientific or technological findings or conclusions considered or relied on in policy decisions." Not only does the Memorandum provide no support for the notion that agencies should be barred from relying on studies where the underlying data is properly restricted from disclosure it additionally discusses disclosure only of findings and conclusions, not underlying data.
Thus, despite EPA's claims to the contrary, the Proposal marks a shift in policy that EPA has up to this point followed EPA arbitrarily fails to acknowledge this shift, to identify good reasons for the change, or to explain why EPA believes the proposed rule would be an improvement over current mechanisms utilized by EPA to ensure the integrity of EPA's actions.
Response: See the final rule preamble at Section III.A and Chapter 2 of this document for an explanation of the need for greater transparency. EPA notes that this final rule does not require EPA to categorically exclude any studies. See response to Comment 4829-2022.
Comment [8750-1682]: Commenter (9227) states that an agency rule is arbitrary and capricious if it "entirely failed to consider an important aspect of the problem." EPA's Proposal completely fails to consider the numerous barriers that currently exist to making underlying data public. As highlighted in OMB and EPA policies, there is an understanding that the worthy goal of ensuring greater transparency of scientific information is in tension with other compelling, competing interests such as privacy and confidentiality. When these two are in tension, existing policies have recognized that this will prevent certain data from being publicly released -- and that agencies still need to be able to use scientific information in these circumstances. Transparency goals should not override the ability of the agency to rely on otherwise valid scientific information as it goes about achieving its core mission. While the Proposal purports to take into account privacy and confidentiality concerns, it appears to do so by either grossly oversimplifying EPA's ability to address these concerns or by deeming all such information unusable -- essentially completely failing to consider the problems of this approach.
OMB Circular A-130 recognizes that the values of openness, transparency, and allowing the free flow of information between the federal government and the public are important values, they must be contextualized. Thus, it cautions: "Promoting openness and interoperability, subject to applicable legal and policy requirements, increases operational efficiencies, reduces costs, improves services, supports mission needs, and increases public access to valuable Federal information." Similarly it states: "The open and efficient exchange of scientific and technical Federal information, subject to applicable security and privacy controls and the proprietary rights of others, fosters excellence in scientific research and effective use of Federal research and development resources." Circular A-130 makes clear that "[p]rotecting an individual's privacy is of utmost importance. The Federal Government shall consider and protect an individual's privacy throughout the information life cycle." It requires that agencies recognize that "Federal information is managed by making information accessible, discoverable, and usable by the public to the extent permitted by law and subject to privacy, security (which includes confidentiality), or other valid restrictions pertaining to access, use, dissemination, and disclosure. . . ."
Further, Circular A-130 requires agencies to "[l]imit the creation, collection, use, processing, storage, maintenance, dissemination, and disclosure of [personally identifiable information] to that which is legally authorized, relevant, and reasonably deemed necessary for the proper performance of agency functions" and "[t]o the extent reasonably practicable. . .reduce all [personally identifiable information] to the minimum necessary for the proper performance of authorized agency functions."
The appendix to the Circular realizes that privacy protections require ongoing progress and:
Emerging technologies and services may continue to shift the ways in which agencies acquire, develop, manage, and use information and technology. As technologies and services continue to change, so will the threat environment. Agency programs must have the capability to identify, respond to, and recover from current threats while protecting their information resources and the privacy of the individuals whose information they maintain.
OMB Memorandum M-14-06 specifically lays out policies intended to help agencies make the most of "administrative data that cannot be made publicly available due to statutory, regulatory, or policy protections," for statistical purposes, including "activities typically characterized as research, evaluation, and analysis, as long as the focus of those activities is on reporting aggregate findings about a group." It notes "[s]ome administrative data can be publicly released, whereas other administrative data cannot be released. . . [and] it is the case that both types of administrative data (public and nonpublic) can be useful for Federal statistical purposes," suggesting agencies should not abandon reliance on data not able to be publicly released.
OMB Memorandum M-11-02 "strongly encourages Federal agencies to engage in coordinated efforts to share high-value data" but notes that in certain cases sharing data will contravene other compelling concerns and that federal agencies need to think about applicable privacy laws, regulations, and policies to "fully protect[] individual privacy" and preserve public trust. Unlike the Proposal, it takes a more nuanced approach recognizing that sharing data is not always appropriate and should only be done "responsibly and appropriately."
OMB recognizes that even when just sharing information among agencies, privacy concerns must be weighed against those benefits that agencies can achieve with sharing data: "Agencies should work together to determine what data sharing opportunities are desirable, feasible, and appropriate. In general, data sharing should only be pursued if the benefits outweigh the costs."
OMB Memorandum M-10-06 also encourages "a plan for timely publication of the underlying data. . . in an open format and as granular as possible, consistent with statutory responsibilities and subject to valid privacy, confidentiality, security, or other restrictions." The memorandum aims to achieve "transparency, participation, and collaboration," recognizing that not making data available does not deter those goals when there are valid concerns and the legitimacy of the data is not otherwise questioned.
EPA's Draft Strategic Data Action Plan Version 1.0 similarly aims to work towards a more open government, and to increase the public's access to high quality data. However, the agency recognizes barriers to this goal, not applying the plan to "data resources containing CBI or sensitive data that are not available for public access." In similarly recognizes that "[i]n order to protect the privacy and security of the public, businesses, and US Government staff and operations, some types of data may be deemed sensitive and will not be made public or published on Data.gov."
The commenter states that these examples all highlight instances where EPA and OMB have recognized that privacy and confidentiality present ongoing concerns that are not easily addressed and that conflict with other aims of federal government. Yet, they recognize that protecting information in these cases is a valid path, and not making data public does not compromise the validity of the findings or conclusions upon which the data is based and should prevent agencies from using those findings, conclusions, and data to inform their work. According to the commenter, the Proposal provides no explanation for why EPA is now changing its view to a conflicting one, making the Proposal arbitrary.
Response: See the final rule preamble at Section III.A and Chapter 2 of this document for an explanation of the need for greater transparency.
Comment [8750-1697]: Commenter (9227) states that the proposed rule asserts that a broad interest of the current Administration is to "ensure that the data and models underlying scientific studies that are pivotal to. . . regulatory action are available to the public" and to "change agency culture and practices regarding data access so that the scientific justification for regulatory actions is truly available for validation and analysis." However, the Proposal specifies a particular interest and initial focus on "dose response data and models" as evident throughout the preamble and proposed regulatory provisions.
Dose-response studies are a critical element of risk assessments for toxicants including air pollutants. Assessment of a toxicants risks typically proceeds through a four-step process: 1) hazard identification, 2) dose-response assessment, 3) exposure assessment, and 4) risk characterization. Dose-response assessment describes the relationship between exposure to a toxicant and observed effect on human or ecological receptor. EPA provides the following description of dose-response on its website: "Dose-Response Assessment...characterizes the quantitative relationship between chemical exposure and each credible health hazard. These quantitative relationships are then used to derive toxicity values." Dose-response plays a central role in the evaluation of chemical risks as it provides the characterization of the potency or effect size of the toxicant. In other words, dose-response assessment is used to determine the levels of exposure at which adverse effects will occur and thus informs what risk management actions should be taken to protect human and ecological health. Dose-response assessments are commonly used to derive chemical toxicity values. The lower a substance's toxicity value the greater its potency and the less exposure is necessary for an effect to occur.
EPA reveals the underlying motivation behind its interest in transparency of dose response data and models on page eight of the Proposal, where it states:
      In addition, this proposed regulation is designed to increase transparency of the assumptions underlying dose response models. As a case in point, there is growing empirical evidence of non-linearity in the concentration-response function for specific pollutants and health effects. The use of default models, without consideration of alternatives or model uncertainty, can obscure the scientific justification for EPA actions. To be even more transparent about these complex relationships, EPA should give appropriate consideration to high quality studies that explore: A broad class of parametric concentration-response models with a robust set of potential confounding variables; nonparametric models that incorporate fewer assumptions; various threshold models across the exposure range; and spatial heterogeneity. EPA should also incorporate the concept of model uncertainty when needed as a default to optimize low dose risk estimation based on major competing models, including linear, threshold, and U-shaped, J-shaped, and bell-shaped models.
This excerpt raises several troubling and erroneous concepts that are contrary to core scientific tenets and best practices in chemical hazard and risk assessment as discussed extensively in a seminal 2009 report by the National Academies (Academies): Science and Decisions: Advancing Risk Assessment (Science and Decisions). The report was requested and sponsored by EPA's National Center for Environmental Assessment and was developed over a three-year period by a 15-member committee that included state environmental agencies, non-governmental organizations, industry, and academic institutions. The committee was specifically tasked with "developing scientific and technical recommendations for improving risk analysis approaches used by EPA, including providing practical improvements that EPA could make in the near term (2-5 years) and in the longer term (10-20 years)." The report has been cited over 400 times in the scientific literature.
The Proposal fails to discuss these best practices for risk assessment, much less provide any persuasive reason for departing from them. The Proposal provides no support for its assertion that there is "growing empirical evidence" of nonlinearity in dose-response relationships; fails to acknowledge or contend with the National Academies' finding that nonthreshold dose-response relationships are common for toxicants, and should be assumed as a default; fails to discuss the well-known rationales put forward by the National Academies for using default models; and irrationally prioritizes consideration of studies that employ a wide range of dose-response models, without any consideration for whether those alternative dose-response models are appropriate for risk assessment. Alarmingly, the Proposal offers no analysis of how the proposed requirements to consider threshold-response relationships and avoid default models would further the protection of human health and the environment -- and gives no indication that the Agency has considered whether its proposed approach affords appropriate protection for the public in evaluating the risks of dangerous pollutants and toxicants. The proposed requirement is irretrievably arbitrary and unjustified and must be withdrawn.
Response: EPA notes that the final rule only applies to dose-response data underlying pivotal science used in significant regulatory actions.
Comment [8750-1761]: Commenter (9227) states that it is arbitrary and capricious to "`entirely fai[l] to consider an important aspect of the problem' when deciding whether regulation is appropriate." Michigan v. EPA, 135 S. Ct. 2699, 2707 (2015) (quoting State Farm, 463 U.S. at 43). As in Michigan, failure to consider the costs and benefits of a regulation where there is no statutory bar to doing so is arbitrary and capricious.
The proposed rule entirely fails to comply with the requirements of non-arbitrary-and capricious rulemaking because it fails to disclose, much less analyze or consider, any of the costs of the rule; barely discusses and does not analyze or quantify the benefits; does not provide any reasoned explanation of why the benefits of the rule justify its costs; and does not consider potential alternatives. The Proposal's discussion of costs and benefits is a scant two paragraphs (and was apparently not included at all in the version sent to the Office of Management and Budget). The proposed rule begins by conclusory asserting that "EPA believes the benefits of this proposed rule justify the costs." It then briefly discusses the perceived benefits, incorrectly suggesting that the NAS shares EPA's view by citing to a publication that discusses both risks and opportunities of expanding access to research data, and does not discuss at all the costs and benefits of ignoring relevant science in regulatory decision-making. It then merely states that the "action should be implemented in a cost-effective manner," citing vaguely to "recent activities of the scientific community and other federal agencies" without any concrete examples or analysis. The preamble's discussion emphasizes that the Proposal does not compel EPA to make information available where it concludes that doing so is not possible, but omits that if compliance is not possible, EPA will not consider the study, which has its own costs. It then concludes by citing the working paper of the Mercatus Center that baldly asserts that improvements in reproducibility "can be thought of as increasing the net benefits of regulation because they would avoid situations in which costs or benefits are wrongly estimated to occur or in which regulatory costs are imposed without corresponding benefits." Setting aside the lack of substantiation for this assertion, it entirely omits situations in which costs and benefits are wrongly estimated because the relevant science is not used -- and the costs that would be imposed on society if EPA inadequately protects communities from harmful pollution or toxic exposures.
Indeed, the Proposal nowhere discusses its significant costs in either quantitative or qualitative terms, costs that have actually been examined by independent organizations, and that are susceptible to analysis. If the Proposal is truly "designed to provide a mechanism to increase access to" data "in a manner consistent with statutory requirements for protection of privacy and confidentiality of research participants," 83 Fed. Reg. at 18,770, then it will have significant costs. And if, as it appears, the Proposal's true "mechanism" is excluding science from regulatory decision-making, its costs will be even greater in the form of insufficiently protective regulations.
If it were not possible to quantify and monetize any of the costs, which is not the case here as discussed below, EPA would still be required under Executive Order 12866 and the requirements of rational rulemaking to identify and discuss the qualitative costs of this Proposal. It is inherently irrational for an agency to take an action without any consideration of any costs, disadvantages, or negative effects of that action. The qualitative costs of this Proposal include the costs to researchers of actions they must undertake to protect the confidentiality of patient and subject data, as well as to compile and make public their raw data, and the potential loss of subjects (and attendant damage to research efforts and results) due to confidentiality concerns. There are also various costs to the agency of administering the regulation, which include contacting researchers, gathering data, ensuring that patient confidentiality and confidential business information are not disclosed. Additional costs could also be incurred through conducting any additional peer reviews required by proposed section 30.7 and any additional analyses imposed by proposed section 30.6's requirement that "EPA shall clearly explain the scientific basis for each model assumption used and present analyses showing the sensitivity of the modeled results to alternative assumptions." Most importantly, there are potentially huge costs of regulating without using the relevant science merely because the underlying raw data is not publicly available. If studies supporting a stronger standard are excluded and EPA can therefore only justify a weaker requirement that leaves large numbers of people at risk of health effects from a pollutant, pesticide, or chemical, then this Proposal could impose enormous costs for each insufficiently protective regulation. Yet the Proposal fails even to mention these costs, let alone discuss their scope and significance.
In addition, many of these costs can be quantified and monetized, but EPA has neither attempted to do so nor explained why it could not. For example, EPA has extensive information available to it on what the agency would need to do to implement this Proposal and how much those activities would cost. In fact, EPA already gathered much of this data and provided it to the CBO for use in estimating the costs of a similar (though not identical) proposal from Congress, the HONEST Act. With respect to the Congressional proposal, CBO concluded, just with respect to the costs to EPA, that "based on information from the EPA and other federal agencies, as well as organizations and researchers in the scientific community that publish in peer-reviewed journals," EPA "could spend between a few million dollars per year to more than one hundred million dollars per year ... to ensure that data and other information underlying studies are publicly available in a format sufficient to allow others to substantially reproduce the results of studies." In the 2017 estimate, CBO concluded that "[i]f the EPA continued to rely on as many scientific studies as it has used in recent years ... then CBO estimates that the agency would need to spend at least $100 million dollars per year to upgrade the format and availability of those studies' data," "on average, $10,000 per scientific study." Such costs would cover the costs of "obtaining all the underlying data used in a study, reviewing the data to address any confidentiality concerns, formatting the data for public access, providing access to the computer codes and models used in the study's analysis, and providing descriptions and documentation on how to access the data." Notably, this does not include the cost to researchers to engage in this effort. As Deputy Assistant Administrator Nancy Beck noted, during the development of the Proposal, requiring "a huge amount of data to be submitted to the agency" would "be incredibly burdensome" and "not practical."
Even the Mercatus working paper -- apparently the only thing EPA relied upon in discussing the costs and benefits of the Proposal, 83 Fed. Reg. 18,772 n. 24, notes, with respect to the HONEST Act, that "[t]he cost of providing access to data has been one of the primary concerns about requiring access to data used by the federal government." Far from concluding, as the Proposal suggests, an increase in net benefits from greater reproducibility, the Mercatus working paper simply explained a figure the authors were suggesting could be calculated (the point where net benefits would be positive); the authors do not themselves calculate the benefits, and admit that their "estimates of the benefits of public access to data supporting federal regulatory decisions fall short of proving that the benefits outweigh the associated costs." And while the Mercatus working paper disagrees with CBO's cost estimates, it does not argue that that requiring access to data is cost-less; indeed, it discusses the "costly activities and services that need to be performed," including activities related to "data collection and data accessibility." According to that working paper, data collection requires "correspond[ing] with researchers and publishers to obtain the data, review[ing] the data for confidentiality concerns, format[ting] the data for public access, publicly post[ing] the computer code and models used in each study's analysis, and provid[ing] descriptions and documentation on how to obtain the date." Data accessibility requires "computer processing services to construct and maintain data bases to store study-related information." While the actual calculations put forward by the Mercatus working paper appear faulty (for example, it entirely omits the cost to researchers to compile and make their data public, does not include the costs of ensuring patient privacy is protected, and makes assumptions about the similarity of a chemical manufacturer collecting its own studies and EPA collecting and disseminating information of other researchers), the working paper at least acknowledges that there are costs, something EPA's Proposal completely ignores.
Nor does the proposed rule disclose the cost -- highlighted on the very first page of a NAS report on data access -- that "perceived risks to privacy and confidentiality reduce survey participation," a cost that the NAS explains is "borne out by research." NAS explains that this "threatens the research enterprise itself, because concerns about privacy and confidentiality are among the reasons often given by potential respondents for refusing to participate in surveys, and those concerns have been shown to affect behavior as well. The NAS panel emphasized: "Any confidentiality breach that became known would be likely to heighten such concerns and, correspondingly, reduce survey response rates. Efforts to increase researchers' access to data must, therefore, take into account the need to avoid increasing the actual and perceived risks of confidentiality breaches." The Proposal does not so much as discuss this potential cost.
This confidentiality risk has a further cost: it affects the quality of the data collected. As the NAS explained:
The reason for confidentiality pledges and for stringent procedures to prevent disclosure is that they improve the quality of data collected from individuals, households, and firms. It is essential that respondents believe they can provide accurate, complete information without any fear that the information will be disclosed inappropriately. Indeed, if the information was disclosed, harm might come to an individual respondent.
The Proposal's only acknowledgment of this complex problem and cost is its statement that "EPA believes that concerns about access to confidential or private information can, in many cases, be addressed through the application of solutions commonly in use across some parts of the Federal government." Remarkably, EPA does not cite a single example of these common solutions, citing only vaguely to "examples from the U.S. Department of Health and Human Services, National Institute of Standards and Technology, U.S. Department of Education, and the U.S. Census Bureau" and some hyperlinks not in the Proposal added to the docket almost a month into the comment period. Accordingly, not only does the Proposal include no analysis of these alleged solutions and their costs and benefits, it does not even explain what the solutions are that EPA believes address this concern.
And if EPA complies with the regulation not by spending the money to make data publicly available, and if the research community does not bear those costs itself, see 83 Fed. Reg. at 18,770-71 ("Nothing in the proposed rule compels the disclosure of any confidential or private information in a manner that violates applicable legal and ethical protections."), then it appears that EPA would simply ignore studies that do not comply with the regulation. See 83 Fed. Reg. at 18,769 n. 3 ("EPA is proposing to exercise its discretionary authority to establish a policy that would preclude it from using such data in future regulatory actions."). That course of action has its own significant costs, and EPA provides no analysis in the Proposal of the magnitude of studies that it has previously relied upon that it could no longer rely upon in regulating. See 2017 CBO Estimate ("EPA officials have explained to CBO that the agency would implement H.R. 1430 with minimal funding and generally would not disseminate information for the scientific studies that it uses to support covered actions. That approach to implementing the legislation would significantly reduce the number of studies that the agency relies on when issuing or proposing covered actions...."). As the SAB noted in its May 12, 2018 letter, "[t]he proposed rule does not include any assessment of the impact of data restrictions on existing or future regulatory programs. Without access to the restricted data, regulatory programs could become more or less stringent than they otherwise would be, with consequences for both regulatory costs and benefits."
Likewise, EPA has included only a cursory mention of the expected qualitative benefits of the Proposal, with no discussion of the anticipated likelihood, scope, or impact of the suggested benefits, let alone any effort to quantify them, much less monetize them. EPA simply assumes that the Proposal will "improve the data and scientific quality of the Agency's actions and facilitate expanded data sharing an exploration of key data sets" without any analysis or evidence. In fact, as we have explained, the likely outcome of the Proposal is that it will degrade the data and scientific quality of the Agency's actions by ignoring relevant science simply because the underlying data is not publicly available. Moreover, EPA's finding is not consistent with the conclusions of the National Academies, as the Proposal suggests. As also explained above, the NAS report highlighted both the risks and benefits of making data publicly available and nowhere concluded that there were benefits to excluding data from the agency's regulatory decisions simply because the underlying data was not publicly available. Nor does the agency analyze how likely its Proposal is to actually facilitate expanded data sharing, and its main aim appears to be excluding science as it does not actually provide any funding, mechanisms, or best practices for sharing data.
It is more than ironic that EPA claims -- without any data or analysis -- that its Proposal will increase the net benefits of other regulations while it does nothing to actually consider the costs and benefits of the Proposal itself. Moreover, there is no reason to think that excluding relevant science merely because the underlying data is not publicly available would increase the net benefits of a regulation. For example, it appears that under the proposed rule EPA would exclude a peer-reviewed, published study whose conclusion had been reproduced based upon numerous different datasets (and whose underlying data, though not publicly available, had been reevaluated by outside experts), while including a study that had had no peer review, was not published, had no corroborating studies, and had not actually been replicated or reproduced, merely because the underlying data was made publicly available. That is simply not a recipe for more accurate decision-making.
The proposed rule also violates the APA and other statutes' requirements for reasoned decision-making by failing to consider any alternative approaches, much less their costs, here. This is particularly irrational in this context where it appears that many of the benefits sought by EPA could be largely achieved with much less burdensome and costly approaches. A critical element of reasoned decision-making is consideration of alternatives which are congruent with agencies' statutory responsibilities and objectives. Motor Vehicle Mfrs. Ass'n v. State Farm Mut. Auto. Ins. Co., 463 U.S. 29, 48, 50 (1983) (safety agency acted arbitrarily in failing to consider alternative safety measures after rejecting passive restraints). EPA failed to consider other methods to ensure scientific robustness at the agency. For example, the SAB letter notes that "[t]he proposed rule fails to mention that there are various ways to assess the validity of prior epidemiologic studies without public access to data and analytic methods." The Proposal does not consider any alternatives to ensuring that studies are reliable even where the underlying data cannot be made public because of privacy or other concerns.
Furthermore, by failing to consider costs and benefits, the Proposal contravenes Executive Order 12866. Executive Order 12866 requires agencies to assess the costs and benefits of proposed regulations and propose or adopt a regulation only upon a reasoned determination that the benefits justify the costs. For "significant regulatory actions," like the proposed rule, 83 Fed. Reg. at 18,772, the agency must provide:
      An assessment, including the underlying analysis, of benefits anticipated from the regulatory action (such as, but not limited to, the promotion of the efficient functioning of the economy and private markets, the enhancement of health and safety, the protection of the natural environment, and the elimination or reduction of discrimination or bias) together with, to the extent feasible, a quantification of those benefits;
      An assessment, including the underlying analysis, of costs anticipated from the regulatory action (such as, but not limited to, the direct cost both to the government in administering the regulation and to businesses and others in complying with the regulation, and any adverse effects on the efficient functioning of the economy, private markets (including productivity, employment, and competitiveness), health, safety, and the natural environment), together with, to the extent feasible, a quantification of those costs; and
      An assessment, including the underlying analysis, of costs and benefits of potentially effective and reasonably feasible alternatives to the planned regulation, identified by the agencies or the public (including improving the current regulation and reasonably viable nonregulatory actions), and an explanation why the planned regulatory action is preferable to the identified potential alternatives.
The agency must also make these assessments and analyses "available to the public." Executive Order 13563 reaffirms these principles and requirements, explaining that agencies "must take into account benefits and costs, both quantitative and qualitative."
Agencies are further encouraged to weigh the costs and benefits of developing higher information quality in OMB's Information Quality Guidelines. Costs that the Guidelines encourage agencies to consider include "costs attributable to agency processing effort, respondent burden, maintenance of needed privacy, and assurances of suitable confidentiality." EPA's existing information quality guidelines track the OMB Guidelines closely. EPA's disregard of the Guidelines' recommended weighing costs and benefits further contributes to the arbitrariness of EPA's failure to consider the costs of the Proposal.
The Proposal's failure to analyze and disclose costs and benefits cannot be cured in a final regulation. Should EPA not abandon this misguided Proposal, it must re-propose it after first analyzing its costs (both to public health, to researchers, and to the agency itself) and benefits, and providing the requisite opportunity for public comment on its analysis. As discussed further below in Section VIII.D [of their comment letter], the public cannot meaningfully comment on the proposed rule without understanding the actual costs and benefits of the Proposal, the alternatives EPA considered, and the analyses underlying EPA's assessments.
Response: See response to Comment 6112-4452. Any further cost-benefit analyses associated with implementing this procedural rule will be conducted as appropriate in forthcoming rulemakings.
Comment [8750-1824]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter states that Exec. Order No. 13563, 76 Fed. Reg. 3821 (Jan. 21, 2011) provides: "Our regulatory system must protect public health, welfare, safety, and our environment while promoting economic growth, innovation, competitiveness, and job creation. It must be based on the best available science."
Executive Order No. 13563 requires agencies to utilize the "best available science" in regulatory actions. This requirement is further encoded in numerous statutes and policies that EPA implements. EPA states in the proposed rule that: "The best available science must serve as the foundation of EPA's regulatory actions." However, as the comments raise more thoroughly, by arbitrarily restricting the scientific studies EPA will consider, this proposed rule will hinder EPA's use of the best available science and therefore violates the command of Exec. Order No. 13563 and other versions of these requirements.
Furthermore, this Executive Order requires agencies to "ensure the objectivity of any scientific and technological information and processes used to support the agency's regulatory actions" consistent with the President's Memorandum for the Heads of Executive Departments and Agencies, ``Scientific Integrity'' (March 9, 2009). As their comments note, however, the proposed rule along with the provision allowing the Administrator to grant discretionary exemptions will harm the objectivity of scientific and technological information and processes at EPA by paving the way for politics, rather than objective scientific criteria, to dictate which scientific studies are considered.
Response: See response to Comment 6163-1114. EPA issued this rulemaking consistent with its obligations under all applicable Executive Orders.
Comment [8750-1826]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter notes that the EPA claims about the proposal that "[b]y better informing the public, the Agency in[sic] enhancing the public's ability to understand and meaningfully participate in the regulatory process." EPA then cites to the Memorandum for the Heads of Executive Departments and Agencies on Scientific Integrity. Not only does the proposal conflict with this memorandum, but it will make it more difficult for the public to meaningfully participate in the regulatory process.
The memorandum sets out a number of actions for agencies to take to ensure scientific integrity. Just one of these factors involves making scientific and technological information publicly available, notably specifying, "Except for information that is properly restricted from disclosure under procedures established in accordance with statute, regulation, Executive Order, or Presidential Memorandum, each agency should make available to the public the scientific or technological findings or conclusions considered or relied on in policy decisions." The memorandum thus supports only making scientific findings and conclusions publicly available, not the data underlying those findings and conclusions. Further, it correctly notes that some information is properly restricted from disclosure. It does not say that the inability to disclose such information should prevent it from being considered by agencies. The memorandum thus provides no support for the notion that agencies should be barred from relying on studies where the underlying data cannot be disclosed. The memorandum's narrow approach to public disclosure should not be taken to support EPA's proposal but rather counsels against the proposal's mandate that all underlying data be made publicly available.
According to the commenter, the EPA's proposal fundamentally conflicts with the heart of the memorandum -- that "[t]he public must be able to trust the science and scientific process informing public policy decisions." To earn this trust, the memorandum declares: "Political officials should not suppress or alter scientific or technological findings and conclusions." By discarding scientific studies where underlying data cannot be made publicly available, this proposal will result in scientific findings being suppressed. By allowing the Administrator to grant exemptions to this policy based on their discretion with no public record or explanation, the proposal allows for the Administrator to pick and choose based on their preference the science informing the agency's actions, eroding the public's trust in the science informing public policy decisions.
The memorandum provides a number of ways in which agencies can ensure scientific integrity which the proposal does not consider including: hiring candidates for science and technology position based on their "knowledge, credentials, experience, and integrity," having in place appropriate rules and procedures to ensure integrity of the scientific process, establishing scientific processes such as peer review and accurately reflecting scientific and technological information, establishing procedures to identify when scientific integrity may be compromised, including establishing whistleblower protections. EPA does not explain why any of these pathways would not serve as a better means of ensuring scientific integrity.
Response: See the final rule preamble at Section III.A and Chapter 2 of this document for an explanation of the need for greater transparency.
Comment [8750-1828]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter states that in footnote 3 of the proposal, EPA notes that "courts have at times upheld EPA's use [sic] non-public data in support of its regulatory actions" and cites to Coalition of Battery Recyclers Ass'n v. EPA, 604 F.3d 613, 623 (D.C. Cir. 2010) and American Trucking Ass'ns v. EPA, 283 F.3d 355, 372 (D.C. Cir. 2002). These cases indeed held that EPA's prior, longstanding position of relying on scientific studies even when the underlying data could not be made publicly available was reasonable. It is well-established that agencies must acknowledge changes in position and "show that there are good reasons for the new policy." This footnote, the only mention of EPA's previous policy, does not sufficiently acknowledge or explain why EPA is now changing its position.
In American Trucking Ass'ns v. EPA the Court held that the CAA did not require EPA to make public underlying data where EPA relied on the study itself and not the raw data underlying the study. The Court stated that such a requirement "would be impractical and unnecessary." They agreed with EPA's then statement that:
If EPA and other governmental agencies could not rely on published studies without conducting an independent analysis of the enormous volume of raw data underlying them, then much plainly relevant scientific information would become unavailable to EPA for use in setting standards to protect public health and the environment.... Such data are often the property of scientific investigators and are often not readily available because of ... proprietary interests ... or because of [confidentiality] arrangements [with study participants].
In Coalition of Battery Recyclers Ass'n v. EPA, the Court cited American Trucking Ass'ns v. EPA and held, again, that EPA was permitted to rely on studies without making the underlying data public. They noted, "raw data often is unavailable due to proprietary interests of a study's scientific investigators or confidentiality agreements with study participants." These court cases thus not only upheld EPA's prior practice as permissible, but went on to agree that EPA's prior practice was preferable and necessary in light of these other policy concerns.
The commenter notes that the EPA provides no response to this history, saying only: "Historically, EPA has not consistently observed the policies underlying this proposal. . . . " EPA fails explicitly to recognize that this proposal changes its past policy and provides no justification in light of the compelling opposing points that both EPA and the Courts previously recognized as deterring this approach.
Response: See the final rule preamble at Section III.A and Chapter 2 of this document for an explanation of the need for greater transparency. EPA notes that neither decision precludes EPA from considering data availability when developing significant regulatory actions or influential scientific information.
Comment [8750-1829]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter states that the EPA claims that the proposal is consistent with Executive Order No. 13777. This Executive Order provides no support for the proposal, and in fact is targeted at eliminating regulations including those that are "unnecessary" and "ineffective," which, as our comments detail, the proposal clearly would be.
This Executive Order creates a Regulatory Reform Task Force and calls for them to identify for repeal, replacement, or modification regulations that among other criteria are inconsistent with the requirements of section 515 of the Treasury and General Government Appropriations Act, 2001 (44 U.S.C. 3516 note), or the guidance issued pursuant to that provision, in particular those regulations that rely in whole or in part on data, information, or methods that are not publicly available or that are insufficiently transparent to meet the standard for reproducibility.
As described in detail in our comments and below, contrary to the inference drawn here in Exec. Order No. 13777, the Data Quality Act and OMB's guidelines issued pursuant to it do not require research data and models to be made publicly available for reproducibility purposes in order for agencies to rely on the scientific findings and conclusions produced using that data.
Executive orders cannot override the statutory requirements that EPA use the best available science or the laws governing administrative procedure including the APA. The proposal's "consistency" with this executive order then cannot serve as a legal basis for EPA to adopt an arbitrary and capricious policy that contravenes these best available science requirements reflected in the statutes EPA administers.
Additionally, Executive Order No. 13777 by its terms requires only the identification of regulations that rely in whole or in part on data not publicly available, it says nothing about precluding agencies from relying on such studies and does not and cannot require agencies to adopt such practices. However, if the proposed rule is to be "consistent" with the executive order then it must also follow section 3(e):
In performing the evaluation described in subsection (d) of this section, each Regulatory Reform Task Force shall seek input and other assistance, as permitted by law, from entities significantly affected by Federal regulations, including State, local, and tribal governments, small businesses, consumers, non-governmental organizations, and trade associations.
There is no evidence that EPA consulted with the many stakeholders impacted by this policy, including the medical or scientific research communities, which have been largely opposed to this policy.
Response: EPA issued this rulemaking consistent with its obligations under all applicable Executive Orders. 
This procedural rule does not interpret or apply provisions of the statutes that it administers. EPA has explained that this procedural rule has limitations on its scope in terms of conflicts with other substantive requirements and that, in situations where this rule applies, EPA will give greater consideration to those studies where the underlying data and models are available for independent validation. If implementing this procedural rule will result in a conflict with an environmental statute that EPA administers, then this procedural rule will yield.
In addition, EPA notes that government and private stakeholders have extensively engaged with this rulemaking. See response to Comment 2157-149. 
Comment [8750-1834]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter states that the EPA claims the proposal is consistent with Executive Order No. 13783. However, Executive Order No. 13783 calls for agencies to consider salient information that the proposal has patently ignored. Executive Order No. 13783 calls for agencies to consider the costs and benefits "that are based on the best available science and economics" to ensure sound regulatory decision-making. The proposal provides no analysis of the costs and benefits of implementing this new policy, despite there likely being high costs to making research data public with little evidence of significant benefits achieved from this policy alone. Further, by arbitrarily excluding scientific information that EPA may use in its regulatory analyses, the proposal conflicts with the executive order's command to employ the best available science and economics.
Response: See the final rule preamble at Section III.A and Chapter 2 of this document for an explanation of the need for greater transparency. See response to Comment 4829-2022 and 6112-4452. EPA issued this rulemaking consistent with its obligations under all applicable executive orders. Regarding cost-benefit analysis, see response to Comment 6112-4452.
Comment [8750-1836]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter states that the EPA wrongly claims that the proposal is "consistent with. . . the focus on transparency in OMB's Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and Integrity of Information Disseminated by Federal Agencies." To say that OMB's Guidelines have a "focus on transparency" that is furthered by EPA's proposal is a gross oversimplification. EPA here appears to suggest that transparency is the highest objective to be achieved, divorced from any consideration of whether transparency hinders or furthers any other goals. The OMB Guidelines, while imposing high standards of quality, objectivity, utility, and integrity of information disseminated by Federal Agencies, recognize the need to implement controls "flexibly, and in a manner appropriate to the nature. . . of the information to be disseminated." They suggest thinking about transparency strategically to further the aims of good government, unlike the proposal, which conflates transparency and quality without consideration of other factors.
As part of ensuring "objectivity" of information these guidelines encourage agencies which disseminate influential scientific, financial, or statistical information, "to include a high degree of transparency about data and methods to facilitate the reproducibility of such information by qualified third parties." However, they emphasize the need to treat certain data differently, due to privacy and confidentiality concerns. While they recommend agencies "identify the sources of the disseminated information" they note that this is "to the extent possible, consistent with confidentiality protections." Importantly, they take great pains to urge agencies not to subject all data to a reproducibility requirement where this could hamper agencies. They require agencies, instead, to consult with "the relevant scientific and technical communities" to identify data that "can practicable [sic] be subjected to a reproducibility requirement, given ethical, feasibility, or confidentiality constraints." There is no indication that EPA consulted with the scientific and technical community, with EPA's own SAB raising concerns about the proposal and finding that "[t]his action merits further review by the SAB." The Guidelines make clear:
Making the data and methods publicly available will assist in determining whether analytic results are reproducible. However, the objectivity standard does not override other compelling interests such as privacy, trade secrets, intellectual property, and other confidentiality protections.
In direct conflict with the reasoning underlying EPA's proposal, the Guidelines specifically provide that it is possible to verify the objectivity of information that cannot be made publicly available through other types of "robustness checks." As an example, they point to the Harvard Six Cities Study, where underlying data could not be made publicly available due to confidentiality concerns, but the raw data was released instead to researchers at the HEI, bound to the same confidentiality requirements as the original researchers, who were able to replicate its results. In contrast, EPA's proposal would not allow for the consideration of this study.
The guidelines also recommend agencies recognize that information quality comes at a cost, and that agencies should weigh the costs and benefits, which EPA has not done in the proposal.
Thus, the proposal completely turns away from OMB's guidelines where OMB "urges caution in the treatment of original and supporting data because it may often be impractical or even impermissible or unethical to apply the reproducibility standard to such data." As the comments discuss further, the proposal rule thus unlawfully conflicts with this flexible approach that prioritizes agencies' ability to use science as set out by OMB under the IQA.
Response: Transparency in addition to quality is important to EPA decision making. See the final rule preamble at Section III.A and Chapter 2 of this document for an explanation of the need for greater transparency.
Comment [8750-1837]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter states that the EPA claims the proposal is consistent with OMB's memorandum on Open Data Policy. This is incorrect, however, as the memorandum supports downstream information processing and dissemination -- not through complete public disclosure without regard to privacy or security -- but through instituting a framework of data collection, formatting, and storage that allows for public dissemination, if possible. Recognizing that not all data can be publicly disclosed, and that such data is still useful, the memorandum declares: "Whether or not particular information can be made public, agencies can apply this framework to all information resources to promote efficiency and produce value."
The proposal is thus inconsistent with the memorandum, which stresses the importance of information stewardship and "review of information for privacy, confidentiality, security, or other restrictions to release." When information cannot be released, the memorandum does not suggest agencies ignore the information or not rely on it for regulatory purposes. It focuses on prescribing agency practices to maximize the downstream usability of data that can be made publicly available, including through "using machine-readable and open formats, data standards, and common core and extensible metadata for all new information creation and collection efforts" as well as "building or modernizing information systems in a way that maximizes interoperability and information accessibility, maintains internal and external data asset inventories, enhances information safeguards, and clarifies information management responsibilities." Thus, while the memorandum centers on how agencies can marginally increase the utility of information they possess for use by the public, the proposal turns this on its head by advocating for discard of otherwise high quality scientific information if the data underlying such information cannot be made publicly available.
OMB stresses that to achieve "open data," agencies should adopt a presumption in favor of openness that is importantly limited by countervailing privacy, confidentiality, security, or other valid restrictions. Thus, agencies are expected to "exercise judgment before publicly distributing data residing in an existing system by weighing the value of openness against the cost of making those data public." The proposal does not at all weigh the costs, to the agency or to the public, of requiring all underlying data to be made publicly available.
While requiring agencies to adopt measures to strengthen privacy protections and data security, the memorandum recognizes serious limitation to data disclosure that EPA completely fails to consider. For example, the memorandum mandates that agencies take into consideration the "mosaic effect," which EPA does not at all acknowledge -- all while making superficial and unsupported statements about how privacy concerns can be easily addressed. The memorandum recognizes and stresses the challenge of responding to this threat, which requires undertaking a "risk-based analysis, often utilizing statistical methods whose parameters can change over time, depending on the nature of the information, the availability of other information, and the technology in place that could facilitate the process of identification." OMB importantly notes this analysis "may affect the amount, type, form, and detail of data released by agencies." According to the commenter, because it ignores these concerns, EPA's proposal is arbitrary and capricious.
Response: EPA notes that the final rule does not require EPA or any party outside of the agency to disclose protected information, nor does it require EPA to exclude studies based on data that cannot be disclosed due to its protected status. See response to Comment 6163-5030 and the final rule preamble at Section III.E.
 Comment [8750-1901]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter states that the EPA purports that the Proposal builds upon "the experience of other federal agencies in this space" but the citations reveal that is simply not the case. To support this statement, EPA provides only a hyperlink to a U.S. Census Bureau website along with vague references to entire executive branch agencies, with no explanation or discussion of which of their policies EPA believes the Proposal is building upon. Without a more specific citation, it is impossible to know which policies EPA is referencing or to respond to them meaningfully.
EPA cites to the U.S. Census Bureau's Federal Statistical Research Data Centers (FSRDC) as an example of use of secure facilities that allow the Census Bureau to provide controlled access to authorized researchers to use restricted-use microdata for statistical purposes only. In order to gain access, researchers must obtain Census Bureau Special Sworn Status by passing a moderate risk background check and swearing to protect respondent confidentiality for life. While this "solution" meets the U.S. Census Bureau's needs by allowing access to confidential information only to researchers whose proposals meet certain criteria, who go through a vetting process, and who agree to protect the information, this is done at a cost -- which EPA has not accounted for --  and would not satisfy EPA's requirement to make data and models "publicly available." Thus, according to the commenter, this example provides no support for the Proposal.
Response: See response to Comment 8750-1837.
Comment [8750-1905]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter cites footnote 10 of the proposal:
      These include policies and recommendations from: the Administrative Conference of the United States' Science in the Administrative Process Project; National Academies' reports on Improving Access to and Confidentiality of Research Data, Expanding Access to Research Data, and Access to Research Data in the 21st Century; the Health Effects Institute; Center for Open Science; members of the Risk Assessment Specialty Section of the Society of Toxicology, the Dose Response Section of the Society for Risk Analysis, and the International Society for Regulatory Toxicology and Pharmacology; and the Bipartisan Policy Center's Science for Policy Project. 
The commenter notes that in footnote 10 of the Proposal, EPA lists a number of organizations whose recommendations and policies the Proposal allegedly took into consideration. In fact, since the Proposal was published, many of these organizations have issued statements opposing the Proposal and contesting EPA's claim that their policies and recommendations endorse the Proposal. In this footnote, EPA provided no hyperlinks or specific citations for which recommendations and policies it was referencing, making it impossible to understand why EPA believed these organizations supported the Proposal or to respond to them. The commenter provides the following specific comments:
I. The Administrative Conference of the United States' Science in the Administrative Process Project
The EPA cites to the Administrative Conference of the United States' Science in the Administrative Process Project -- Recommendation 2013-3: Science in the Administrative Process. Wendy Wagner, sole author of ACUS's final report Science in Regulation: A Study of Agency Decision-Making Approaches and who served on the panel that produced the recommendations strongly opposed the notion that the Proposal builds upon these recommendations, saying: "They don't adopt any of our recommendations, and they go in a direction that's completely opposite, completely different. . . . They don't adopt any of the recommendations of any of the sources they cite. I'm not sure why they cited them."
While ACUS recommends agencies increase transparency of how they rely on scientific information and strive to make data underlying scientific information publicly available, nowhere do they suggest that agencies should not consider or rely on studies where underlying data and models cannot be made publicly available, or that these circumstances make scientific information less valid. They instead suggest that information be made publicly available for assessment and reproducibility purposes "[c]onsistent with the limitations in the Information Quality Act (IQA) guidelines issued by the Office of Management and Budget and its own IQA guidelines." They acknowledge valid limitations such as legal protections for privacy, trade secrets, and CBI. Thus, they recommend data be made public only "[t]o the extent practicable and permitted by law and applicable policies." Unlike the Proposal, the recommendation acknowledges that agencies may still use information where underlying data cannot be publicly disclosed, and suggest agencies "note that fact and explain why they used the results if they chose to do so." It thus provides a much more nuanced policy recommendation than that outlined in the Proposal -- which suggests EPA either find a way to make underlying data and models public, despite the numerous potential obstacles and concerns in doing so, or completely disregard the research study.
II. National Academies Improving Access to and Confidentiality of Research Data
Rather than containing any particular recommendations or policy proposals, this report discusses a number of issues pertaining to data disclosure and privacy protection, the tradeoffs "between increasing data access on the one hand and improving data security and confidentiality on the other," and "alternative approaches to limiting disclosure risk while facilitating data access the benefits and limitation of various approaches to these issues." Thus, rather than calling on agencies to rely only on scientific studies where the underlying data and models are made public, the report in fact discusses challenges and obstacles to achieving greater data disclosure, for which the Proposal provides no substantive or meaningful explanation.
The report discusses why exercising caution with respect to disclosing confidential personal information is so important, because if such information is exposed it could lead to
being arrested for a crime, being denied eligibility for welfare or Medicaid, being charged with tax evasion, losing a job or an election, failing to qualify for a mortgage, or having trouble getting into college. Disclosure of a history of alcoholism, mental illness, venereal disease, or illegitimacy can result in embarrassment and loss of reputation. Less directly, research results based on personal data can cause harm by affecting perceptions about a group to which a person belongs.
The report reveals very legitimate reasons why researchers and study participants would be reluctant to allow underlying data to be made publicly available -- and these reasons in no way compromise the validity of the scientific conclusions based upon this data.
The report also discusses the nuances of selecting methods to protect privacy while making underlying data publicly available. For example, while EPA casually makes claims that controlled access is an example of a solution in place across federal agencies -- this report points out the drawbacks of such an approach:
The use of restricted access arrangements, which has been deemed necessary to provide adequate protection for confidential information about individuals and businesses, results in increased costs to conduct research. Custodians of the data files need additional resources to process applications, operate inspection systems, staff research data centers, and inspect outputs to ensure that disclosure does not occur. Researchers require resources to prepare applications for access, to provide appropriate physical security for the data, or to visit a secure site.
The report also discusses the difficulty of funding such centers -- noting that while the costs are currently covered by a combination of federal agency budgets and user fees, including grants from the NSF and National Institute on Aging, federal funding may no longer be able to support such efforts. EPA's cursory mention to use of restricted access facilities as a potential solution to the concerns implicated by the Proposal fail to mention or address any of these challenges.
III. National Academies Expanding Access to Research Data: Reconciling Risks and Opportunities 
EPA's Proposal in no way takes into consideration the recommendations of the National Academies report Expanding Access to Research Data: Reconciling Risks and Opportunities. This report considers competing approaches to increase use of research data while protecting confidentiality, and concludes that "no one way is optimal for all data users or all purposes" and, importantly, that "the nation's statistical and research agencies must provide both unrestricted access to anonymized public-use files and restricted access to detailed, individually identifiable confidential data for researchers under carefully specified conditions." In other words, the report finds that making data publicly available without restriction while respecting confidentiality concerns is not currently feasible or compatible with the missions of federal agencies.
Furthermore, the report mainly concerns itself with how agencies might increase access to data in their control and possession to allow for more research in social issues and provide a better basis for more informed policy decisions -- it does not discuss whether federal agencies should make data publicly available in order to allow for independent validation of scientific research they rely on for regulatory purposes and thus cannot be a basis for the Proposal. While the report discusses that one of the benefits of data sharing is that it allows for "verification, refutation, or refinement of original results," nowhere does the report suggest that agencies should rely only on research studies that make data publicly available or that such verification is necessary to validate a research study. Indeed, it details a discussion on this topic that presents competing views on requirements to make research data available to the public to allow for replication. John Bailar raised concerns that researchers would be deterred from doing certain kinds of work if they feared it would be subject to "hostile scrutiny" and that competitors could seize data for their interests. Others disagreed with this position. However, EPA failed to engage any of these considerations or at all justify its decision to implement a policy that could have severe negative implications. None of the researchers stated agencies should disregard the study if underlying data could not be made public.
The "recommendations" made by the report do not endorse EPA's proposal. The report provides 15 recommendations in Chapter 5. Recommendations 1-4 concern documentation and data access and call on agencies to better document how the data they make available is used; to use a variety of modes to provide access to data they produce or fund using a combination of restricted access to confidential data and unrestricted access to appropriately altered public-use data; to support research to guide more efficient allocation of resources among different data access modes; and to involve users in planning modes of access to their data.
In this Proposal, EPA does nothing to better document use of data that it makes public, has only called for a requirement to make research data and models "publicly available" rather than recognizing that a variety of modes and levels of access may be necessary, and does nothing to support more research into methods of making data more widely available without compromising confidentiality -- indeed blithely assuming that such means are already available and sufficient -- and also has not indicated that there has been any widespread call for EPA to make such data available or pointed to any comments of users of this data in this process.
Recommendations 5-8 concern public use data and call on agencies to support research on techniques to provide useful innovative public-use data that minimizes the risk of disclosure; streamlined procedures to allow researchers access to public-use microdata through existing and new data archives; a warning on all public-use data that they are provided for statistical purposes only and that any attempt to identify an individual is a violation, and requiring users to attest to having read the warning; and restricting access to public-use data to those who agree to abide by confidentiality protections, subject to meaningful penalties.
EPA's proposal once again ignores these recommendations that call for greater research and a measured approach to making data more widely available. The Proposal provides no ideas or methods or support for research that would help strengthen confidentiality protections while making data more available.
Recommendations 9-13 concern research data centers, remote access, and licensing agreements and call on the Census Bureau to (1) broaden the interpretation of the criteria for assessing the benefits of access to data; (2) maintain the continuous review cycle; and (3) take account of prior scientific review of research proposals by established peer review processes when awarding access to research data centers; for more research on cost effective means of providing secure access to confidential data by remote access; increasing use of licensing agreements for access to confidential data; working with data users to develop flexible, consistent standards for licensing agreements and implementation procedures for access to confidential data; and including auditing procedures and legal penalties in licensing agreements for willful misuse of confidential data.
EPA's proposal does not increase any research into use of remote data centers or licensing agreements, simply making passing references to these modes as potential solutions with no discussion or explanation -- and ignoring the recommendations here suggesting that more work is needed to realize their potential.
Recommendations 14-15 concern maintaining the public's trust and call on agencies to give certain basic information about confidentiality and data access to everyone asked to participate in statistical surveys; and to support continuing research on the views of data providers and the public about research benefits and risks.
EPA's proposal does not involve anything that increases the public knowledge about confidentiality protections or their views on research benefits and risks.
Recommendations 16-19 concern training, monitoring, and education to complement other protections on data. They call on data collection agencies to provide employees with continually updated written guidelines on confidentiality protection and training in confidentiality practices and data management and to institute procedures for monitoring violations of confidentiality protections practices and confidentiality breaches. They also call on educational and professional organizations to provide training in ethical issues for all those involved in the design, collection, distribution, and use of data obtained under pledges of confidentiality and for the development of strong codes of ethical conduct that reflect the need to protection confidentiality.
EPA's proposal also contains no provisions on increasing training, monitoring, or education, within the agency or among researchers to allow for more careful handling of confidential data.
Thus, EPA's Proposal completely ignores the careful research and thinking the National Academies and researchers have done on what is needed from federal agencies in order to make data more publicly available, and how to do so in a responsible manner. It does not implement any of the recommendations in the report, and in no way builds upon this work.
IV. National Academies Access to Research Data in the 21st Century: An Ongoing Dialogue Among Interested Parties: Report of Workshop
EPA cites to the National Academies' Access to Research Data in the 21st Century: An Ongoing Dialogue Among Interested Parties: Report of Workshop as one for which it took into consideration "policies or recommendations," despite the fact that this report comes with the explicit limitation that:
The goal of the workshop was not to reach conclusions or recommendations; nor could it address other pressing issues beyond the regulatory process, such as protection of intellectual property, the influence of broader access on scientific competition, the potential for increased administrative burdens and changes in the research process, and the challenge of providing data access in an increasingly electronic world.
Thus, this report stresses the many unanswered, challenging policy questions that must be addressed as agencies contemplate how to make data publicly available. These are the questions EPA should have addressed in its Proposal but did not.
The Report offers a look into the scientific review process that also calls into question the underlying assumption in EPA's proposal -- that making data publicly available is necessary to ensure the validity of a scientific finding. The report notes that scientific claims "are not `binary'" they instead "fall in the category of being uncertain to various degrees." The reliability of a particular scientific finding can be assessed using various mechanisms, starting with an examination of the strength of the design, methods, and statistical results. Then "one asks whether there is consistency within the data (pertaining to mechanisms of effect or related outcomes) and with other studies and scientific theories." Finally, "the robustness of the findings is evaluated through the use of different analytical approaches."
The report describes how studies may be validated through a range of approaches. While it notes that in some cases it is possible to exactly replicate the original study, this is not always the case, especially in large epidemiological studies where "repeating a study is seldom either possible or desirable." Then "replication" can take a variety of forms, not all of which require access to underlying data, including:
 Additional analyses done on the data set by the original or collaborating Investigators;
 New results generated from older data sets;
 New studies addressing the same hypothesis;
 Independent analysis of the same data set by different people;
 Monitoring of the results of actions taken on the basis of the findings.
Another form of replication the report describes is meta-analysis, which is a systematic strategy for comprehensively describing and summarizing a body of research evidence from two or more studies. The goal is to produce a quantitative synthesis of the evidence presented in multiple studies that relate to a research question. In a typical meta-analysis, all the data used have been published in the public domain and are easy to inspect and analyze.
The report specifically mentions the Harvard Six Cities Study as an example of a study where data could not be made publicly available, but which was verified to allow the agency to justifiably rely on it to set important air standards. Thus, unlike the Proposal the report acknowledges the many different pathways that exist to for researchers to assess other studies, and does not suggest that allowing the general public access to underlying data and models is necessary.
One of the panels of the workshop discussed the Shelby Amendment, and public access to data underlying agency regulation. A bench scientist expressed concerns that, though the idea of sharing data was a good idea, because any person could request information for any reason, this mechanism could be used to harass scientists whose work was found objectionable. A representative of NIH similarly stated that while sharing data with other researchers was good scientific practice, allowing for indiscriminate public access to data serves "little purpose for those without the skills to reanalyze it." Additionally, access through FOIA does not allow for limitations to be put on the use of the data, which is typically available in other data-sharing modes. A representative from EPA raised issues including:
The Shelby Amendment. . . raises several questions for the EPA about rulemaking as a legal and deliberative process. At what point should the agency disclose what type of regulation is going to be considered or issued? The timing of the release can influence its reception. Should the agency use contracts to support the research needed for regulations? Contracting, as opposed to grants that support more flexible work, might narrow the type of information the agency receives and could possibly limit the scope of the science underlying the regulation.
These questions and concerns are highly relevant to the Proposal as well, yet EPA provides no indication that it has given them any consideration.
Finally, a representative from NRDC pointed to other mechanisms that are already in place to ensure agencies rely on high quality data. For example, under the APA, agencies must respond to any comments that raise questions about a scientific studies design, performance, or conclusion. Courts can determine whether an agency was reasonable in its decision to refuse to accept the findings of a study because it could not access underlying data or refuses a request from a study participant. EPA does not explain why these existing mechanisms are not sufficient to ensure the integrity of the science it relies on.
V. The HEI
In the original Federal Register notice, EPA provided no specificity as to which Health Effects policy EPA was referring to or why it supported the Proposal. Such a vague and unspecified reference does not meet the notice requirements of the APA and other statutes and makes it impossible to respond.
VI. COS
In the original Federal Register notice, EPA provided no specificity as to which COS policy EPA was referring to or why it supported the Proposal. Such a vague and unspecified reference does not meet the notice requirements of the APA and other statutes and makes it impossible to respond.
VII. Members of the Risk Assessment Specialty Section of the Society of Toxicology (SOT), the Dose Response Section of the Society for Risk Analysis (SRA), and the International Society for Regulatory Toxicology and Pharmacology (ISRTP)
In the original Federal Register notice, EPA provided no specificity as to which policy of the Members of the Risk Assessment Specialty Section of the SOT, the Dose Response Section of the SRA, and the ISRTP EPA was referring to or why it supported the Proposal. Such a vague and unspecified reference does not meet the notice requirements of the APA and other statutes and makes it impossible to respond.
VIII. Bipartisan Policy Center's Science for Policy Project
In the original Federal Register notice, EPA provided no specificity as to which Bipartisan Policy Center's Science for Policy Project policy EPA was referring to or why it supported the Proposal. Such a vague and unspecified reference does not meet the notice requirements of the APA and other statutes and makes it impossible to respond.
Response: Regarding the original proposal, see response to Comment 2157-149. 
Comment [8750-1908]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter cites footnote 11 of the proposal:
      For example, see related policies from the Proceedings of the National Academy of Sciences, PLOS ONE, Science, and Nature.
The commenter states that the EPA claims that the Proposal takes into consideration policies adopted by scientific journals, but does not specify which "related policies" from these journals. While some of these journals have adopted certain policies encouraging or requiring researchers to share underlying data for the studies they publish, they all allow for exceptions when data cannot be released for compelling reasons, such as confidentiality protections.
Furthermore, the editors of these journals have issued a joint statement opposing the Proposal and noting that their policies do not endorse such an approach by EPA. They note that some data sets cannot be shared publicly, and that there are still other methods available to verify scientific findings. The statement also strongly condemns the notion of excluding scientific information from consideration when underlying data cannot be made publicly available:
It does not strengthen policies based on scientific evidence to limit the scientific evidence that can inform them; rather, it is paramount that the full suite of relevant science vetted through peer review, which includes ever more rigorous features, inform the landscape of decision making. Excluding relevant studies simply because they do not meet rigid transparency standards will adversely affect decision-making processes.
Thus, EPA cannot claim that the Proposal is in any way supported by the data sharing policies of these scientific journals.
Response: See response to Comment 4829-2022.
Comment [8750-1911]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter cites footnote 12 of the proposal:
      See: https://www.nature.com/articles/s41562- 016-0021; http://journals.plos.org/plosmedicine/ article?id=10.1371/journal.pmed.0020124; http://science.sciencemag.org/content/343/6168/229.long; https://www.economist.com/news/leaders/21588069-scientific-research-has-changedworld-now-it-needs-change-itself-how-science-goes-wrong.; http://stm.sciencemag.org/content/8/341/341ps12.full. 
The commenter states that the EPA claims that the Proposal is informed by the policies of scientific journals in response to the "replication crisis." EPA provides no explanation or evidence to support the fact that such a "crisis" is occurring or that EPA's Proposal would do anything to address the crisis. The sources EPA cites for this proposition speak to a concern about scientific studies being reproducible or replicable due to a number of different conditions related to poor scientific practices. While some of the articles speak about making data more available as an ideal to aspire to, none of them support the idea that a research study whose underlying data has not been made publicly available should, for that reason alone, be considered invalid. Further, many of these articles speak to how current scientific norms do not result in underlying data being available, which is a huge barrier to EPA's Proposal that EPA does not at all address. Specific commenter provided by the commenter include the following:
I. Marcus R. Munafó et. al, A Manifesto for Reproducible Science, 1 Nature Human Behavior 1 (2017)
Far from suggesting that agencies rely only on scientific studies if the underlying data is made public, or even that making underlying data public is necessary to ensure validity of scientific conclusions, the article discusses at a high level a number of systemic and cultural challenges to reproducible science. By ignoring the nuances of this article and presenting it without any explanation as support for its Proposal, EPA runs into the problem the article specifically cautions against, warning: "Some solutions may be ineffective or even harmful to the efficiency and reliability of science, even if conceptually they appear sensible."
This article does not endorse the existence of a "replication crisis" and in fact says, "[w]hether `crisis' is the appropriate term to describe the current state or trajectory of science is debatable." Instead it notes a very different problem than the one EPA appears to target with the Proposal. It points broadly to an issue of there being "substantial room for improvement with regard to research practices to maximize the efficiency of the research community's use of the public's financial investment in research."
This article makes clear that open data requirements are just one of many solutions and steps to take towards increasing efficiency of use of resources and robustness of scientific findings -- and never suggests that a lack of publicly available underlying data should automatically disqualify a research finding from consideration. It discusses a number of other improvements including protecting against cognitive biases through blinding, improving methodological training, implementing methodological support, encouraging collaboration and team science, promoting study pre-registration, improving quality of reporting, diversifying peer review, and changing incentives to promote efficient and effective research instead of just innovative outcomes.
While the article recognizes transparency as a "scientific ideal" it notes many challenges that currently exist to achieving this ideal, which EPA does not at all address. The article notes, "In reality, science often lacks openness: many published articles are not available to people without a personal or institutional subscription, and most data, materials and code supporting research outcomes are not made accessible, for example, in a public repository." It further finds "substantial barriers to meeting these ideals, including vested financial interests (particularly in scholarly publishing) and few incentives for researchers to pursue open practices." Nowhere does the article suggest that the many scientific studies for which data is not available due to prevailing scientific norms and practices be completely discarded. These challenges suggest that many studies EPA wishes to rely on may not be able to meet the rigid requirements of EPA's proposal severely restricting the science EPA can use, degrading the quality of its decision-making.
Marcus R. Munafó, lead author on this paper, has since published a piece specifically dismissing science policy approaches that overemphasize the importance of replication. It states that the overemphasis on replicability is detrimental to science -- that "[i]f a study is skewed and replications recapitulate that approach, findings will be consistently incorrect or biased." Instead, the author suggests that "an essential protection against flawed ideas is triangulation" or "the strategic use of multiple approaches to address one question." This involves looking at a broad base of different scientific studies and does not require underlying data to be made publicly available, not individual studies based on whether or not they can be replicated. By excluding scientific studies from EPA's consideration, the Proposal overemphasizes the value of replication to the detriment of being able to evaluate a study in the context of many other studies examining the same issue through a variety of methods. The Proposal may well lead to reliance on less robust science and is thus arbitrary.
II. John P.A. Ioannidis, Why Most Published Research is False, 2 PLoS Medicine 0696 (2005)
The article suggests "the high rate of non-replication (lack of confirmation) of research discoveries is a consequence of the convenient, yet ill-founded strategy of claiming conclusive research findings solely on the basis of a single study assessed by formal statistical significance, typically for a p-value less than 0.05." It looks at a number of different contributors to false positive findings and discusses solutions to this problem. Importantly, it stresses the need to focus on large studies, consider the totality of the evidence, and improve understanding of pre-study odds. These solutions each involve considering more evidence and more scientific studies to contextualize any one given study. Nowhere does the article suggests requiring underlying data be made public or fewer studies be considered. EPA's proposal contrarily emphasizes data disclosure above all other practices for ensuring scientific integrity -- and will result in fewer studies being considered to shed light on the scientific truth.
The author of this article has specifically criticized EPA's Proposal, saying that, if it is finalized, "science will be practically eliminated from all decision-making processes" and "[r]egulation would then depend uniquely on opinion and whim." The author highlights the inherent problem in EPA's Proposal, that "most of the raw data from past studies are not publicly available" and that indeed "[i]n a random sample of the biomedical literature (2000 - 2014) none of 268 papers shared all of their raw data. . . [and] [o]nly one shared a full research protocol." EPA has not addressed this major issue that suggests the Proposal would bar EPA from relying on massive amounts of scientific research. The article notes that reproducibility issues vary across the disciplines and that in many areas in which EPA operates, a solid and large foundation of scientific research has produced credible and widely-affirmed findings, including "in fields such as air pollution and climate change." Even in these other fields, however, it firmly states that "simply ignoring science that has not yet attained such standards, is a nightmare."
III. Marcia McNutt, Reproducibility, 343 Science 229 (2014)
EPA cites an announcement by Science that, in response to reports "that a troubling proportion of peer-reviewed preclinical studies are not reproducible," Science is adopting new policies requiring authors making submissions to the journal to disclose "whether there was a pre-experimental plan for data handling (such as how to deal with outliers), whether they conducted a sample size estimation to ensure a sufficient signal-to-noise ratio, whether samples were treated randomly, and whether the experimenter was blind to the conduct of the experiment." While the article considers steps to increase reproducibility of science, it notes that data availability is not a necessary or sufficient step to ensure credibility of research findings, and that "ultimate responsibility lies with authors to be completely open with their methods, all of their findings, and the possible pitfalls that could invalidate their conclusions." EPA's Proposal ignores the ability to assess studies through these other important indicators to assure their validity.
VI. How Science Goes Wrong, Economist (Oct. 21, 2013)
This article opposes the view that verification of a study depends solely on the underlying data being made publicly available. While it identifies that much scientific research is unable to be replicated, the solution it proposes include tightening standards, particularly in statistics, registering research protocols in advance and monitoring them, and: "[w]here possible, trial data also should be open for other researchers to inspect and test." Thus, even to the extent it discusses data availability, it suggests data should be open for other researchers, as opposed to the public, and recognizes this may not always be possible.
VII. Steve N. Goodman, What does research reproducibility mean?, 8 Science Translational Medicine 1 (2016)
Rather than saying anything about agencies relying only on scientific studies where underlying data is made public, this article discusses the importance of clearly defining key terms in the discussion about scientific reproducibility, noting that there is a lack of standardized definitions of terms such as "reproducibility, replicability, reliability, robustness, and generalizability." This raises a key issue of vagueness in EPA's proposal -- EPA does not provide definition for key terms such as "independently validate" or "reproducible" and confusing mentions a "replication crisis" while citing to articles that speak to a "reproducibility crisis."
While providing definitions for these various terms, the article notes that there terms all represent various methods of attempting to verify studies to ensure "scientific claims based on scientific results are true" and cautions against "treating reproducibility as an end in itself --  rather than as an imperfect surrogate for scientific truth." Instead, it promoted the view of looking across studies to "assess their cumulative evidential weight." EPA Proposal thus directly contradicts the suggestions of this article.
Response: See the final rule preamble at Section III.A and Chapter 2 of this document.
Comment [8750-1912]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter cites footnote 13 of the proposal:
      EPA has not consistently followed previous EPA policy (e.g, EPA's Scientific Integrity Guidance, referenced above) that encouraged the use of non-proprietary data and models. 
The commenter states that while EPA in a footnote suggests that EPA has not consistently followed EPA's Scientific Integrity Policy encouraging the use of non-proprietary data and models, it misses the fact that EPA's policy was not written as an absolute standard, but was intended to be a flexible one. The policy states only that "the use of non-proprietary data and models are encouraged, when feasible, to increase transparency." EPA must thus explain and justify its deviation from its prior flexible approach that the Proposal now imposes.
Response: See the final rule preamble at Section III.A and Chapter 2 of this document for an explanation of the change in policy and the need for greater transparency.
Comment [8750-1913]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter cites footnote 14 of the proposal:
      https://www.whitehouse.gov/wp-content/uploads/2017/11/2005-M-05-03- Issuance-of-OMBs-Final-Information-Quality-Bulletin-for-Peer-Review-December-16- 2004.pdf
The commenter states that the Proposal appears to issue a requirement for independent peer review of all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review. EPA cites to OMB's Final Information Quality Bulletin for Peer Review, explaining existing peer review requirements that nowhere does EPA suggest are not already being complied with.
According to the commenter, there is some vagueness as to whether the Proposal maintains, expands, or narrows these already existing requirements. OMB's bulletin underwent a rigorous stakeholder process including response to comments on multiple drafts from stakeholders, a federal agency workshop at NAS, outreach to major scientific organizations and societies, a formal interagency review. EPA's Proposal has not gone through nearly the same level of review, or as our comments detail, even met the minimum legal requirements for consultation and review. OMB's guidance further provides that agencies should consider the "tradeoffs between depth of peer review and timeliness" This includes considering a benefit cost framework for peer review that takes into account "the direct costs of the peer review activity and those stemming from potential delay in government and private actions that can result from peer review." As our comments detail, EPA has not provided any meaningful benefit-cost analysis of the Proposal. Thus, it would be improper and in conflict with OMB's guidance for EPA to be expanding the peer review requirements through this Proposal.
Response: See the final rule preamble at Section III.H and Chapter 13 of this document.
Comment [8750-1914]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter cites footnote 15 of the proposal:
      February 22, 2002 (67 FR 8453) OMB's Guidelines Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information (2002) https://www.federalregister.gov/documents/2002/02/22/R2-59/guidelines-for-ensuring-andmaximizing-the-quality-objectivity-utility-and-integrity-of-information. 
The commenter states that as discussed on their comments regarding Proposal Federal Regiser notice footnote 6, EPA's attempt to align its proposal with OMB's guidelines is misguided.
Response: See Section III.A of the final rule preamble and Chapter 2 of this document.
Comment [8750-1916]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter cites footnote 16 of the proposal:
      See examples from the U.S. Department of Health and Human Services, National Institute of Standards and Technology, U.S. Department of Education, and the U.S. Census Bureau. 
The commenter states that in the original Proposal EPA provided no specific "examples" and this vague cite provided very little direction about what EPA was referencing here -- making it impossible to review these examples or respond to them.
Response: See Section III.A of the final rule preamble and Chapter 2 of this document.
Comment [8750-1917]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter cites footnote 17 of the proposal:
      https://www.hhs.gov/hipaa/for-professionals/privacy/special-topics/deidentification/index.html. 
The commenter states that the EPA states that other agencies have tools to de-identify information private information but fails to recognize that these methods are not transferable to EPA's context. EPA links to guidance on de-identification requirements under HIPAA. This guidance provides two methods for de-identifying data: (1) expert determination method, where an expert determines that, after application of statistical and scientific principals and methods, the risk is very small that the information alone or with other available information could be used to identify the subject; and (2) the safe harbor method, requiring that a number of identifiers are removed. The first method requires case-by-case work and EPA has provided no information regarding how EPA could implement it or how much it might cost and thus the feasibility of requiring researchers or EPA to de-identify data this way is questionable. The second method requires removal of much information useful for research that may be necessary to be able to independently validate the research, so it is unclear that it would satisfy the Proposal's demands. Furthermore, the safe harbor method has been shown to provide potentially insufficient privacy protections.
Response: See Section III.A of the final rule preamble and Chapter 2 of this document.
Comment [8750-1918]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter cites footnote 18 of the proposal:
      https://www.nap.edu/catalog/11434/expanding-access-to-research-datareconciling-risks-and-opportunities. 
The commenter states that in this footnote, EPA cites to a report by the National Academies for the proposition that "The National Academies have noted that simple data masking, coding, and de-identification techniques have been developed over the last half century. . . " This incorrectly makes it seem as though the National Academies have identified simple techniques to de-identify data for public release without compromising personal piracy. A full review of the report reveals the opposite is true, that The National Academies in fact recognize that complex, evolving, and yet undeveloped techniques are needed to resolve these concerns. It offers recommendations that are intended to improve upon existing techniques, indicating that this area is under constant change and many advances are left to be made. Further, the report notes this improvement requires "strong partnership between the research community and statistical and research agencies in the design of innovative research on disclosure avoidance techniques and data access modalities and in the implementation of the advances that result from such research." The Proposal takes no steps towards advancing design of new techniques or providing resources to undertake all that needs to be done to make the Proposal remotely feasible.
Further, the Report notes that a changing landscape is making it increasingly difficult to apply past techniques to sufficiently protect data from identification, saying: "Initially, relatively simple data masking techniques, such as top coding income amounts. . . were used to generate restricted data products [,] [d]uring the last decade the increasing risks of confidentiality breaches have led researchers to develop increasingly sophisticated methodologies for restricted data products." They state, "more research is clearly needed to assess the relative ability of different masking methods, and of synthetic data, to reduce the risk of disclosure while preserving data utility." EPA does not acknowledge these newly emerging concerns.
The National Academies recognize the current limitations of producing restricted data that sufficiently limits identifiability to allow it to be made publicly available in a useful form. They note that "well-informed policy making" requires "[r]esearch using detailed confidential data" that cannot be made public -- which the Proposal fails to acknowledge to the detriment of the quality of EPA's policy decisions. Just because certain information cannot be made public for legitimate reasons does not mean the government should refuse to use it to inform policy. And much of the data useful for environmental and health research is particularly sensitive -- the report notes there is increased vulnerability in "[d]ata with geographic detail, such as census block data" and longitudinal data obtained in panel surveys, which is often salient in environmental research. In the meantime, the National Academies state that more work is needed to allow "[h]igh-quality public-use files" that still assure "the inferential validity of the data while safeguarding their confidentiality."
They also point to broader implications of not implementing sufficient privacy protections that EPA does not consider at all may result from the Proposal. The quality of data collected is likely to suffer as "[i]t is essential that respondents believe they can provide accurate, complete information without any fear that the information will be disclosed inappropriately." Essentially, the report leaves as an open question "decisions about how much disclosure risk is acceptable in order to achieve the benefits of greater access to research data involve weighing the potential harm posed by disclosure against the benefits potentially foregone." Thus, EPA wrongfully points to this report as supporting the notion that simple techniques exist to address privacy concerns. The report recommends only more research to reduce risks and increase data utility along with consultation with data users and providers about these issues -- which the Proposal does not implement and thus the report does not support the Proposal.
Response: See Section III.A of the final rule preamble and Chapter 2 of this document
Comment [8750-1919]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter cites footnote 19 of the proposal:
      https://www.cep.gov/content/dam/cep/report/cep-final-report.pdf; https://www.nap.edu/catalog/24652/innovations-in-federal-statistics-combining-datasources-while-protecting-privacy; https://www.nap.edu/catalog/24893/federal-statisticsmultipledata-sources-and-privacy-protection-next-steps. 
The commenter states that the EPA claims that "the National Academies and the Bipartisan Commission on Evidence Based Policy have discussed the challenges and opportunities for facilitating to secure access to confidential data for non-government analysts." The proposal does not explain how these examples are relevant, as there is no indication that secure access to underlying data would meet the requirements of making underlying data "publicly available." Further, even if it were relevant, a review of the sources cited reveal that they do discuss many challenges in this space -- which the Proposal does not at all address -- and provide no support for the Proposal. The commenter specific comments follow:
I. Commission on Evidence-Based Policymaking, The Promise of Evidence-Based Policymaking (2017)
This report centers on how to enhance infrastructure to increase the access and use of data between federal agencies to support government policy-making, rather than increase public access to data to non-governmental analysts for purposes of independently validating regulatory science. Further, its focus is to help efforts to make more data available for government purposes to better inform policies. The Proposal on the other hand seeks to make data available to validate individual studies while ultimately making less data available for EPA to consider as it creates policies.
To the extent the report does speak to making more data publicly available, it envisions an entirely new framework to provide adequate privacy protections. Chapter Three of the report discusses increasing threats to privacy as "the amount of information about individuals that is publicly available has grown and the technology that can permit unauthorized re-identification has improved." It notes that forming solutions to this problem while preserving the quality of data is difficult, and that a challenge is "ensuring that enhanced statistical disclosure methods do not change the data in ways that increase the difficulty of reproducing research results." It thus specifically notes that protecting confidentiality can be in tension with allowing data to be used for reproducibility purposes.
The report recommends: (1) amending federal statutes to require Federal departments to conduct a comprehensive risk assessment on de-identified confidential data intended for public release and release de-identified confidential data subject to the Privacy Act and Confidential Information Protection and Statistical Efficiency Act (CIPSEA) only after a disclosure review board approves the release and publicly provides the risk assessment and a description of steps taken to mitigate risk; (2) federal departments to adopt state-of-the-art database, cryptography, privacy-preserving, and privacy-enhancing technologies for confidential data used for evidence building; (3) federal departments assign a senior official the responsibility for coordinating access to and stewardship of the department's data resources; (4) new legislation ensuring that data acquired under a pledge of confidentiality are kept confidential and used exclusively for statistical purposes. The Proposal does not discuss or contribute to any of these efforts.
Chapter Four recognizes that some data cannot be made publicly available without sacrificing the utility of the evidence and thus sets forth recommendations for creating a new National Secure Database Service to allow researchers to access "detailed data that cannot be made publicly available, and only for exclusively statistical purposes." This report thus implicitly recognizes the value of using confidential data to "securely generate evidence about government policies and programs." While transparency is a crucial goal, using data that cannot be made publicly available can help inform government policies in important ways.
The Report details the many obstacles to making data publicly available, and ultimately concludes that much more work is needed in this area, none of which is being furthered by EPA's Proposal.
II. NAS, Innovations in Federal Statistics: Combining Data Sources While Protecting Privacy (2017)
This report provides recommendations to increase sharing and use of data by the federal government and between agencies. It places maintaining privacy and confidentiality at the forefront. The report provides a discussion of the benefits and challenges to allowing external researchers to access data held by government agencies. This assumes that agency has access to data in the first place -- which may not be the case with the studies EPA wishes to rely on that would be barred by its Proposal.
The report notes multiple risks to privacy and confidentiality from data breaches, identity theft, and the threat from the ability to combine multiple data sources to re-identify anonymized data as more and more data is made publicly available. The solutions that the report proposes to minimize these risks include: data minimization, restricted data, restricted access (including licensing agreements, FSRDCs, nongovernment data enclaves). The Proposal does not allow for data minimization since it is aimed at making public complete underlying data that is likely to involve salient personally identifiable information for an unlimited amount of time. Data restriction involves "removing explicit identifiers and applying a variety of statistical disclosure limitation methods to the dataset to reduce the risk of disclosure." However, because these techniques "decrease the precision of the variables in the dataset and. . . introduce errors" it is unclear that they would preserve data for independent validation while also sufficiently protecting privacy. Restricted access involves using "administrative procedures and technology to restrict who can access the dataset and what kinds of analyses can be done with the data to reduce the risk of disclosure." This specifically limits access to data from the general public, which seemingly would not meet the requirements of EPA's proposal. Thus, EPA has not addressed how it would meet any of the challenges raised in this document.
III. NAS, Federal Statistics, Multiple Data Sources, and Privacy Protection: Next Steps (2017) 
This report is not directly relevant as it discusses ways to combine diverse data sources from government and private sector sources and the privacy issues that arise from combining multiple data sets. The purpose of the report is to help "federal statistical agencies examine and evaluate data from alternative sources and then combine them as appropriate to provide the country with more timely, actionable, and useful information for policy makers, businesses, and individuals." EPA's proposal will in fact restrict the information that EPA can use.
The report notes that the "privacy status of data is dynamic over time, that datasets that are not individually identifiable today may in the future become individually identifiable" with the availability of new techniques and auxiliary data. It notes that as data sets are linked, these privacy threats increase. The Proposal does not discuss or address threats to privacy from data linkages.
The panel highlighted a number of threats to privacy and data security, including from security threats and inferential disclosure, and concluded "there is awareness of weaknesses of current statistical disclosure limitation methods, but the feasibility for federal statistical agencies of implementing new technologies, such as differential privacy, has not been clearly demonstrated." Finally, they state:
      Overall, much work, interaction, and collaboration will be needed across the various disciplines and stakeholders as agencies seek to move forward to provide stronger privacy protection for the data they either collect from respondents or acquire access to from other administrative and private-sector sources for statistical purposes. It will be critical for there to be robust discussions of the implications of this approach for all stakeholders and these discussions will need to be informed by concrete examples to help everyone understand how use of these technologies will affect them.
The report notes that in order to provide greater access to data much more research and resources are needed. The Proposal identifies no such resources or processes needed to develop needed methods and techniques to allow for greater data disclosure.
Response: See Section III.A of the final rule preamble and Chapter 2 of this document. EPA notes that this final rule does not require EPA to categorically exclude any studies. See response to Comment 4829-2022.
Comment [8750-1920]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter cites footnote 20 of the proposal:
      For example, see policies or recommendations of publishers Taylor & Francis, Elsevier, PLOS, and Springer Nature.
The commenter states that the EPA cites to "policies or recommendation" of several journals that require data be deposited in public data repositories as an example of the Proposal's requirement of data availability. EPA provided only a list of journals with no reference to any specific policies making it difficult to respond fully to this statement.
Each of these journals, however, has exceptions to its data availability requirements when there are valid reasons preventing authors from making their data publicly available via a public data repository. Further, the editors of these journals released a joint statement that explains why their policies with regards to data availability should not be used to support a policy by a federal agency that would in fact restrict the scientific studies it could rely on. Given the vastly different contexts and aims of federal agencies and scientific journals when it comes to making data publicly available, journal policies should not inform EPA's direction. None of these journals claims that lack of data availability in itself calls into question the validity of a scientific conclusion based on that data -- and thus these policies do not support the Proposal.
Response: See Section III.A of the final rule preamble and Chapter 2 of this document.
Comment [8750-1921]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter cites footnote 21 of the proposal:
      For example: https://osp.od.nih.gov/scientific-sharing/requesting-access-tocontrolled-access-data-maintained-in-nih-designated-data-repositories-e-g-dbgap/; https://www.census.gov/fsrdc. 
The commenter states that as examples of controlled access to data in federal research data centers, EPA cites to the NIH's policy for requesting access to controlled-access data maintained in NIH-designated data repositories and the U.S. Census Bureau's website on FSRDC, secure facilities providing authorized access to restricted-use microdata for statistical purposes only. NIH requires researches to be a tenure-track professor, senior scientist, or equivalent and go through required procedures prior to gaining access. The U.S. Census Bureau requires researchers to obtain Census Bureau Special Sworn Status, which requires passing a moderate risk background check and swearing to protect respondent confidentiality for life, with significant financial and legal penalties under Title 13 and Title 26 for failure to do so.
It is unclear how these policies are informing EPA's proposal. EPA's proposal would require data to be made "publicly available," and these forms of restricted access specifically do not make data publicly available. They require significant resources and infrastructure and careful thought about who will be permitted to access such data and under what conditions --  none of which EPA has provided any discussion of in the Proposal.
Response: See Section III.A of the final rule preamble and Chapter 2 of this document. 
Comment [8750-1922]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter cites footnote 22 of the proposal:
      These recommendations are consistent with those of Lutter and Zorn (2016). https:// www.mercatus.org/system/files/Mercatus-Lutter-Public-Access-Data-v3.pdf.we re.
The commenter states that the EPA cites to a working paper by Randall Lutter and David Zorn as supporting the proposition that "EPA should collaborate with other federal agencies to identify strategies to protect confidential and private information in any circumstance in which it is making information publicly available. These strategies should be cost-effective and may also include: Requiring applications for access; restricting access to data for the purposes of replication, validation, and sensitivity evaluation; establishing physical controls on data storage; online training for researchers; and nondisclosure agreements."
Lutter and Zorn reference these strategies as ones agencies could use to minimize the risks to personally identifiable information when agencies make data publicly available. However, EPA's proposed regulations do not discuss or propose implementation of any of these strategies. The Proposal would result in a rule that mandates only that data be made "publicly available" without any possibility for more restricted release. As the comments discuss, EPA has further not consulted with other federal agencies on this Proposal.
Lutter and Zorn additionally do not argue that agencies should immediately disregard studies where data cannot be made publicly available, and provide alternative procedures agencies should utilize in those cases when still relying on studies. In a separate statement on the HONEST Act, which contains similar requirements as the Proposal, Lutter and Zorn stated that the legislation "should also allow agencies to regulate in instances where they do not possess data." While these additional procedures they recommend agencies follow could still be overly burdensome and barriers to EPA promulgating important safeguards, it is important to note that even they see the dangers in a rule that would force the agency to disregard studies when underlying data could not be made public.
Response: The final rule does not require EPA or any entity outside of the agency to make data publicly available. EPA notes that the supplemental proposal underwent interagency review.
Comment [8750-1923]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter cites footnote 23 of the proposal:
      https://www.nap.edu/catalog/11434/ expanding-access-to-research-datareconciling-risks-and-opportunities.
The commenter states that the Proposal claims "[t]he benefits EPA ensuring that dose response data and models underlying pivotal regulatory science are publicly available in a manner sufficient for independent validation are that it will improve the data and scientific quality of the Agency's actions and facilitate expanded data sharing and exploration of key data sets." EPA cites to a National Academies report. This report does speak to many benefits of making data available to researchers, including helping to maintain and improve data quality; promoting new research and exploration of new questions using existing data; and allowing for verification, refutation, or refinement of original results.
However, the report simply considers the benefits of making data publicly available in a broad sense, it does not consider the issue in the Proposal -- which is that new data is not necessarily being made publicly available that was not before, and at the same time EPA's consideration of scientific research is being limited. Thus, it does not consider the costs to government policy-making that come from EPA's refusing to consider scientific research where underlying data is not publicly available. Since it is questionable whether the Proposal will result in any new data being made available to the public, and certain that it will result in EPA's ignoring valid scientific findings, it is unlikely that this Proposal will "improve the data and scientific quality of the Agency's actions" as EPA claims.
Response: See Section III.A of the final rule preamble and Chapter 2 of this document. EPA notes that the final rule does not require EPA or any entity outside of the agency to make data publicly available.
Comment [8750-1924]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter cites footnote 24 of the proposal:
      https://www.mercatus.org/system/files/Mercatus-Lutter-Public-Access-Datav3.pdf. 
The commenter states that the EPA cites to a paper by Randall Lutter and David Zorn for its analysis that ``an increase in existing net benefits from greater reproducibility, which, if it occurred, would cover the costs of obtaining the data and making the data available." However, there are important limitation to this analysis that seriously call this conclusion into question.
First, the statement that EPA cites to is taken out of context. The entire sentence is: "More specifically, we can calculate an increase in existing net benefits from greater reproducibility, which, if it occurred, would cover the costs of obtaining the data and making the data available." This statement is not a conclusion that the benefits of making publicly available data underlying research that federal agencies use to promulgate significant public policies would outweigh the costs. It is describing the figure that Lutter and Zorn go on to calculate -- the threshold level of increase in net benefits required by this policy to equal the costs of implementation. They find that "an improvement in net benefits of 0.02 to 2.08 percent would imply that the net benefits of requiring data access are positive." They themselves note that this estimate "fall[s] short of proving that the benefits outweigh the associated costs."
Their analysis itself is suspect because it differs greatly from the cost estimate provided by the CBO for H.R. 1430, Honest and Open New EPA Science Treatment Act of 2017. The CBO estimated that, if the agency were to choose to rely only on studies that met the Act's requirements from the outset, implementing this legislation would cost about $5 million from 2018-2022. They assumed it would cost $10,000 per study to make data available to enable use of studies. They estimated costs of at least $100 million per year if EPA were to continue to rely on as many studies to support its actions as it has done in recent years. An older cost estimate from CBO on a prior version of the HONEST Act estimated that it would cost "about $250 million a year for the next few years." This assumed that EPA would spend from $10,000 to $30,000 per study to make the data available and that EPA would reduce the number of studies it relies on by about one-half.
Zutter and Lorn calculated an alternative amount for the costs to EPA of this legislation. They find that "the total cost to the EPA for data collection and public accessibility would be $2,558 per study, or about 26 percent of the $10,000 per study cost estimated by CBO." They used estimates that EPA reported under the Paperwork Reduction Act for time that entities in the chemical industry would need to spend to comply with EPA's Health and Safety Data Reporting Rule (40 C.F.R. 716). While they purport that the requirements of that rule are similar to the activities that EPA would undertake to comply with the HONEST Act and similar legislation, they provide no further basis for this. Given the great discrepancy between their and CBO's estimates, it is unclear that their estimate sufficiently accounts for the numerous costs associated with EPA locating underlying research data not currently in its possession and upgrading it to enable it to be made publicly available.
They also rely on questionable assumptions in their calculation. They assume that "given modern technology, by the time research has been published, almost all relevant underlying data and computer code and models will be in electronic format" so time spend photocopying studies will be reduced. This does not consider that EPA may want to rely on older studies where all relevant information is not available in electronic, easily accessible formats. They provide unsupported estimates for activities that EPA would need to undertake to comply with HONEST Act-like legislation that has no corresponding requirement in EPA's Health and Safety Data Reporting Rule -- such as estimating 10 hours for EPA to format unformatted data for public access.
They additionally produce their own estimate for the number of studies that EPA relies on each year, looking at materials posted in dockets on regulations.gov and coming to a total of 18,000 pieces of scientific research per year. CBO estimated 50,000 scientific studies per year. Assuming that EPA continued to rely on all 18,000 studies per year, Zutter and Lorn came to total implementation costs of about $46 million per year, far below the estimate by CBO assuming EPA still relied on at least half of the studies it does currently. Thus, one should view this cost estimate with suspicion, and there is no reason it should be relied on over CBO's cost estimates and does not suffice for EPA providing its own cost benefit analysis.
Response: See Section III.J of the final rule preamble.
Comment [8750-1926]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter cites footnote 10 of the proposal:
 Administrative Conference of the United States' Science in the Administrative Process Project: https://www.acus.gov/research-projects/science-administrativeprocess 
 Improving Access to and Confidentiality of Research Data: https://www.nap.edu/read/9958 
 Expanding Access to Research Data: https://www.nap.edu/catalog/11434/expandingaccess-to-research-data-reconciling-risks-and-opportunities 
 Access to Research Data in the 21st Century: https://www.nap.edu/catalog/10302/access-to-research-data-in-the-21st-century-anongoing 
 Health Effects Institute: https://www.healtheffects.org/system/files/AppendixD-dataaccess_3.pdf 
 Center for Open Science: https://osf.io/x2w9h/?_ga=2.15543670.1160736397.1518527893- 776332106.1518527893 
 Members of the Risk Assessment Specialty Section of the Society of Toxicology, the Dose Response Section of the Society for Risk Analysis, and the International Society for Regulatory Toxicology and Pharmacology: http://www.isrtp.org/GMU%20WEBINAR_DEC_2013/GMU%20Study%20Docum ent4.pdf 
 Bipartisan Policy Center's Science for Policy Project: http://bipartisanpolicy.org/wpcontent/uploads/sites/default/files/BPC%20Science%20Report%20fnl.pdf
Specific comments provided include the following:
I. Health Effects Institute
The EPA provides a link to the HEI Policy on The Provision of Access to Data Underlying HEI funded Studies. This policy is "to provide access expeditiously to data for studies that it has funded and to provide that data in a manner that facilitates review and verification of the work but also protects the confidentiality of any volunteers who may have participated in the study and respects the intellectual interests of the original investigator of the work." It is written to be consistent with OMB Circular A-110, which requires agencies to respond to FOIA requests for data underlying federally supported research used to develop federal agency actions with the force and effect of law. EPA already has policies in place to make public the data underlying research that it funds, and already must comply with OMB Circular A-110, thus, it is unclear how this Proposal builds upon this policy.
Furthermore, the policy specifically excludes "personal and medical information and similar information that is personally identifiable, and the disclosure of which would constitute a clearly unwarranted invasion of personal privacy, such as information that could be used to identify a particular person in a research study" and requires the requestor to pay reasonable costs. In this manner, it further deviates from the Proposal.
II. Center for Open Science
The EPA links to the COS's 2017-2020 Strategic Plan. While the strategic plan outlines COS's own mission to "increase openness, integrity, and reproducibility of scholarly research" and to meets its goal of creating "a future scholarly community in which the process, content, and outcomes of research are openly accessible by default" nothing in this strategic plan suggests anything like EPA's Proposal. It does not discuss barring use of studies or ensuring access to underlying data -- and thus is completely irrelevant to the Proposal.
III. Members of the Risk Assessment Specialty Section of the Society of Toxicology, the Dose Response Section of the Society for Risk Analysis, and the International Society for Regulatory Toxicology and Pharmacology
EPA links to a survey conducted by the Center for Media and Public Affairs and Center for Health and Risk Communication at George Mason University. They surveyed members of the Risk Assessment Specialty Section of the SOT, the Dose Response Section of the SRA, and the ISRTP. However, the survey thus does not represent any official recommendation or policy position from these professional organizations and represent only the views of the members who chose to participate in the survey.
Thus, while the survey found 69 % of those surveyed "regard it as "very important" for assessors to have access to underlying raw data for the most critical studies in order to independently analyze their results," this should be viewed in the rightful context. The survey did not ask whether agencies should continue to rely on scientific studies where the underlying data cannot be made public or independently analyzed. The survey question further appears to have only asked whether researchers assessing studies should have access to underlying data to independently analyze results, not whether underlying data should be made publicly available.
Further, the Dose Response Section of the SRA has since submitted a comment to EPA that states this footnote and the claim that EPA makes that the Proposal took into consideration these recommendations and policies is "inaccurate" and that "the `Dose-Response Section [sic] of the Society for Risk Analysis' has never adopted any `policies or recommendations' on this or any other topic." They have asked that EPA remove all references to the organization and make clear in the comment response for this rule that "'third party Organizations' whose policies and recommendations were considered do not include the Society for Risk Analysis or the Dose-Response Specialty Section."
The SOT similarly have said this survey does not constitute support from the Specialty Section or the SOT as a whole, and requesting "that any and all references to "members of the Risk Assessment Specialty Section of the Society of Toxicology" be removed from the Final Rule." They also specifically comment that "invalidating data solely on the basis of public availability is inappropriate."
IV. Bipartisan Policy Center's Science for Policy Project
The EPA provides a hyperlink to the Final Report of the Science for Policy Project Improving the Use of Science in Regulatory Policy. This report makes a number of recommendations, none of which endorse the Proposal. In relevant part, Recommendation Three suggests "Agencies and their scientific advisory committees should cast a wide net in reviewing studies relevant to regulatory policy, and should make their methods for filtering and evaluating those studies more transparent." They urge agencies to increase availability of data and information on research studies and subject all studies relied on in the formulation of regulation to be subject to the requirements of the Shelby Amendment and OMB Circular A-110 regardless of who funded the study. Importantly, those requirements contain important exception for confidentiality and privacy concerns -- and thus do not support the Proposal. This recommendation is also aimed at increasing use of science in regulatory policy and does not suggest that agencies not rely on studies where those data access requirements cannot be met because of other concerns. It also highlights that the use of CBI to prevent access to data appears to be overused and urges agencies to make procedures more stringent to allow only for legitimate claims of CBI -- which EPA does not address in its Proposal. Recommendation Four states: "The federal government, universities, scientific journals and scientists themselves can help improve the use of science in the regulatory process by strengthening peer review, expanding the information available about scientific studies, and setting and enforcing clear standards governing conflict of interest." As part of this recommendation, the report "Federal agencies, universities and journals should encourage or require on-line publication of the methods and data underlying published scientific studies."(citation omitted) However, it once again does not say that agencies should not consider research studies where this is not possible due to privacy or other compelling reasons.
Wendy Wagner, who served on the panel that produced the recommendations has stated: "They don't adopt any of our recommendations, and they go in a direction that's completely opposite, completely different. . . . They don't adopt any of the recommendations of any of the sources they cite. I'm not sure why they cited them."
Response: See Section III.A of the final rule preamble and Chapter 2 of this document.
Comment [8750-1927]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter cites footnote 11 of the proposal:
 Proceedings of the National Academy of Sciences: http://www.pnas.org/page/authors/journal-policies#xi 
 PLOS ONE: http://journals.plos.org/plosone/s/data-availability 
 Science: http://www.sciencemag.org/authors/science-journals-editorial-policies 
 Nature: http://www.nature.com/authors/policies/data/data-availability-statementsdata-citations.pdf 
The commenter states that while EPA links to journal policies that encourage or require, in some instances, sharing data, they contain exceptions when privacy would be compromised. The editors of these journals issued a joint statement opposing the Proposal. They note that some data sets cannot be shared publicly, and that there are still other methods available to verify scientific findings. The statement also strongly condemns the notion of excluding scientific information from consideration when underlying data cannot be made publicly available:
It does not strengthen policies based on scientific evidence to limit the scientific evidence that can inform them; rather, it is paramount that the full suite of relevant science vetted through peer review, which includes ever more rigorous features, inform the landscape of decision making. Excluding relevant studies simply because they do not meet rigid transparency standards will adversely affect decision-making processes.
Thus, journal policies encouraging the sharing of underlying data do not support a proposal by a regulatory agency to exclude from consideration studies when the underlying data is not publicly available.
Response: See Section III.A of the final rule preamble and Chapter 2 of this document. EPA notes that this final rule does not require EPA to categorically exclude any studies. See response to Comment 4829-2022.
Comment [8750-1931]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter cites footnote 16 of the proposal:
 U.S. Department of Health and Human Services: https://www.hhs.gov/hipaa/forprofessionals/privacy/special-topics/de-identification/index.html 
 National Institute of Standards and Technology: https://nvlpubs.nist.gov/nistpubs/ir/2015/NIST.IR.8053.pdf 
 U.S. Department of Education: https://studentprivacy.ed.gov/sites/default/files/resource_document/file/data_deident ification_terms.pdf 
 U.S. Census Bureau: https://www.census.gov/about/adrm/linkage/technicaldocumentation/processing-de-identification.html 
The commenter states that the EPA suggests the examples linked to could address concerns about privacy and confidentiality arising from the Proposal. However, the cited sources provide no assurance that the Proposal could be implemented to expand disclosure of personal data without serious risks to privacy. The commenter provides the following specific comments:
I. U.S. Department of Health and Human Services 
EPA first points to guidance on de-identification requirements under HIPAA. This guidance provides two methods for de-identifying data: (1) expert determination method, where an expert determines that, after application of statistical and scientific principals and methods, the risk is very small that the information alone or with other available information could be used to identify the subject; and (2) the safe harbor method, requiring that a number of identifiers are removed. The first method requires case-by-case work and EPA has provided no information regarding how EPA could implement it or how much it might cost and thus the feasibility of requiring researchers or EPA to de-identify data this way is questionable. The second method requires removal of much information useful for research that may be necessary to be able to independently validate the research, so it is unclear that it would satisfy the Proposal's demands. Furthermore, the safe harbor method has been shown to provide potentially insufficient privacy protections.
II. NIST
EPA links to a NIST document entitled De-Identification of Personal Information as a potential solution to address concerns about confidentiality and privacy. This document discusses different techniques and issues with de-identification of personal information. However, the document does not discuss de-identification of personal information specifically for the purposes of making research data publicly available for independently validating scientific studies. The document instead notes that:
The purpose of de-identifying data is to allow some uses of the de-identified data while providing for some privacy protection by shielding the identity of the data subjects. These two goals are antagonistic, in that there is a trade-off between the amount of de-identification and the utility of the resulting data. However, de-identification opens up new uses for the data that were previously prohibited due to privacy concerns. It is thus the role of the data controller, standards bodies, regulators, lawmakers, and courts to determine the appropriate level of security, and thereby the acceptable trade-off between de-identification and utility.
EPA completely fails to note this obstacle, that as data is stripped of identifiable material it also loses utility to researchers. EPA cites to broad privacy protection techniques without explaining whether they could be applied to protect privacy while still allowing enough utility in the data set to allow for independent validation as required by the Proposal.
The document notes many of the challenges to protecting privacy including that: "de-identification approaches based on suppressing or generalizing specific fields in a database cannot provide absolute privacy guarantees, because there is always a chance that the remaining data can be re-identified using an auxiliary dataset." The harms of data linkages and increasing difficulty to preserve privacy as more and more information about individuals is made available is another challenge that EPA has not addressed.
III. U.S. ED
EPA links to a document of the Privacy Technical Assistance Center, Data Deidentification: An Overview of Basic Terms, which provides a high-level overview of key terms and practices to help educational agencies and institutions comply with the FERPA. EPA has not explained why the requirements of FERPA are applicable here. This document is concerned with data disclosure that occurs "when schools, districts, or states publish reports on student achievement or share students' data with external researchers" not to make information publicly available for independent validation. Thus it's unclear that methods used to de-identify but preserve data for those purposes would be adequate in this context.
For example, one of the methods that the U.S. ED uses for disclosure avoidance for tabular data is to not release information for any cell that has a size below some minimum, which essentially means not disclosing information where there are small numbers in a certain cell. This could obviously lead to a loss of information that would prevent a deidentified data set from being used to independently validate research findings.
IV. U.S. Census Bureau 
EPA provides a link to a website titled Data Ingest and Linkage that details the U.S. Census Bureau's approach to linking data across many records held by the Bureau, permitting more detailed information to be linked back to one individual to allow for analysis and research. The website links to a working paper that describes the method by which the Bureau assigns a unique person identifier to records it holds that enables it to link records together to create the final file. It is totally unclear how this process on linking together records is a solution that EPA could implement to protect privacy of individuals when disclosing data as it concerns how to identify data to specific people -- not how to make data available while protecting their privacy.
Response: See Section III.A of the final rule preamble and Chapter 2 of this document.
Comment [8750-1936]: Commenter (9227) states that the EPA has provided no sources or authorities that support or provide a reasoned basis for the proposed rule and that many of the sources raise key implementation concerns that EPA fails at all to address -- rendering the proposal arbitrary and capricious.
As an example, the commenter cites footnote 20 of the proposal:
 Taylor & Francis: https://authorservices.taylorandfrancis.com/data-repositories/
 Elsevier: https://www.elsevier.com/authors/author-services/research-data 
 PLOS: http://journals.plos.org/plosone/s/data-availability 
 Springer Nature: https://www.springernature.com/gp/authors/research-datapolicy/repositories 
The commenter states that the EPA cites to "policies or recommendation" of several journals that require data be deposited in public data repositories as an example of the Proposal's requirement of data availability. While these journals have policies that encourage authors to deposit data in public data repositories, they all have important exceptions in cases where this is not feasible or ethical.
The hyperlink for Taylor & Francis links to a page that provides information about how to find public data repositories to submit date to in order to comply with journal sharing policies. However, Taylor & Francis' basic data sharing policy "which applies across many of [their] journals" does not require data be submitted to a public data repository, but "encourages authors to share and make data open where this does not violate protection of human subjects or other valid subject privacy concerns." Thus, this policy is flexible and allows exceptions for when privacy concerns are at stake.
The hyperlink for Elsevier links to a page providing general information about data sharing. While the web page notes that researchers "are increasingly encouraged, or even mandated, to make. . . research data available, accessible, discoverable and usable," it also provides important qualifications. It notes, "there are times when the data is simply not available to post or there are good reasons why it shouldn't be shared." In these cases, authors are encouraged to provide a data statement explaining why the data cannot be shared.
The hyperlink for PLOS links to a page describing PLOS's data availability policies. It explains, "PLOS journals require authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception." The policy recommends deposition of the data into a public repository, however, it recognizes that there are instances when this may not be ethical or legal, for instance because the "underlying data pose privacy or legal concerns e.g., where data might reveal the identity or location of participants." In these instances, it allows an exception to this policy.
The hyperlink for Springer Nature links to a page listing recommended repositories. While Springer Nature's data policies support data sharing via public data repositories, it notes, "reasonable restrictions on data availability are permitted to protect human privacy, biosafety or respect reasonable terms of use for data obtained under license from third parties."
Response: See Section III.A of the final rule preamble and Chapter 2 of this document.
Comment [8750-2620]: Commenter (9227) states that the EPA has failed to explain why it has singled out dose response studies to be excluded if their underlying data and models are not publicly available but has not similarly targeted any other types of studies commonly used by EPA.
EPA also has proposed to target the requirements for public availability specifically to the data and modeling underlying one specific subset of scientific research -- dose response studies. EPA has provided no explanation or justification for targeting dose response studies in particular or for not including other types of studies or scientific information. EPA has not suggested that these studies are inherently less reliable than other studies, that they more commonly fail to publicly disclose data and modeling information, that replication is more necessary for these studies than others, or any other conceivable reason. Absent any explanation from the agency, it is impossible to comment on the factual predicates for EPA's proposed decision, or the reasonableness of EPA's justification, except to state that it appears completely arbitrary in the absence of any rationale. See, e.g., Transactive Corp., v. United States, 91 F.3d 232, 237 (D.C. Cir. 1996) ("A long line of precedent has established that an agency action is arbitrary when the agency offered insufficient reasons for treating similar situations differently.").
Response: See Sections III.A, III.B and III.E of the final rule preamble. EPA notes that this final rule does not require EPA to categorically exclude any studies. See response to Comment 4829-2022.
Comment [8750-2896]: Commenter (8227) states that the EPA has failed to explain why it has singled out dose response studies to be excluded if their underlying data and models are not publicly available, but has not similarly targeted any other types of studies commonly used by EPA.
EPA also has proposed to target the requirements for public availability specifically to the data and modeling underlying one specific subset of scientific research -- dose response studies. EPA has provided no explanation or justification for targeting dose response studies in particular or for not including other types of studies or scientific information. EPA has not suggested that these studies are inherently less reliable than other studies, that they more commonly fail to publicly disclose data and modeling information, that replication is more necessary for these studies than others, or any other conceivable reason. Absent any explanation from the agency, it is impossible to comment on the factual predicates for EPA's proposed decision, or the reasonableness of EPA's justification, except to state that it appears completely arbitrary in the absence of any rationale. See, e.g., Transactive Corp., v. United States, 91 F.3d 232, 237 (D.C. Cir. 1996) ("A long line of precedent has established that an agency action is arbitrary when the agency offered insufficient reasons for treating similar situations differently.").
Response: EPA notes that this final rule does not require EPA to categorically exclude any studies. See response to Comment 4829-2022.
Comment [8750-3567]: Commenter (9227) asserts that the EPA arbitrarily fails to offer a reasoned explanation for its departure from existing policies that broadly require the agency to consider all available scientific information when undertaking rulemakings.
In addition to the statutes that require EPA to use the best available science when making regulatory decisions, a number of EPA's own policies embed this requirement as well. By arbitrarily limiting the science EPA considers when making regulatory decisions, the Proposal contravenes these policies, injuring the scientific integrity of EPA's actions. As discussed in more detail in Section II.E [of the commenter's letter] because EPA is changing course from established policy, EPA must fully acknowledge and justify its decision, which it has failed to do in the Proposal.
EPA's own existing Scientific Integrity Policy states:
      To support a culture of scientific integrity within the Agency, this policy. . . [r]ecognizes . . . policy makers within the Agency weigh the best available science, along with additional factors such as practicality, economics, and societal impact, when making policy decisions.
The Proposal conflicts with this policy by restricting what may be the best available science on a given topic from EPA's consideration solely because the underlying data cannot be made public. As described above, public availability of data is neither necessary nor sufficient to ensure that studies constitute "best available science." The Proposal does not acknowledge this departure from the agency's Scientific Integrity Policy, much less explain why such a departure is reasonable.
Likewise, the Proposal is in tension with EPA's Information Quality Guidelines, developed in response to OMB guidelines issued under Section 515(a) of the Treasury and General Government Appropriations Act for Fiscal Year 2001, which require EPA to ensure the objectivity of influential scientific information it disseminates by using "the best available science and supporting studies conducted in accordance with sound and objective scientific practices." EPA considers information to be disseminated when EPA prepares and distributes information to support an Agency decision or regulation or when EPA distributes information in a way that suggests EPA agrees with it, that it supports EPA's viewpoint, or if in the distribution EPA proposes to use it to support or formulate a regulation or agency decision. Thus, the Proposal conflicts with the Guidelines by restricting scientific studies that EPA may use to support regulations, which may cause it to disseminate other information to support its regulations that is not based on the best available science.
EPA's Peer Review Handbook similarly acknowledges that "EPA strives to ensure that the scientific and technical bases of its decisions meet two important criteria: (1) they are based upon the best current knowledge from science, engineering, and other domains of technical expertise; and (2) they are credible." EPA's Science Policy Council Handbook on Risk Characterization also requires reasonableness in the agency's risk assessments, which is achieved when "the characterization is based on the best available scientific information." These policies clearly impact EPA's regulatory actions, and thus will be impacted by the Proposal. Yet EPA completely fails to analyze the impact the Proposal will have on its ability to comply with these policies and fails to explain why it is changing course or justify its decision to do so. Indeed, the Proposal fails to even acknowledge that the agency is changing positions.
Response: See Section III.A of the final preamble and Chapter 2 of this document. EPA notes that this final rule does not require EPA to categorically exclude any studies. See response to Comment 4829-2022.
Comment [9066-1950]: Commenter (6188) asserts that the Proposed Rule is arbitrary and capricious, as written and as it would be applied. See 5 U.S.C. § 706(2)(A) (an agency's action can be set aside if it is "arbitrary, capricious, an abuse of discretion, or otherwise not in accordance with law"). For example, the Proposed Rule applies only to EPA regulatory actions, which often regulate industry to protect human health and the environment, and omits from its ambit Agency adjudications, which often involve private companies seeking regulatory exemptions or permissions from EPA and are more likely to implicate industry-provided studies that are specific to the particular adjudication in question. The NPRM offers no explanation for this distinction, but documents obtained through the FOIA suggest that EPA introduced the distinction mid-way through the rulemaking process, when the Agency realized the implications of its Proposed Rule for pesticide registration proceedings, many of which rely on industry-generated and proprietary data, which cannot be publicly disclosed. Similarly, while the Proposed Rule suggests that EPA "should be guided by this policy to the maximum extent practicable during ongoing regulatory action," it also provides that the Administrator -- and the Administrator alone -- would be empowered to offer case-by-case exemptions in myriad instances. See 83 Fed. Reg. 18771. As discussed further in the their comment letter, they suggest that this would likely lead to inconsistent application of the Proposed Rule, suggesting its implementation would be arbitrary at best.
Implementation of the Proposed Rule could also impair future Agency rulemaking actions by forcing EPA to ignore key comments and regulatory record evidence in violation of the APA. See Motor Vehicle Mfrs. Ass'n of the United States, Inc. v. State Farm Mut. Automobile Ins. Co., 463 U.S. 29, 43 (1983) (an agency must "examine the relevant data" and acts arbitrarily and capriciously when it "entirely fail[s] to consider an important aspect of the problem"). For example, if a commenter were to cite studies that are based upon data that is not entirely publicly available, the Agency would not be able to consider those studies -- or fully evaluate the comment -- as part of the rulemaking action, regardless of the scientific validity of the comment. In other words, the Proposed Rule would expose the Agency to challenges not only to arbitrary and capricious application of the rule itself, but also to the arbitrariness of future rules.
Simply put, the commenter contends that the Proposed Rule fails to satisfy the basic requirements of the APA and denies the public a meaningful opportunity to engage in this rulemaking process and in future rulemaking processes.
Response: See Section III.A of the final rule preamble and Chapter 2 of this document. Also, see general response regarding compliance with the APA. Regarding the Administrator exemption, see response to comment 6112-3062.
As discussed in the final rule preamble at Section III.A, EPA will issue statute-specific regulations following this procedural rule, some of which will be substantive and therefore subject to public notice and comment. Any implementation issues will be addressed as appropriate in subsequent rulemakings. 
Comment [9217-3571]: Commenter (9225) states that substantively, the rule violates numerous public health and environmental provisions contained in these laws, as well as requirements to use the best available science or to consider all available information, while procedurally, it violates the APA and a number of other laws that set forth specific procedures EPA must follow during its rulemaking process. 
Response: See response to Comment 4829-2022.
Comment [9298-4772]: Commenter (8771) argues that the proposal is arbitrary and capricious for the three reasons stated above: (1) failing to interpret one's data via a model EPA likes is in no way a reason to ignore the data themselves -- so this part of the proposal must be stricken; (2) requirements for transparency must be applied to cost and valuation studies as well as risk ones, or else the proposal is arbitrary; and (3) "reducing reliance on defaults" is tantamount to "ignoring good science," which would be contrary to law.
Response: See Section III.D of the final rule preamble. EPA notes that this final rule does not require EPA to categorically exclude any studies. See response to Comment 4829-2022.
Comment [6891-1526]: Commenter (6912) states that the proposed rule is illegal because it is arbitrary and capricious, and it is arbitrary and capricious in at least three ways. First, the proposed rule grants the Administrator broad discretion to determine whether its terms apply, effectively making it a rule establishing the whim of the Administrator. Particularly in an agency so captured by industry, this cannot be tolerable in a nation of laws. Second, the proposed rule is not supported by any independent authority. Third, the proposed rule is not the product of reasoned decision-making; it is a deliverable, pre-ordained by interested parties laden with bias.
Separately, and in addition, the proposed rule is illegal because it is the result of an effective delegation of rulemaking authority to private interests.
The commenter provides that the APA permits courts to set aside agency actions found to be "arbitrary, capricious, an abuse of discretion, or otherwise not in accordance with the law." In determining whether an agency action was "arbitrary and capricious, the courts look to several factors, whether: "(1) the agency 'relied on factors which Congress has not intended it to consider,' (2) the agency 'failed to consider an important aspect of the problem,' (3) the agency explained its decision in a way 'that runs counter to the evidence,' or (4) the action 'is so implausible that it could not be ascribed to a difference in view or the product of agency expertise. "'
Courts have also held that a rule is arbitrary and capricious if the promulgating agency did not "genuinely engage in reasoned decision making" or if it did not "articulate a satisfactory explanation for its action including a 'rational connection between the facts found and the choice made."'
Response: See response to Comments 2157-149, 4829-2022, 6112-3062, and 6163-1114.
Comment [6168-4100]: Commenter (6178) provides the NPRM proposes to establish a new EPA regulation that will `ensure the data and models underlying the science pivotal to what the Agency deems `significant regulatory actions' is publicly available in a manner sufficient for validation and analysis', hereinafter referred to as the `Transparency Rule' or `Rule'. Specifically, the commenter raises the following concerns with the Transparency Rule proposed by the NPRM: 
      1. The Agency cites specific sections of statutes it administers as the basis of the Agency's authority to adopt the Transparency Rule; specifically: the CAA sections 103, 301(a), 42 U.S.C. 7403, 7601(a); CWA sections 104, 501, 33 U.S.C. 1254, 1361; SDWA sections 1442, 1450(a)(1), 42 U.S.C. 300j - 1, 300j - 9(a)(1); RCRA sections 2002(a)(1), 7009, 42 U.S.C. 6912(a)(1), 6979; CERCLA (as delegated to the Administrator via Executive Order 12580) sections 115, 311, 42 U.S.C. 9616, 9660; EPCRA section 328, 42 U.S.C. 11048; FIFRA sections 25(a)(1), 136r(a), 7 U.S.C. 136r(a), 136w; and the TSCA, as amended, section 10, 15 U.S.C. 2609 (collectively the `Acts'). The Agency rightly cites acts of Congress in an attempt to support the NPRM as such authority is necessary for the Agency to adopt regulations pursuant to the above listed Acts.
      2. However, Congress, in each of the Acts' sections cited, never instructs the Agency to consider transparency to the public of supporting data and/or models of the scientific research when deciding to rely on such research as support for regulations passed pursuant to any of the Acts. Consequently, the Transparency Rule requires the EPA to rely on factors which Congress has not intended it to consider when promulgating regulations pursuant to the Acts, rendering the Transparency Rule arbitrary and capricious under the APA.
      3. Further, not only does the Transparency Rule lack sufficient Congressional intent to avoid being deemed arbitrary and capricious, implementing the Rule would actually contradict Congressional instruction to conduct scientific research with a focus on `promotion' and `coordination' among itself and other institutions that produce scientific research material to the objectives of the Acts Congress requires the EPA to enforce. This provides additional cause for finding the Transparency Rule arbitrary and capricious under the APA.
Response: See response to Comments 4829-2022 and 6163-1114.
Comment [4837-3057]: Commenter (4848) states that the Administrative procedure requires that EPA consider data submitted by the public in evaluating regulations. Let's be clear. Scientific studies submitted have always been of uneven quality. EPA has processes in place (including use of a SAB, testimony and written and oral public notice and comment) using both internal and external peer review to evaluate data. Depending on context, some studies are given greater weight than others. Some studies are disregarded entirely.
It is inappropriate, and likely unlawful under the APA and the express language of numerous environmental statutes including the SDWA and the TSCA, for EPA to categorically eliminate certain types of studies, and hence certain types of data, without considering context.
Response: The final rule does not require EPA to categorically exclude any studies. See response to Comment 4829-2022.
Comment [1929-3901]: Commenter (1943) states that for the reasons stated below, The Minerals, Metals, and Materials Society (TMS) finds the proposal to be arbitrary and limiting, both of which are antithetical to the principles of scientific exploration. We strongly believe that our federal government's regulatory rulemaking process is better served by consideration of all available data and information, especially that which has been carefully reviewed and objectively approved by the scientific community. On behalf of all TMS members, we urge the EPA to reconsider its proposal and cease any development of a final rule to limit the science that underlies the Agency's regulatory process.
Response: The final rule does not require EPA to categorically exclude any studies. See response to Comment 4829-2022.
Comment [6852-2780; 6128-2235]: Commenters (6874, 6153) state that the EPA's process for promulgating this rule represents a gross deviation from the agency's typical action development process. EPA failed to perform an appropriate regulatory impact analysis, despite indications that implementation costs could be substantial. EPA has not received sufficient input from the OIRA, which conducted a highly truncated review and does not appear to have met with outside groups during that process, or from its SAB, which states that the rule "deals with a myriad of scientific issues for which the Agency should seek expert advice from the Science Advisory Board." Taken together, these deviations from typical process leave EPA unable to consider its proposal accurately, and constitute an arbitrary and capricious approach to rulemaking.
Response: See response to Comment 2157-149. EPA's Action Development Process is not legally binding and is a purely internal, collaborative process. In any event, the supplemental proposal underwent OMB and interagency review and was thoroughly reviewed by the SAB. The final rule discusses the SAB's findings and the SAB's report is available in the rulemaking docket.
Comment [6163-1155]: Commenter (6133) notes that their comments discuss the failure of statutory authorities cited by EPA to provide any legal support or authorization whatsoever for the Proposal and its approaches. The commenter notes that the Proposal also cites various executive orders, memoranda, reports, guidelines and the like with the suggestion or implication that these materials somehow provide support for the Proposal. They do not, and thus the Proposal violates the law. See, e.g., Public Citizen Health Research Group v. Tyson, 796 F.2d 1479 (D.C. Cir. 1986) (reversing and remanding agency decision to carry out last-minute directive by White House OMB without any apparent justification in the administrative record). First, of course, EPA's proposed rulemakings must be authorized by federal statutes. Executive orders provide no legal authority for agency rulemakings. Nor may executive orders contradict or alter legal responsibilities an agency has under federal statutes or justify arbitrary and capricious agency action. Equally obvious, memoranda, reports, guidelines, and the like provide no legal authority for agency rulemakings, nor may they justify arbitrary and capricious agency action. See, e.g., Medellin v. Texas, 552 U.S. 491, 524 (2008) ("The President's authority to act, as with the exercise of any governmental power, `must stem either from an act of Congress or from the Constitution itself.'" (citation omitted)); Chrysler Corp. v. Brown, 441 U.S. 281, 302 (1979) ("The legislative power of the United States is vested in the Congress, and the exercise of quasi-legislative authority by governmental departments and agencies must be rooted in a grant of such power by the Congress and subject to limitations which that body imposes."). Second, an agency's proposed rulemaking may not be at odds with federal statutes, may not be creatures of the agency's imagination or policy preferences, and may not be otherwise arbitrary, capricious, or inconsistent with law. The Proposal fails on all of these scores. 
Response: See response to Comment 6163-1114.
Comment [6163-2477]: Commenter (6133) states that when an agency reverses course, it must "provide reasoned explanation for its action." FCC v. Fox Television Stations, Inc., 556 U.S. 502, 515 (2009);Motor Vehicle Mfrs. Ass'n of U.S., v. State Farm Mut. Auto. Ins. Co., 463 U.S. 29, 43 (1983). And when that reversal "rests upon factual findings that contradict those which underlay [the agency's] prior policy," a "more detailed justification" is needed. Fox, 556 U.S. at 515. Indeed, "an agency's decision to change course may be arbitrary and capricious if the agency ignores or countermands its earlier factual findings without reasoned explanation for doing so." Id. at 537 (Kennedy, J., concurring).
As the Supreme Court explained in its 2016 Encino Motorcars decision, an agency must supply "good reasons" for a policy revision, cannot leave "unexplained inconsistency," and must address "serious reliance interests." Encino Motorcars, LLC v. Navarro, 136 S. Ct. 2117, 2126 (2016). In Encino, the DOL reversed its decades-long practice of treating service advisors at automobile dealerships as exempt from the Fair Labor Standards Act's overtime provisions, offering minimal explanation for the policy change. Id. at 2123. The Court overturned the rule, holding that the Department had not met its obligation to offer a "reasoned explanation," especially given the decades of reliance on the policy. Id. at 2126. It was not enough that the Department included conclusory statements declaring its new policy to be a reasonable interpretation of the statute because the Department failed to provide any good reasons for the new policy. Id. at 2127. As explained by the Court, "[t]his lack of reasoned explication for a regulation that is inconsistent with the Department's longstanding earlier position results in a rule that cannot carry the force of law." Id.
In Organized Village of Kake v. United States Department of Agriculture, the USDA, relying on a detailed factual record, decided not to exempt the Tongass National Forest from a rule that would limit road construction and timber harvesting in national forests, explaining that the benefits would outweigh the potential economic loss. 795 F.3d 956, 959 - 61, 967 - 68 (9th Cir. 2015) (en banc). Just two years later, on "precisely the same record," the agency issued a new decision reversing course. Id. at 968. The court concluded that the "absence of a reasoned explanation for disregarding previous factual findings violate[d] the APA." Id. at 969. The court also recognized that "[e]lections have policy consequences," but even when reversing a policy after an election, "an agency may not simply discard prior factual findings without a reasoned explanation." Id. at 968.
EPA previously routinely used and considered science and studies for which the underlying data was not publicly available in regulatory actions. As explained above, EPA has not identified even one example in which EPA has observed the policies underlying the Proposal, and our research has likewise uncovered no such instance. The Proposal essentially admits as much, stating:
      Historically, EPA has not consistently observed the policies underlying this Proposal, and courts have at times upheld EPA's use non-public data in support of its regulatory actions. See Coalition of Battery Recyclers Ass'n v. EPA, 604 F.3d 613, 623 (D.C. Cir. 2010); American Trucking Ass'ns v. EPA, 283 F.3d 355, 372 (D.C. Cir. 2002).
83 Fed. Reg. at 18,769, n.3. The Proposal then goes on to say that "EPA is proposing to exercise its discretionary authority to establish a policy that would preclude it from using such data in future regulatory actions." Id. The Proposal's categorical exclusion of non-publicly available "dose response data" is also departure from EPA's previous practice, as described in the 2002 EPA Guidelines, of weighing all relevant information.
In short, according to the commenter, EPA provide no basis for changing course on this issue, especially when EPA has enshrined the previous policy in agency guidelines and litigation. EPA's failure to explain this change in course violates the law.
Response: See Section III.A of the final rule preamble and Chapter 2 of this document.
1.6.1.1.2 Was the Proposed Rule Issued in Compliance with the APA?
Comment [6099-566]: Commenter (6109) notes that the EPA states that the proposed rule is "consistent with requirements in the APA to ensure public participation in the rulemaking process." However, this proposed rule lacks justification or a practical explanation of how EPA will implement the rule and thereby fails to meet the legal threshold of seriousness defined by the APA, § 307 of the CAA, and court decisions that require an agency to give public notice about the who, what, when, where, why and how of its plans when it wants to create or change a regulation. 
Response: See response to Comment 2157-149. 
As discussed in the final rule preamble at Section III.A, EPA will issue statute-specific regulations following this procedural rule. Forthcoming substantive rulemakings will be subject to public notice and comment and judicially reviewable, and any implementation issues will be addressed as appropriate. 
Comment [6112-1622]: Commenter (6137) provides that the APA establishes rulemaking procedures and judicial review requirements for agency rules that EPA has failed to follow here. 5 U.S.C. §§ 553, 706. These requirements  -  which EPA entirely and unlawfully ignores  -  are central both to ensure an opportunity for the affected public to comment, and to ensure an adequate record for judicial review. This includes: 
 Providing a meaningful opportunity for notice-and-comment; 
 Ensuring that the docket contains all documents on which EPA relies; 
 Providing additional information regarding what the public is being asked to comment on, and
 Adding details that are missing from the proposal, before taking further comment. 
According to the commenter, the EPA has failed to comply with these mandates, rendering this rulemaking unlawful under the APA, for several reasons. 
First, in issuing this Proposed Rule, EPA violated the APA by failing to place in the administrative record all of the documents on which it purports to rely. The preamble to the Proposed Rule cites more than thirty specific documents on which EPA purportedly relies. See, e.g., 83 Fed. Reg. at 18,769-72 & nn.1-2, 4-24. Of the specifically cited and relied upon documents, only about 12 are included in the docket, available at www.regulations.gov, Docket ID: EPA-HQ-OA-2018-0259, as of the date of publication of the Proposed Rule. In addition, EPA provides no support for its proposed conclusion that "EPA believes the benefits of this proposed rule justify the costs." 83 Fed. Reg. at 18,772. Nothing in the record demonstrates what costs or benefits EPA considered to reach this determination, or whether it evaluated at all the harm to public health and to privacy that this Proposed Rule would cause. 
Absent the ability to review such documents, the public is deprived of adequate notice or an ability to provide informed comments regarding the Proposal and EPA's rationale for issuing the Proposed Rule. EPA's failure to put the documents on which it relies into the record violates notice-and-comment under the APA and other statutes that supplement such notice requirements. See, e.g., Am. Radio Relay League v. FCC, 524 F.3d 227 (D.C. Cir. 2008) (agency may not cherry-pick documents on which it relies for public review; it is a violation of public notice and comment to refuse to put the documents in full on which the agency relied into the record). 
Second, failing to place all of the documents on which EPA relies into the record also violates the judicial review provision of the APA by making it impossible for commenters to provide adequate comments or a complete record for judicial review of any final action EPA takes. 5 U.S.C. § 706 ("In making the foregoing determinations, the court shall review the whole record or those parts of it cited by a party, and due account shall be taken of the rule of prejudicial error." (emphasis added)); Am. Radio Relay League, 524 F.3d at 242-43 (Tatel, J., concurring) (reiterating importance of the requirement for agency to place unredacted studies into the docket because not doing so "undermines this court's ability to perform the review function APA section 706 demands"); see also Walter O. Boswell Mem'l Hosp. v. Heckler, 749 F.2d 788, 792 (D.C. Cir. 1984) (citing Citizens to Preserve Overton Park, Inc. v. Volpe, 401 U.S. 402, 420 (1971)) ("[R]eview is to be based on the full administrative record that was before the Secretary at the time he made his decision." (emphasis in original)). 
Not only is EPA's failure to include in the regulatory docket all of the documents upon which it relies a violation of the APA, but it likewise contravenes procedural requirements in Title 28 of the U.S. Code that govern judicial review of agency action. See, e.g., 28 U.S.C. § 2112(b) ("The record to be filed in the court of appeals . . . shall consist of the order sought to be reviewed or enforced, the findings or report upon which it is based, and the pleadings, evidence, and proceedings before the agency . . . concerned." (emphasis added)); Fed. R. App. P. 16 ("The record on review or enforcement of an agency order consists of . . . any findings or report on which it is based." (emphasis added)); see also 28 U.S.C. §§ 2071-77 (providing federal courts with authority to establish binding rules of procedure and evidence). Thus, EPA's slipshod creation of the regulatory docket for the Proposed Rule cannot withstand scrutiny under either the APA or the judicial review provisions for Title 28. 
Third, EPA has violated the APA (and all statutes that track its notice-and-comment requirements) by failing to provide sufficient specificity regarding the Proposed Rule. Under the APA, notice is only sufficient "`if it affords interested parties a reasonable opportunity to participate in the rulemaking process,' and if the parties have not been `deprived of the opportunity to present relevant information by lack of notice that the issue was there.'" WJG Tel. Co., Inc. v. FCC, 675 F.2d 386, 389 (D.C. Cir. 1982) (citations omitted); see Fla. Power & Light Co. v. Nuclear Regulatory Comm'n, 846 F.2d 765, 771 (D.C. Cir. 1988). The Proposed Rule is plagued by a lack of detail as to how EPA will interpret the broad and vague terms it creates, how it will use the Rule in connection with particular rulemakings, and how it will implement the new "independent peer review" and "exemption" provisions, and these shortcomings undermine the sufficiency of the notice provided by the Proposal. For example: 
 The new definitions of "dose response data and models," "pivotal regulatory science," "regulatory decisions," "regulatory science," and "research data," in § 30.2 are extremely broad and impermissibly vague. EPA provides no specific examples of what these terms mean or how they will actually be implemented under the long list of statutes implicated by this rule. 
:: Sections 30.1, 30.2, 30.3  -  discussing the purpose of the Proposed Rule, the applicable definitions, and how the Proposed Rule will apply  -  provide broad requirements for EPA without giving any information regarding how or when EPA will implement them in connection with any given proposed or final rule.
 EPA states in § 30.5 that the Proposal will require EPA to "ensure that dose response data and models underlying pivotal regulatory science are publicly available in a manner sufficient for independent validation . . ." Yet EPA provides no indication of what this means, how it will so "ensure," or what it will do if it is not possible to ensure this, among other shortcomings.
 Section 30.6 addresses the "additional requirements" related to the use of dose response data and models underlying "pivotal regulatory science," but it is devoid of details necessary to assure meaningful public review and comment on how, when, or in what way EPA will implement this. It also cites no specific science and gives no indication of how or why EPA is attempting to redefine scientific information on dose response differently than it has done in the past to allow for informed consideration of why EPA is even attempting to address this issue.
 Sections 30.7 and 30.9 describe the "role" of "independent peer review" in the Proposal and permissible "exemptions" under the Rule, yet these descriptions fail to provide the process that will be used to inform the public of when a given document or study impacted by this rule does or does not qualify for an exemption. There is no indication of whether EPA will provide notice or comment, or any steps allowing for public participation, when deciding whether or not to exclude a given study from consideration.
 The preamble to the Proposed Rule is replete with broad (and unsubstantiated) statements regarding limits on EPA's ability to consider and use science in rulemakings, as well as environmental issues EPA envisions being impacted by the Proposed Rule  -  for example, the NAAQs  -  but EPA fails to provide any specific information to inform public notice and comment on the purported impact of Proposed Rule. Commenters have done their best in view of this to provide comment on all of the likely harm that EPA's vague and general statements would cause if fully implemented in regulatory language, but they are severely prejudiced due to EPA's refusal to provide specific notice as required by the APA.
In sum, the commenter contends that the EPA has failed to satisfy the requirements for specific public notice that can assure meaningful comment, falling far short of the fundamental threshold requirements for notice and comment. This shortcoming is fatal to its ability to finalize this Proposed Rule. 
In addition, the commenter notes that it is especially problematic that EPA's Proposal states that EPA will apply the Proposed Rule to "regulatory decisions" which it defines as "final regulations determined to be `significant regulatory actions' by [OMB]." 83 Fed. Reg. at 18,773 (proposed § 30.2). Under this definition, EPA can exclude and ignore science in future rulemakings without specifying where or how it will do so and without providing any public notice or comment in a future rulemaking that would be impacted by this Rule. EPA's attempt to limit the rulemaking process for undefined other rules in this manner is a violation of the APA for all of the reasons described above. 
Finally, the commenter states that the EPA's APA violations would cause significant harm to Commenters and the public if the Rule were finalized without correcting these problems. The inability to review and attempt to understand the documents on which EPA relies and thus to comment meaningfully and receive effective judicial review cause severe prejudice to the affected public. See 5 U.S.C. § 706(2). Even if EPA were to add documents to the docket at a later date, that would not cure this fatal flaw. Given the sweeping scope of this Proposed Rule  -  which would restrict the consideration and use of important health science in rulemakings which, in turn, would likely lead to a weakening of air, water, waste, chemical, pesticide and other protections  -  the failure to publish the documents for review for an adequate time period has undermined the public's ability to comment meaningfully, and impermissibly prevented the affected public from being able to seek and receive effective judicial review based on the record. 
Response: See response to Comment 2157-149. As discussed in the final rule preamble at Section III.A, EPA will issue statute-specific regulations following this procedural rule. Forthcoming substantive rulemakings will be subject to public notice and comment and judicially reviewable, and any implementation issues will be addressed as appropriate.
Comment [6163-1115]: Commenter (6133) states that, as EPA is aware, when an agency drafts a proposed rule pursuant to congressionally delegated authority, the exercise of that authority is governed by the informal rulemaking procedures outlined in the APA, 5 U.S.C. § 553.5. EPA is required to provide the public with adequate notice of a proposed rule, followed by a meaningful opportunity to comment on the rule's content. 5 U.S.C. § 553 (b)-(c). 
The requirement under § 553 to provide the public with adequate notice of a proposed rule is generally achieved through the publication of a notice of proposed rulemaking in the Federal Register, and the APA requires that the notice of proposed rulemaking include "(1) the time, place, and nature of public rulemaking proceedings; (2) reference to the legal authority under which the rule is proposed; and (3) either the terms or substance of the proposed rule or a description of the subjects and issues involved." 5 U.S.C. § 553(b)1-3. Generally speaking, the notice requirement of § 553 is satisfied when the agency "affords interested persons a reasonable and meaningful opportunity to participate in the rulemaking process." Forester v. Consumer Prod. Safety Comm'n, 559 F.2d 774, 787 (D.C. Cir. 1977). 
The commenter adds that they believe that the Proposal fails to reference any other legal authority to support its adoption. The agency claims its Proposal is "consistent with" Administrative Procedure Act provisions to ensure public participation in the rulemaking process, 83 Fed. Reg. at 18,769/2, but this faint "consistent with" falls far short of any legal authority for the Proposal, or even any claim of such authority. The APA provides no authority for the Proposal and, tellingly, EPA does not and cannot identify any authority therein. Even were this "consistent with" claim an attempt by EPA to claim any legal authority for the Proposal, the throw-away statement fails to provide sufficient notice for the public to comment on the proposed rule or any asserted legal authority in the APA. 
Finally, the commenter states that the Proposal's solicitation of comment -- "on whether additional or alternative sources of authority are appropriate bases for this proposed regulation" -- does not and cannot itself provide any justification for EPA finalizing a rule based on additional or alternative sources of legal authority. This fails to provide sufficient notice for the public to comment on the proposed rule or any other possible legal authorities. For all these reasons, the commenter contends that the EPA lacks any basis to finalize a rule invoking any other legal authorities to support its adoption. 
Response: See response to Comment 6163-1114.
Comment [6163-1215]: Comment (6133) states that the proposed rule's definitions are vague, arbitrary, and capricious, and fail to provide fair notice to the public of how EPA would implement any final rule.
The APA requires notices of proposed rulemakings to include "the terms or substance of the proposed rule or a description of the subjects and issues involved." 5 U.S.C. § 553(b)(3). Proposals must "provide sufficient factual detail and rationale for the rule to permit interested parties to comment meaningfully." Honeywell International, Inc. v. EPA, 372 F.3d 441, 445 (D.C. Cir. 2004).
The instant Proposal lacks any statutory authority for regulatory terms and text, concepts, and other inventions that make up its foundation. Moreover, many of these regulatory terms and text are vague, unexplained, internally inconsistent, and otherwise arbitrary and capricious.
Response: See response to Comment 2157-149.
Comment [6351-3238]: Commenter (6373) suggests that the Rule includes several definitions and requirements that are vague or appear to contradict standard scientific practice, which will make it difficult if not impossible for the scientific community to comply with its mandates. According to the commenter, this is impermissible under the APA, which requires that "general notice of proposed rulemaking shall be published in the Federal Register," including the "terms or substance of the proposed rule." As currently drafted, the Rule's ambiguity about what constitutes "independent validation" prevents clear direction to researchers. While this Rule governs EPA actions, the commenter notes that third party researchers must rely on its direction in order to design studies that the agency may use to inform regulations.
Additionally, the commenter states that the Executive Orders cited as providing the Rule's authority require that regulations must be "simple and easy to understand, with the goal of minimizing uncertainty and litigation," and must specify regulations' effect "in clear language." The commenter contends that the Rule is inconsistent with these mandates. 
Response: See response to Comment 2157-149. The final rule does not cite to any Executive Orders as authority.
Comment [6895-3299]: Commenter (6916) states that the rulemaking process set forth in the APA, and in more detail in the CAA, provide a Congressionally-directed framework to assure that the Agency's rulemaking decisions are not arbitrary  -  that they are made transparently and based on an adequate record that is subject to public scrutiny. Similarly, the commenter notes that the ERDDAA requirement to submit CAA regulatory proposals to SAB review serves as a check to ensure that the scientific basis underlying the rule is robust. 
The commenter contends that the EPA failed to submit the rule to its SAB, and additionally did not prepare a regulatory impacts analysis of the Proposal as required by Executive Order 12,866. Nor did the Administrator provide sufficient time for a meaningful OMB OIRA review, having signed the rule on April 24, 2018 after having submitted the Proposal to OIRA on April 19, 2018  -  despite the requirement to provide OIRA with at least 10 working days to decide whether to waive review.
According to the commenter, nor did the Agency comply with other Executive Orders requiring analyses of other potential aspects of rulemaking proposals, despite their relevance to this proposed action. For example, the Agency did not satisfy the requirements of Executive Order No. 13,045, "Protection of Children from Environmental Health Risks and Safety Risks," 62 Fed. Reg. 19,885 (Apr. 21, 1997), which requires federal agencies to "make it a high priority to identify and assess environmental health risks and safety risks that may disproportionately affect children," in proposing regulatory actions. Instead, despite the fact that among the "dose response data and studies" the Agency is questioning are those analyzing the disproportionate health effects of air pollution on children's health, and the fact that as Executive Order 13,045 itself notes, children "breathe more air in proportion to their body weight than adults," EPA simply states that the Proposal "does not concern an environmental health risk or safety risk." Similarly, EPA dismisses Executive Orders 12,898 "Federal Actions to Address Environmental Justice in Minority Populations and Low-Income Populations," and 13,175, "Consultation and Coordination with Indian Tribal Governments," single sentence stating for each that they do not apply. EPA's failure to subject the Proposal to these required analyses denies the public "transparency"  -  because information is withheld on which the affected public might base meaningful comments  -  as well as denying OMB the information it needs to complete its reviews. 
EPA's Proposal claims to be "consistent with" Executive Order 13,777, which ordered a Regulatory Reform Task Force to identify regulations that "rely in whole or in part on data, information, or methods that are not publicly available or are insufficiently transparent to meet the standard for reproducibility," and 13,783, which required a focus on energy independence, but also the "achieve[ment of] environmental improvements for the American people."] But the Executive Order 13,777 Task Force report does not identify any regulations that rely on insufficiently transparent data, nor is this Proposal - which would deny access to scientific studies - consistent with the goal of advancing environmental improvements for the American public. 
Response: See responses to Comments 2157-149 and 6112-4452. 
Comment [9069-3312]: Commenter (6185) states that the Proposal is too vague as written to provide the public with a meaningful opportunity to comment. According to the commenter, the Proposed Rule, as written, lacks credible specificity and is overly vague in its terms and scope. Under the APA, a federal regulatory agency must publish notice of either the substance of a proposed rule or a "description of the subjects and issues" covered by a proposed rule (5 U.S.C. § 553(b)(3)). In Fertilizer Institute v. EPA, 935 F.2d 1303 (D.C.Cir. 1991), the court observed that it "has consistently interpreted that requirement to mean that an agency's notice must `provide sufficient detail and rationale for the rule to permit interested parties to comment meaningfully'" [citing Florida Power & Light Co. v. United States, 846 F.2d 765, 771 (D.C. Cir. 1988), cert. denied, 490 U.S. 1045 (1989)]. As such, the commenter notes that the EPA is required to articulate the specifics of its proposed rulemakings in a manner that provides a valid opportunity for public comment. 
In this proposal, the commenter provides that the EPA solicits comment across a long list of topic areas, but fails to provide the Agency's own "sufficient detail and rationale" on the solicited comment areas, in contravention to APA § 553(b)(3). Commenters are left in the position of speculating on EPA's views and on those of other commenters that would presumably shape EPA's final rule. It is well settled law that this approach fails to provide adequate notice for informed public comment. In Fertilizer Institute v. EPA, the court held "Commenting parties cannot be expected to monitor all other comments submitted to an agency." In commenters' trying to anticipate potential rule revisions in response to comments from others, the court has also stated, "the EPA must itself provide notice of a regulatory proposal. Having failed to do so, the commenter argues that it cannot bootstrap notice from a comment" [Fertilizer Institute v. EPA, at 1312, quoting Small Refiner Lead Phase-Down Task Force v. EPA, 705 F.2d 506, 547 (D.C.Cir. 1983) (emphasis in original)].
Response: See response to Comment 2157-149.
Comment [9078-3457]: Commenter (6194) provides that the statutory requirements that Congress established for the process by which agencies develop regulations -- while often described as "procedural" in nature -- assure that agencies reach substantively valid and informed outcomes. These statutory and other rulemaking procedures were put in place for a reason; failing to follow them suggests a lack of informed analysis and, without an authorized basis for foregoing the process, an agency decision cannot be upheld. Thus, the commenter states that while lack of compliance with required procedures is itself a fatal flaw in the Proposal, it also undermines the basis of and credibility for the substance of the Proposal. Here, the contends that the proposal violates multiple procedural requirements, including but not necessarily limited to: compliance with the APA and Paperwork Reduction Act (PRA); consultation with the OMB, other federal agencies, and EPA's own SAB; and application of Executive Orders regarding the Protection of Children from Environmental Health Risks and Safety Risks, and Environmental Justice in Minority Populations and Low-Income Populations.
The commenter states that under the APA, notices of proposed rulemakings must reference the legal authority under which the rule is proposed. 5 U.S.C. § 553(b)(2). The Federal Register notice of the Proposal fails to meet this requirement. The Proposal cites provisions from several statutes, but does not explain how any of those provisions provide authority for the Proposal. Even if relevant authority could be found somewhere in the statutes, the APA does not countenance placing the burden on the public to search for it.
The commenter adds that notices of proposed rulemakings must also include "the terms or substance of the proposed rule or a description of the subjects and issues involved." 5 U.S.C. § 553(b)(3). The purpose of this requirement is to "provide sufficient factual detail and rationale for the rule to permit interested parties to comment meaningfully." Honeywell International, Inc. v. EPA, 372 F.3d 441, 445 (D.C. Cir. 2004). The Federal Register notice of the Proposal fails to meet this requirement. It is too vague to allow for fully meaningful and targeted comments. For example, the purported need and objective of the Proposal is couched in references to "transparency" and "validation" but these terms are not defined in the Proposal, nor do they have a single definition amongst professionals in the fields impacted by the Proposal. According to the commenter, the Proposal also improperly uses distinct concepts, such as "reproducibility" and "replicability," as though they are interchangeable. 
The commenter states that the notice similarly presents key issues beyond definitions at only broad stroke levels, raising issues in such a general manner that it fails to provide the public with adequate notice of what EPA intends. For instance: 
 The Proposal requires that certain data and models be made "publicly available in a manner sufficient for independent validation," and indicates that information will satisfy this standard when "it includes the information necessary for the public to understand, assess, and replicate findings." 83 Fed. Reg. at 18,773-74. But it is not clear what degree of availability or explanation would satisfy the Proposal. For example, the "public" referenced in the Proposal could be scientists trained in the relevant field or members of the general public. 
 The Proposal provides that, "where necessary, data would be made available subject to access and use restrictions." Id. What this process would look like is not addressed in the Proposal or discussed at all in the preamble. This leaves significant open questions, such as who will pay for and implement these restrictions and who will decide which people are allowed to access the data and under what standards. 
 The Proposal appears to require EPA to conduct independent peer review of all "pivotal regulatory science," but provides no detail as to what this means or how it will be implemented. Id. For example, it is unclear who within the agency would perform this peer review, when it would occur, and whether (and when) prior peer review in the journal publication process would serve as a substitute for EPA peer review. 
 The Proposal includes an exemption mechanism but does not provide information about the process or conditions under which exemptions shall or can be sought and granted. 
Given these and other ambiguities, the public cannot adequately assess what EPA thinks the Proposal means or how it will implement it. Nevertheless, for purposes of these comments, we assume EPA intends a broad interpretation that would preclude a swath of scientific studies and information from use and consideration in regulatory proceedings. 
Moreover, the commenter states that many of the sources cited in the Proposal either do not support the text of the proposed regulation, do not support the proposition for which they are cited, or are so broad that the public does not have fair notice of, and therefore cannot evaluate, EPA's rationale for the Proposal. For example, EPA asserts that it considered policies or recommendations of third-party organizations that advocate for open science. The Proposal, however, does not cite specific policies or recommendations, instead merely listing the names of institutions. 83 Fed. Reg. at 18,770 & n.9. Even when a specific report is referenced, such as the National Academies' reports on Improving Access to and Confidentiality of Research Data, Expanding Access to Research Data, and Access to Research Data in the 21st Century, the Proposal fails to cite the specific sections EPA purportedly considered or relied upon or the specific policies embodied in these reports that are allegedly advanced by the Proposal. Id. at 18,770 n.10. Other references are similarly vague. 
The overall lack of clarity is illustrated by EPA's own solicitation of feedback; it covers so wide a range of issues -- posing over two dozen questions -- that it makes clear EPA has not identified a proposed path forward on which the public can comment. The Proposal would be more appropriately posited as an Advanced Notice of Proposed Rulemaking; it does not meet the APA's standards for proposed rules. 
Response: See response to Comment 2157-149.
Comment [6112-3191]: Commenter (6137) states that, to the extent EPA decides to apply this Proposed Rule in the context of enforcement, adjudicatory, and permit actions  -  which it should not  -  it must first issue a proposal setting out its basis for such a rule and provide opportunity for comment under the APA and CAA § 307(d). EPA cannot rely on the comment opportunity provided by its April 30, 2018 Proposal, as the Proposal provides no notice whatsoever of the substance of what EPA might or might not include in a rule applying to enforcement and other actions excluded from the regulatory text of the proposal. See 83 Fed. Reg. at 18,773 (proposed 40 C.F.R. § 30.3). 
Response: As discussed in the final rule preamble at Section III.A, EPA will issue statute-specific regulations following this procedural rule. Any substantive regulations will be issued via notice and comment rulemaking.
Comment [6141-2555]: Commenter (6166) suggests that procedurally, the Proposal violates the APA and a number of other laws that set forth specific procedures EPA must follow during its rulemaking process. 
Response: See response to Comment 2157-149.
Comment [6133-2161]: Commenter (6158) states that a lack of analysis is unfair to the public and may also itself constitute a violation of the APA and Executive Order requirements for promulgation of agency rules. The lack of analysis and rationale is further evidence that this proposed rule is unnecessary and burdensome and should be withdrawn. 
Response: See response to Comment 2157-149.
Comment [9066-1949]: Commenter (6188) contends that, in addition to the substantive deficiencies, EPA has failed to satisfy the appropriate procedural requirements for the Proposed Rule's promulgation. 
First, the NPRM does not meet the federal APA requirement to cite adequate authority providing a basis for EPA's promulgation of the rule. See 5 U.S.C. § 553(b)(2) (requiring that an agency make reference to the authority under which the rule is proposed in a NPRM). As discussed, none of the statutory authority identified by EPA authorizes it to support the proposed rule, and further, EPA has solicited comment on this basic question, asking the public to supply the basis for EPA's rulemaking authority. This is not permissible under the APA. 
The NPRM, by soliciting comment on the very nature of the rule itself, also suggests that EPA could adopt a Final Rule that deviates significantly from the Proposed Rule, an outcome which would violate the APA's notice provisions were the rule not first subjected to another round of notice and comment. 5 U.S.C. § 553(b)(3); see National Black Media Coalition v. FCC, 791 F.2d 1016 (2d Cir. 1986) (final rule that deviated significantly from NPRM did not provide the public with adequate opportunity to engage in the rulemaking process as required by the APA). Given the Proposed Rule's potential to affect scores of significant regulatory decisions, its brevity and ambiguity are breathtaking. Indeed, the Proposed Rule is more akin to a brainstorming document than a precisely noticed regulatory action. Without a clear statement of the Proposed Rule's reach and implications, members of the public cannot engage as informed participants in the rulemaking process, the very purpose of the APA. As the Third Circuit recognized in Prometheus Radio Project v. FCC, 652 F.3d 431, 450 (3d Cir. 2011): 
      ...there must be an exchange of views, information, and criticism between interested persons and the agency...Consequently, the notice required by the APA...must disclose in detail the thinking that has animated the form of a proposed rule and the data upon which that rule is based...[A]n agency proposing informal rulemaking has an obligation to make its views known to the public in a concrete and focused form so as to make criticism or formulation of alternatives possible." (quoting Home Box Office, Inc. v. FCC, 567 F.2d 9, 35-36 (D.C. Cir. 1977) (internal citations and footnotes omitted)) 
Here, according to the commenter, despite EPA's purported commitment to transparency, the Proposed Rule contains no "concrete and focused" explanation of its impacts upon which the public can rely to formulate opinions regarding the rule's potential adoption. For example, nowhere does the Proposed Rule assess possible impacts on environmental quality and public health from implementation of this rule, including the likelihood that EPA would be unable to consider important scientific research in its rulemakings and other activities; discuss how issues associated with patient privacy and CBI will be resolved; or identify any authority or data supporting the rule's promulgation. 
Instead, the Proposed Rule creates a confused and muddled picture, soliciting comments on "additional or alternative sources of authority" for the rulemaking and "whether other alternative or additional regulatory or other policy vehicles on a programmatic or statutory level would be appropriate as alternative or additional steps." 83 Fed. Reg. 18769, 18771. Far from satisfying the APA's requirements, the Proposed Rule entirely fails to "describe the range of alternatives being considered with reasonable specificity." Horsehead Res. Dev. Co. v. Browner, 16 F.3d 1246, 1268 (D.C. Cir. 1994) (citing Small Refiner Lead Phase - Down Task Force v. EPA, 705 F.2d 506, 549 (D.C. Cir. 1983)). In other words, the Proposed Rule, on its face, provides the public with no opportunity to understand the authority for its implementation or precisely how its mandates would apply to future regulatory decision-making, and leaves the door open to adoption of a Final Rule that looks very different from EPA's proposal. Given the vagueness of the Proposed Rule and the broad request for comments, the commenter asserts that the EPA must engage in another round of notice and comment before finalizing the rule if it intends to follow any substantive suggestions offered by commenters. 
Response: See response to Comment 2157-149.
Comment [8750-1820]: Commenter (9227) notes that EDF has two FOIAs directly related to the substance of this rulemaking pending at EPA, for which we have received no responsive documents thus far, despite the passage of the statutory deadlines for a response. The first request (No. EPA-HQ-2018-005636) was submitted on March 20, with a determination from EPA statutorily due by April 19 -- which has not been provided. EDF submitted a second request (No. EPA-HQ-2018-007397) on May 4. According to the commenter, given the lack of transparency and information around the basis for this rule, its impacts, and its true motivations, EDF and the public cannot provide informed comment on this rule without the public records that have been requested. The commenter suggests that, for EPA to close the public comment period on this Proposal before all relevant records are released to the public is arbitrary and prevents our ability to meaningfully comment. 
Response: Both FOIA requests EPA-HQ-2018-005636 and EPA-HQ-2018-007397 have been addressed and closed. The EPA disagrees that there was insufficient opportunity for informed comment at the proposed rule stage because the rulemaking docket contains the materials relevant to the public's review.
Comment [8750-1806]: Commenter (9227) provides that Section 553 of the APA, 5 U.S.C. § 553(b)(3), requires that an agency proposing a rule "provide sufficient factual detail and rationale for the rule to permit interested parties to comment meaningfully." The CAA requires even more, that the Federal Register notice be accompanied by a statement of basis and purpose that includes a summary of the factual data on which the proposed rule is based, the methodology used in obtaining the data and in analyzing the data; and the major legal interpretations and policy considerations underlying the proposed rule. The commenter notes that all data, information, and documents on which the proposed rule relies must be included in the docket on the date of publication of the proposed rule.
These core requirements are "designed (1) to ensure that agency regulations are tested via exposure to diverse public comment, (2) to ensure fairness to affected parties, and (3) to give affected parties an opportunity to develop evidence in the record to support their objections to the rule and thereby enhance the quality of judicial review." In addition, "a chance to comment ... [enables] the agency [to] maintain[] a flexible and open-minded attitude towards its own rules," and "avoid[s] the inherently arbitrary nature of unpublished ad hoc determinations." The "notice required by the APA ... must disclose in detail the thinking that has animated the form of a proposed rule and the data upon which that rule is based .... [A]n agency proposing informal rulemaking has an obligation to make its views known to the public in a concrete and focused form so as to make criticism or formulation of alternatives possible." Home Box Office, Inc. v. FCC, 567 F.2d 9, 35-36 (D.C. Cir. 1977); see also Horsehead Res. Dev. Co., Inc. v. Browner, 16 F.3d 1246, 1268 (D.C. Cir. 1994) ("[A]n agency must describe the range of alternatives being considered with reasonable specificity. Otherwise, interested parties will not know what to comment on, and notice will not lead to better-informed agency decisionmaking.") (internal citations and quotation marks omitted). 
According to the commenter, the failure to include critical documents relevant to the proposed rule in the docket, as required by the CAA, itself constitutes a notice violation because "absence of those documents, or of comparable materials. . . makes impossible any meaningful comment on the merits of EPA's assertions." By failing to provide a more developed docket, EPA is frustrating the terms and purposes of these statute's notice requirements. These procedures are in place to form a "specific" proposal that can serve as a "focus for comments," Small Refiner Lead Phase Down Task Force v. EPA, 705 F.2d 506, 548-49 (D.C. Cir. 1983); see Home Box Office, Inc. v. FCC, 567 F.2d 9, 36 (D.C. Cir. 1977) (agency must "make its views known . . . in a concrete and focused form so as to make criticism or formulation of alternatives possible"). Because EPA has not provided supporting evidence, has not included key items it points to as major considerations underlying the Proposal, and has generally presented a vague and unspecified proposed rule and docket, EDF [the commenter] and the public are hindered in our ability to provide specific comment focused on the underpinnings of the Proposal, because we do not know and can only guess as to what they are. 
The commenters adds that even the text of EPA's proposed rule and the statement of basis and purpose fails to provide the requisite notice to allow meaningful comment. At the most fundamental level, the commenter contends that it contains vague and contradictory statements about the actual effect of the Proposal. The Proposal generally appears to make its requirements mandatory -- i.e., failure to make information publicly available will preclude the agency from relying on the study at all. See 83 Fed. Reg. at 18,769 n. 3 ("EPA is proposing to exercise its discretionary authority to establish a policy that would preclude it from using such data in future regulatory actions."); id. at 18,771 ("the regulatory text would impose requirements"); see also id. at 18,769 ("EPA will ensure that the data and models underlying the science is publicly available...") (emphasis added) and proposed section 30.5 ("When promulgating significant regulatory actions, the Agency shall ensure that does response data and models underlying pivotal regulatory science are publicly available in a manner sufficient for independent validation"). In a few places, however, the Proposal makes it sound as if its aims are more aspirational. See id. at 18,770 ("Where available and appropriate, EPA will use peer-reviewed information, standardized test methods, consistent data evaluation procedures, and good laboratory practices to ensure transparent, understandable, and reproducible scientific assessments.") (emphasis added); id. at 18,772 ("The proposed rule directs EPA to make all reasonable efforts to" make data publicly available, but "does not compel the Agency to make that information available where it concludes after all such reasonable efforts that doing so in way [sic] that complies with the law and appropriate protections is not possible.") (emphasis added); see also id. at 18,768 ("EPA should ensure that the data underlying those are publicly available...") (emphasis added). The commenter states that the difference between a requirement precluding use of science and making all best efforts to make data publicly available is enormous. 
To the extent EPA intends to propose a rule that would preclude use of science, as it appears the Proposal would do, the commenter contends that the proposed rule is further flawed because it contains no analysis of how that would affect regulations. How many studies does EPA typically rely on in promulgating regulations? What percentage of these would meet EPA's new requirements? For those that do not, how many could not meet these requirements for patient privacy, CBI, or other reasons? How would EPA set standards if it must rely on many fewer studies? Would EPA be precautionary in the face of less evidence? Would EPA delay promulgating regulations in order to comply with this new mandate? How does this mandate interact with statutory deadlines or statutory requirements that EPA look at a wide range of science? None of these very basic questions are addressed in the proposed rule and without answering them, it is impossible for the public to assess the import and likely consequences of the Proposal. Even more basically, the agency gives no notice as to the Proposal's impacts, its costs, its benefits, why it applies only to regulatory requirements but not to any regulatory actions (like licensing or permitting) that confer a benefit, substantive and procedural criteria for adjudicating waivers, or even the legal theory under which the Proposal issues -- the plaintive solicitation for comment as to "additional or alternative sources" of authority, 83 Fed. Reg. at 18771, does not suffice. 
The commenter states that to the extent the Proposal is intended to solicit comment on how EPA may make reasonable efforts to make data publicly available it is also unlawfully vague. The proposed rule includes numerous footnotes referencing entire websites or even Departments of the Executive Branch. For example, the Proposal claims that "EPA believes that concerns about access to confidential or private information can, in many cases, be addressed through the application of solutions commonly used across some parts of the Federal government." To support this proposition, EPA remarkably cites (without any further elaboration or explanation in the proposal itself) to "examples from the U.S. Department of Health and Human Services, National Institute of Standards and Technology, U.S. Department of Education, and the U.S. Census Bureau." See Small Lead Refiner Phase Down, 705 F. 2d at 548 (requirement that comments are to raise issues with "reasonable specificity" applies equally to the agency giving notice). For example, it is not possible to identify whether the sources referenced support EPA's claim that there are approaches available to address the serious privacy issues raised by the Proposal --  without providing the specific policies and recommendations, a public commenter has no way of knowing whether they are consistent or why EPA believes them to be consistent. It is impossible to respond in a meaningful way without significant guesswork. 
Similarly, in footnote 10, where EPA lists a number of organizations whose "policies and recommendations" the Proposal allegedly took under consideration -- no explanation is provided. In addition, in the proposed rule EPA fails to adequately define key terms like "validation", "independence", "reproducibility," "replication," and "uncertainty," while also citing a "replication crisis" in science. The commenter states that it is important that these terms are defined clearly as these terms are not defined consistently across the scientific community nor governments -- which has implications for the scope and purview of the proposed rule. 
The commenter asserts that this amount of information is wholly insufficient to allow a public commenter to provide meaningful comments about these issues. Courts have been reluctant to find that important information appearing solely in the footnote of a rulemaking document satisfied the notice requirement of the APA, holding that "an agency may not turn the provision of notice into a bureaucratic game of hide and seek." Referencing a key document without further discussion in the rulemaking document itself, and without incorporating it by reference or publishing it in the Federal Register, also does not satisfy the notice requirements of the APA. Subsequent publication of the document may not be enough to cure a defect of notice where an important issue is "belied by the obscurity of the footnote intended to give notice" and further agency procedure is required to provide the public with "the opportunity to comment on a significant part of the agency's decisionmaking process as required by section 553." Thus, the undifferentiated citations in the footnotes of the Proposal do not give adequate notice for public comment.
Response: See response to Comment 2157-149. The rulemaking docket contains the materials relevant to the public's review.
Comment [8750-1779]: Commenter (9227) asserts that the EPA has failed to provide a properly developed record in support of the proposed rule. EPA has not identified sufficient supporting evidence in the Proposal or in its docket and has failed to provide adequate notice of the supporting evidence for the public to respond to meaningfully, as the APA, the CAA and other substantive statutes require. 
Under the APA, agencies must base their actions on examination of the facts, "the agency must examine the relevant data and articulate a satisfactory explanation for its action including a `rational connection between the facts found and the choice made.'" The factual determination underlying the agency decision must be based on substantial evidence and will be set aside "if the agency `relied on factors which Congress has not intended it to consider, entirely failed to consider an important aspect of the problem, offered an explanation for its decision that runs counter to the evidence before the agency, or is so implausible that it could not be ascribed to a difference in view or the product of agency expertise.'" 
Rulemaking under the CAA is subject to the same general requirements of statutory conformity and reasoned decision-making derived from the APA and basic principles of administrative law. CAA rules cannot be "arbitrary, capricious, an abuse of discretion, or otherwise not in accordance with law," "in excess of statutory jurisdiction, authority, or limitations, or short of statutory right," or "without observance of procedure required by law." 
As noted in Appendix A (of their comment letter) and in Section VIII.D (in their comment letter), the commenter states that the EPA's citations for support in the Proposal are vague and uninformative, and even where the particular citation can be identified and located, it is often not clear how EPA thinks the citation supports the Proposal. The commenter asserts that this does not meet the standards of the APA and CAA. 
Additionally, the commenter contends that the EPA has failed to meet the docket requirements of the CAA. CAA section 307(d)(3) requires that publication of the proposed rule in the Federal Register include a summary of the factual data on which the proposed rule is based, the methodology used in obtaining the data and in analyzing the data, and the major legal interpretations and policy consideration underlying the proposed rule. It also requires the agency to place "[a]ll data, information, and documents. . . on which the proposed rule relies" in the rulemaking docket on the date of publication of the proposed rule. The undifferentiated citation of articles and policies, most of which contradict the Proposal or otherwise offer no support for it, fails abjectly to satisfy these requirements. Any document that becomes available after the proposed rule has been published and that is of central relevance to the rulemaking must also be placed in the docket as soon as possible after its availability. The agency must allow enough time for participants in the rulemaking to respond to those documents with comments. 
As of the date of the publication of the Proposal, the commenter states that the docket at regulations.gov contained only the following 12 documents: (1) OIRA Review Start Document (Apr. 17, 2018); (2) OIRA Review Conclusion Document (Apr. 23, 2018); (3) White House Memorandum on Scientific Integrity (Mar. 9, 2009); (4) Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies; Republication, 67 Fed. Reg. 8,452 (Feb. 22, 2002); (5) Exec. Order 13,777, Enforcing the Regulatory Reform Agenda, 82 Fed. Reg. 12,285 (Feb. 24, 2017); (6) EPA, Plan to Increase Access to Results of EPA-Funded Scientific Research (Nov. 29, 2016); (7) OMB Memorandum M-05-03 on Issuance of OMB's "Final Information Quality Bulletin for Peer Review" (Dec. 16, 2018); (8) EPA, Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental Protection Agency (Oct. 2002); (9) Exec. Order 13,563, Improving Regulation and Regulatory Review, 76 Fed. Reg. 3,821 (Jan. 18, 2011); (10) Exec. Order 16, 093, Promoting Energy Independence and Economic Growth, 82 Fed. Reg. 16,093 (Mar. 28, 2017); (11) OMB Memorandum M-13-13: Open Data Policy-Managing Information as an Asset (May 9, 2013); (12) Commission on Evidence-Based Policymaking, The Promise of Evidence-Based Policymaking (Sep. 2017). 
According to the commenter, this clearly is not enough to meet the APA's or CAA's requirements. Aside from the drafts of the proposed rule submitted to OIRA, each of these documents was a pre-existing memorandum, policy document, or executive order that contains no specific analysis -- factual, legal, policy or otherwise -- that pertains to the impacts of or at all justifies this proposed rule. While EPA in the proposed rule cites to some of these documents as purportedly being consistent with these prior policies, see, e.g., 83 Fed. Reg. at 18,769-70, as is discussed in Section II and in Appendix A (of their comment letter), these policies do not in fact provide any basis for the Proposal. The record that EPA provides clearly fails to support its proposed action. Some of the factual data, legal interpretations, and policy considerations that EPA has not sufficiently provided evidence for include: the number of scientific studies that would be precluded from consideration under the Proposal; whether there are fields of research where the Proposal would result in insufficient scientific information available for EPA to meet its statutory duties; how EPA will address the substantial privacy concerns implicated by the Proposal; how application of this Proposal will impact substantive agency actions; what the costs of implementing this Proposal are if EPA intends to not just exclude studies from consideration where too costly to provide access, etc. 
The commenter notes that the EPA, for instance, includes Executive Order 13,563 in the docket to support its statement that "[t]he best available science must serve as the foundation of EPA's regulatory actions."While Executive Order 13,563 makes that statement, it does not support EPA's Proposal, which as explained above, hinders EPA's use of the best available science. EPA provides no evidence or explanation in the docket or Proposal for why EPA believes this policy would further that goal. The executive order only states that agencies should make available to the public the scientific or technological findings or conclusions on which rules rely, as opposed to underlying raw data that EPA has targeted with this Proposal. Meanwhile, EPA blatantly violates the executive order's provisions requiring agencies to weigh costs and benefits; to write regulations that are easy to understand; and to provide the scientific and technical findings underlying the rule for the public to comment on. 
Section 307(d)(3) of the CAA requires that "[a]ll data, information, and documents ... on which the proposed rule relies shall be included in the docket on the date of publication of the proposed rule." Many items that EPA cites to in the Proposal as providing a basis for the proposed rule do not appear in the docket. For example, EPA states: "The proposed rule takes into consideration the policies or recommendations of third party organizations who advocated for open science." In a footnote, EPA provides: "These include policies and recommendations from: The Administrative Conference of the United States' Science in the Administrative Process Project; National Academies' reports on Improving Access to and Confidentiality of Research Data, Expanding Access to Research Data, and Access to Research Data in the 21st Century; the Health Effects Institute; Center for Open Science; members of the Risk Assessment Specialty Section of the Society of Toxicology, the Dose Response Section of the Society for Risk Analysis, and the International Society for Regulatory Toxicology and Pharmacology; and the Bipartisan Policy Center's Science for Policy Project." Many of these policies and recommendations did not appear in the docket on the data of publication of the Proposal and still do not appear in the docket -- a clear violation of the CAA -- nor are the specific documents or reports even identified or properly cited so that they may be tracked down. This is evidently prejudicial to commenters -- it undermines commenters ability to submit meaningful feedback when the agency is hiding the ball in this manner. 
These policies and recommendations are not easily identifiable on their own either, even after significant internet research. This is also true of footnote 16, where EPA lists a number of agencies to support its claim that the federal government is already implementing solutions to data disclosure. EPA cites, for example, the National Institute of Standards of Technology. NIST has numerous policy documents on protecting privacy concerns and keeping data secure as well as its own internal policies on releasing data. It is hard to see how any are relevant here, but without a particular cite the public is denied even a chance to respond to whatever EPA is trying to use as support -- or must respond to everything that might be being referenced, creating a burdensome task. Throughout these comments, as we attempt to respond to EPA's Proposal, we have been very practically limited by our inability, even after much research and consideration, to be fully certain we have identified the appropriate policies to respond to. This presents a situation that the CAA's docket requirement was exactly formulated to prevent. 
On May 25, 2018, EPA added a memorandum to the docket for this rulemaking. This memorandum contains hyperlinks apparently intended to accompany various citations in the footnotes of the Proposal. This document does not cure the former procedural defect, as the CAA requires information the proposed rule relies on to be placed in the docket on the day the proposed rule is published. Further, these hyperlinks still link ambiguously to various documents and agency websites without providing any information about what specifically EPA intends to cite or how the cited information is being used or considered by EPA. Additionally, simply adding such a document to the docket does not provide adequate notice to the public. Someone who had access only to the proposed rule and was not carefully monitoring the docket would have no indication or notice of this new document. Either EPA is failing to comply with the CAA's requirements by failing to include in the docket factual data, legal interpretations, and policy considerations that support the Proposal, or these supporting items do not exist, deeming this rulemaking completely arbitrary -- in either case the Proposal fails to meet the standards of the APA and CAA. Under the CAA the rulemaking docket "must provide the entire basis for the final rule and the exclusive record for judicial review," this docket clearly cannot support a final rule.
Response: See response to Comment 2157-149. The rulemaking docket contains the materials relevant to the public's review. EPA notes that this is an internal procedural rule and is not subject to CAA § 307(d). As discussed in the final rule preamble at Section III.A., EPA will issue statute-specific transparency regulations or programmatic regulations after this procedural rule.
Comment [8750-1776]: Commenter (9227) provides that the APA, the CAA, and other federal statutes proscribe procedures that must be followed in agency rulemaking, and which EPA has failed to meet in its Proposal. This proposed rule does not fit into any of the exceptions the APA provides for the procedural requirements of rulemaking -- it is neither an interpretive rule, general statement of policy, or a rule of agency organization, procedure or practice.
The commenter states that the proposed rule does not purport to clarify or explain an already existing statute or rule, and thus is not an interpretive rule. The proposed rule is not a general statement of policy, because it establishes a standard of conduct, which has the force of law. It uses mandatory language indicating a requirement: "When promulgating significant regulatory actions, the Agency shall ensure that dose response data and models underlying pivotal regulatory science are publicly available in a manner sufficient for independent validation." Unlike a general statement of policy, which "does not establish a `binding norm,'. . . [and] is not finally determinative of the issues or rights to which it is addressed," EPA here makes no qualifications that it has any leeway to not follow the Proposal's new requirements in all future regulatory actions. The provision allowing the EPA Administrator to grant exceptions in a limited number of cases does not turn this rule into a general statement of policy because it also binds the Administrator's discretion, allowing deviation from the policy only when they make specific findings. EPA has not indicated that "in subsequent proceedings it will thoroughly consider not only the policy's applicability to the facts of a given case but also the underlying validity of the policy itself," but seems poised to apply the policy in all instances -- granting exceptions only in limited circumstances where compliance is deemed impracticable. It nowhere indicates that EPA may reassess in each case whether following this rule is the best means to achieve scientific integrity as it undertakes regulatory action. The Proposal has other indications of a binding rule, including that EPA intends to codify it in the CFR, and EPA has itself characterized the Proposal as a binding rule.
The commenter adds that this rule is also not a rule of agency organization, procedure or practice, for purposes of the APA. Agency actions in this category are those "that do not themselves alter the rights or interests of parties, although it may alter the manner in which the parties present themselves or their viewpoints to the agency." An agency action that "trenches on substantial private rights and interests" does not fall under this exemption. By restricting the scientific studies on which EPA may base final significant regulatory actions, EPA severely limits parties from relying on excluded studies in advocating for particular safeguards. In the preamble, EPA makes clear that the rule is about "EPA's regulatory actions" and underlying conclusions. Because the rule substantively impacts agency conclusions and regulations, it impacts private rights and interests. The rule does not allow private individuals to submit for consideration (or renders such submittal a nullity) studies that they would have been permitted to prior to the proposed rule, thus impacting the substantive standards that EPA is able to justify setting -- which has implications for the regulated community as well as for public health. The Proposal "encodes a substantive value judgment or puts a stamp of approval or disapproval on a given type of behavior" by requiring regulatory actions to be supported only by certain scientific information deemed acceptable by the proposed rule.
In CropLife Am. v. E.P.A., the commenter provides that the Court held that a similar rule promulgated by EPA, barring third-party human studies from agency consideration during pesticide registrations was a binding regulation because it used "clear and unequivocal language" reflecting "an obvious change in established agency practice" that created a "binding norm." The Court stated: "EPA's stated rule is binding on petitioners, who are now barred from relying on third-party human studies (even in cases where such studies formerly were approved), and is binding on the agency because EPA has made it clear that it simply `will not consider' human studies." Similarly, the Proposal appears to bind EPA to not consider scientific information it could consider before, unless it falls under certain narrow, ambiguously defined exceptions, and binds the public and organizations such as EDF who can no longer submit studies to EPA that EPA would previously have been required to consider as part of the rulemaking process. 
Response: See response to Comments 84-1403 and 4829-2022.
Comment [6894-3628]: Commenter (6915) argues that the EPA's skeletal outline falls far short of the APA's notice requirements and fails entirely to "examine the relevant data and articulate a satisfactory explanation for its action including a rational connection between the facts found and the choice made." Natural Res. Defense Council, Inc. v. U.S. Envtl. Prot. Agency, 859 F.2d 156, 209 (D.C. Cir. 1988). The commenter states that the EPA should withdraw the proposal on these grounds alone. 
Response: See response to Comment 2157-149.
1.6.1.2 APA (SNPRM Comments)
Comment [25-96]: Commenter (10972) writes that Section 553(c) gives interested persons a right to participate and a right to consideration of their submissions.
Interested persons have a right to participate in a rulemaking through submission of written data, views, or arguments. The EPA here fails to comply with the APA's participation requirements in three ways. First, the scheme suggested in section 30.5 of the proposed regulation, particularly the option 1, channels the work of interested persons from the submission desk into the dustbin on a bright-line basis which has nothing to with the substance of the data, views or arguments expressed therein. This serves to guillotine interested persons' right to participate, in contravention of the APA.
Second, the SNPRM itself is non-compliant with section 553(b)(3) of the APA as read with the participation requirement in section 553(c) in that it negates realistic participation by providing no clarity on the nature and scope of the problem sought to be regulated. The SNPRM fails to meaningfully discuss the fundamental expansion it works on the NPRM. It provides no justification for jumping from a rule applicable to dose-response models to a rule that applies to essentially any research the EPA may use; as noted above, there seems to be no real basis for this. An explanation of the problem that necessitates this expansion is necessary. This is not least because the EPA has dealt with such data for a long time and it must  -  if this proposed rule is in any way supportable  -  undoubtedly be able to point to incidents where regulatory error arose (especially beyond the dose-response context) due to apparent transparency failings. Realistic participation of the public is seriously compromised when the agency fails to provide clarity on the nature and scope of the problem to be regulated. How is the public supposed to participate in determining the size of the bridge to be built when the size of river underneath is concealed by a vast tarp? The inconsistencies of this SNPRM with substantive laws that the EPA itself points to  -  together with the vast scale of application introduced by this SNPRM  -  suggest that Executive Order 12866 ought to apply. Under Executive Order 12866, regulation should only occur when law or compelling public need requires it. Yet the expansion of the NPRM to the SNPRM was carried out with nary a word on what compelling public need required it; that is improper.
Third, the very fact that this is an SNPRM attaches it to the NPRM to which it is supplementary. The NPRM was about dose-response models and data  -  a feature of a very limited set of sciences; the SNPRM is about all models and data across all sciences  -  including the social sciences. To make that type of vast expansion as a "supplement" to the very narrow initial concept raises the probability  -  well-nigh certainty  -  of restricting participation. The SNPRM then points out that there will be no consideration of anything that relates to the NPRM. Further, the EPA has limited comments to 60 days in the middle of a national emergency. All this highlights the procedural invalidity of the SNPRM.
Section 553(c) gives a right to consideration of such relevant matter as is presented by interested persons. The determination of relevance is surely a matter of substance. The EPA cannot purport to turn it into a non-substantive procedural fiat as it seeks to under section 30.5 of the proposed rule. Section 30.5 of the proposed rule is therefore unlawful.
Given the rights to participate and to consideration of their submissions, an interested person would expect that exclusion of their submission would occur in terms of a clear-cut procedure with appropriate safeguards of their rights. Yet the proposed rule makes no such provision. Exclusion of an interested person's study from consideration is also a substantial repudiation of that person; it is a restrictive action or limitation that can be classified as a sanction as defined under section 551 of the APA, with all the legal consequences which that entails. The proposed regulation is also silent as to whether a reanalysis will be sent to an initial author for comment and response and how this will be administered.
Response: As explained in Section III.E of the final rule preamble, the final rule adopts a weighting approach to considering data availability and not the categorical exclusion presented as an option in the NPRM. EPA has explained that this procedural rule has limitations on its scope in terms of conflicts with other substantive requirements and that, in situations where this rule applies, EPA will give greater consideration to those studies where the underlying data and models are available for independent validation. That means that the agency will consider all relevant studies when taking covered actions.
EPA disagrees that this rulemaking does not comply with the APA. EPA explains the expanded scope and the basis for that expansion in Section I.B of the SNPRM preamble (85 FR 15396) and Sections II.B and III.B of the final rule preamble. EPA notes that this rule is purely procedural in nature and therefore is not subject to APA notice and comment. Nevertheless, EPA initiated two rounds of notice and comment of 60 days each and made changes to both the SNPRM and the final rule in response to those comments. While EPA was not required to take comment on this procedural rule, the 60-day comment period meets the recommendation of Executive Order 12866 pertaining to the recommended length of comment periods for significant rulemakings.
Comment [34-154]: Commenter (11192) writes that this SNPRM demonstrates a disingenuous request for public input from the EPA. The EPA is obligated through the APA to notify the public and invite public comment on proposed regulations. While fulfilling the procedural requirements of the APA through this SNPRM, the EPA downplays the public interest of this sweeping proposed regulation, repeatedly asserts that this proposed rule is an affair internal to the agency, and sets up a path for the agency to dismiss public input regarding the STRS rule.
In the first paragraph of the Executive Summary, the EPA appears to dissuade public comment by understating the interest of members of the general public in a proposed rule designed to significantly alter regulations protecting public health. First, the EPA frames the proposed rule as internal to the agency, stating "This SNPRM does not regulate any entity outside the Federal Government. Rather, the proposed requirements would modify the EPA's internal procedures regarding the transparency of science underlying regulatory decisions." The EPA then concedes that "any entity interested in EPA's regulations may be interested in this proposal," providing an example of "entities that conduct research or another scientific activity that is likely to be relevant to EPA's regulatory activity."
Response: Please see the response to comment 25-96.
Comment [34-156]: Commenter (11192) writes that in addition to asserting that the EPA can introduce these changes under its housekeeping authority, the EPA also lays the groundwork in this SNPRM to subvert public comment in the future. The EPA states that it is "considering how to proceed, apart from this supplemental proposal, to establish regulations interpreting provisions of, and/or exercising substantive rulemaking authority delegated to it by programmatic statutes." Creating regulations specifically about how the EPA interprets or carries out its statutes could result in a process whereby many, perhaps all, new environmental regulations pursuant to the EPA's statutes, including the CAA and CWA, would be considered internal procedures and thus give the EPA leeway to circumvent the APA's required notice-and-comment rulemaking. The EPA has been acting with an aggressive deregulatory agenda for the last three years. This action not only shuts the public out of federal environmental decisions but could lay the groundwork for discarding all of the EPA's regulatory accountability. We firmly oppose the establishment of regulations that could allow the EPA to circumvent notice-and-comment rulemaking.
Response: This action underwent two rounds of public notice and comment despite being a procedural rule and therefore not subject to those requirements. Furthermore, and as mentioned in Section III.D of the final rule preamble, any substantive actions implementing this procedural rule will be subject to public notice and comment.
Comment [48-502]: Commenter (12727) writes that the Supplemental Notice would violate the APA and EPA's authorizing statutes by depriving the public of an adequate opportunity to comment on future rulemakings.
The SNPRM would prevent the public from commenting meaningfully on future rulemakings by withholding information on the studies the agency is considering or will consider. In order to comment in an informed way, it is essential for the public to understand the agency's rationale for ignoring otherwise relevant information. If the agency fails to explain why it has excluded a study in issuing a proposal, or will not consider it when submitted by commenters, the public cannot object to its exclusion and has no indication whether the agency has rejected it because of the requirements of this rule or because the agency deemed the study irrelevant to its regulation. The latter, if true, would be crucial to an understanding of the agency's view of the scope and purpose of its regulation. Yet the Supplemental Notice suggests that EPA will only provide an explanation for disregarding or discounting a study under its alternative "weighting" option. It has not indicated that it will identify the studies it has entirely ruled out under the preferred "tiered access" approach. Without an understanding of how this rule is operating to suppress scientific information in future rulemakings, commenters cannot understand the basis of the agency's proposals and may develop and submit information that the agency will discard out of hand. They would also be unable to oppose EPA's disregard of otherwise relevant information, which it may be required to consider under the controlling statute. This `black box' precludes fully informed comment on future rulemakings and therefore violates the APA and EPA's authorizing statutes.
Response: As mentioned in Section III.D of the final rule preamble, any substantive actions implementing this procedural rule will be subject to public notice and comment. EPA will make available to the public all information necessary to ensure meaningful comment on those actions upon their proposal.
Comment [48-1481]: Commenter (12727) writes that EPA's failure to consider how its proposed rule could interfere with its ability to discharge its statutory duties and fulfill its mission of protecting public health and the environment renders its proposed action arbitrary and capricious under the APA.
Response: EPA disagrees that it failed to consider this issue, as evidenced by its specific request for comments on the appropriateness of relying on the substantive statutes that EPA administers as authority for the rule. See NRPM and SNPRM at Section I. C. of preamble (83 FR 18768) and Section I. B. of preamble (85 FR 15396), respectively.
Comment [54-2034]: Commenter (11361) writes that in addition to asserting that the EPA can introduce these changes under its housekeeping authority, the EPA also lays the groundwork in this SNPRM to subvert public comment in the future. The EPA states that it is "considering how to proceed, apart from this supplemental proposal, to establish regulations interpreting provisions of, and/or exercising substantive rulemaking authority delegated to it by programmatic statutes." Creating regulations specifically about how the EPA interprets or carries out its statutes could result in a process whereby many, perhaps all, new environmental regulations pursuant to the EPA's statutes, including the CAA and CWA, would be considered internal procedures and thus give the EPA leeway to circumvent the APA's required notice-and-comment rulemaking. The EPA has been acting with an aggressive deregulatory agenda for the last three years. This action not only shuts the public out of federal environmental decisions but could lay the groundwork for discarding all of the EPA's regulatory accountability. We firmly oppose the establishment of regulations that could allow the EPA to circumvent notice-and-comment rulemaking.
Response: Please see response to comment 48-502.
Comment [81-590]: Commenter (12715) writes that the supplemental proposal is arbitrary, and too vague to allow for meaningful public participation.
The APA requires that "general notice of proposed rulemaking shall be published in the Federal Register," including the "terms or substance of the proposed rule." 5 U.S.C. § 553(b). The straightforward purpose of this requirement is to give the affected public an opportunity to provide meaningfully informed comment on an agency's proposal. See Home Box Office, Inc., supra, 567 F.2d at 35-36. Further, an agency's regulations cannot be arbitrary, capricious, or contrary to the agency's statutory authority. 5 U.S.C. § 706. Courts will not hesitate to strike down final rules based on proposals lacking in specificity. See, e.g. Horsehead Res. Dev. Co. v. Browner, 16 F.3d 1246, 1268 (D.C. Cir. 1994) ("general notice that a new standard will be adopted affords the parties scant opportunity for comment"). Here, however, in the words of EPA's own SAB, "[g]iven the relatively skeletal nature of the Proposed Rule, it is not possible to define the implications of the rule with confidence." SAB Report, at 2. In particular, it is unclear which definitions are included in the proposed rule, EPA fails to define key features of the two "options" proposed for determining which data and models may be used, and the proposed rule's exemptions provision contains insufficient standards to guide the Administrator's discretion. The supplemental proposal thus runs afoul of the APA's substantive and procedural requirements due to its vagueness as well as its arbitrariness, thus depriving the public of a meaningful opportunity to comment. Accordingly, if EPA does not abandon the proposal altogether, it should reclassify the proposal as, at most, an Advance Notice of Proposed Rulemaking.
Response: EPA clarifies the relevant definitions in the final rule preamble at Section III.C elaborates on key features for determining which data and models may be used at Section III.E, and outlines the standards the Administrator will be required to apply in determining whether to grant an exemption at Section III.G.
Comment [81-593]: Commenter (12715) writes that the supplemental proposal's definitions section is contradictory and unclear.
The supplemental proposal's contradictory and confusing discussion and presentation of the definitions section of the proposed rule violates the APA's requirement that a proposal must allow for meaningfully informed comment. Specifically, the narrative section of the supplemental proposal states that EPA is "modifying, deleting and proposing new regulatory text" in the definitions section, including "deleting the first paragraph of the 2018 proposed rulemaking regulatory text" and the definition of research data. 85 Fed. Reg. at 15,398. But in the proposed rule section, EPA states that it is "[r]evis[ing] § 30.2 by adding the [listed] definitions . . . to read as follows." Id. at 15,404 (emphasis added). Unlike other sections of the supplemental proposal where EPA appears to have included the full revised provision, the proposed rule text for section 30.2 appears to include only the added definitions. It is thus unclear which, if any, portions of the original proposed rule text are being "modif[ied]" or "delet[ed]" as the narrative description indicates. EPA's unclear presentation of the changes it is making to this section of the supplemental proposal deprives the public of the opportunity to provide meaningful feedback on this portion of the rule.
The definitions are also problematic because the proposed rule repeatedly refers to "significant regulatory decisions," id. at 15,405-06, but the proposal fails to provide any definition of the terms "significant regulatory decisions" or "significant." Even more confusingly, "regulatory decisions" are defined in the original proposal as "final regulations determined to be `significant regulatory actions' by the Office of Management and Budget." 83 Fed. Reg. 18,768, 18,773 (Apr. 30, 2018). Moreover, numerous other definitions in the proposed rule are also fatally flawed as described in our Technical Comments below.
The confusion throughout the definitions section infects the entire proposal because understanding whether and how EPA has defined key terms is critical to understanding how the other substantive provisions of the rule would operate. The supplemental proposal, like the original proposal, is therefore unlawfully vague and arbitrary.
Response: EPA made several changes to the definitions section to address these concerns. Please see the final rule preamble at Section III.C and final regulatory text at §30.2.
Comment [82-711]: Commenter (12464) writes that multiple aspects of the supplemental notice are incomplete, ambiguous, or otherwise fail to provide adequate notice of the contents of the proposal.
Under the APA, a notice of proposed rulemaking must include "the terms or substance of the proposed rule or a description of the subjects and issues involved." 5 U.S.C. § 553(b)(3). The purpose of this requirement is to "provide sufficient factual detail and rationale for the rule to permit interested parties to comment meaningfully." The Supplemental Notice fails to meet this requirement. Several aspects of the Proposal remain too incomplete or vague to allow for meaningful and targeted comments.
While we identify multiple topics on which the Supplemental Notice has not provided sufficient detail throughout this comment letter, some of the most important issues are highlighted below.
Response: Both the SNPRM and final rule contain a description of the subjects and issues involved. See 85 FR 15396 and final rule Federal Register notice for this action, respectively. 
Comment [97-901]: Commenter (11479) writes that while EPA does not cite the APA as a source of its authority, this proposal would be violating the law because the agency would be arbitrarily and capriciously selecting which information to use in decision-making. This is especially true given section 30.9 of the rule, which would allow the administrator to exclude certain studies from the rule's requirements on a case-by-case basis. Furthermore, under the APA, agencies are required to consider and respond to all information presented in the docket. If EPA chooses to ignore a portion of this evidence because it cannot access or publicize the underlying data, models, and codes, such action would be arbitrary and capricious.
Response: Per section 30.9 of the final rule, the Administrator may grant an exemption, but in so doing, the Administrator must apply specifically enumerated standards and provide a justification for the Administrator's decision. EPA has articulated the standards to be applied and imposed a process for documenting an exemption decision which demonstrates that this rule will not be applied in an arbitrary or capricious manner. Furthermore, the rule does not require EPA to "ignore" underlying data and models. Rather, in lieu of a categorical exclusion, EPA has finalized a weighting approach to considering such information. EPA has explained that this procedural rule has limitations on its scope in terms of conflicts with other substantive requirements and that, in situations where this rule applies, EPA will give greater consideration to those studies where the underlying data and models are available for independent validation. That means that the agency will consider all relevant studies when taking covered actions.
Comment [157-1480]: Commenter (12422) writes that under the provisions of the APA, the EPA administrator does not have the authority to refuse to consider any comment submitted to the agency. If she or he thinks it is not valid, inaccurate, or inapplicable, she or he must explain why. Under the APA, submissions, including scientific studies, cannot arbitrarily or capriciously be discarded because the underlying data are not provided. When I was OSHA Administrator, we wanted to protect the integrity of the science used in setting regulations, so we explored asking for conflict-of-interest disclosures, similar to those requested by every leading scientific journal. Our legal experts determined that we could request this disclosure, but we could not reject submissions that failed to include them. This is a comparable situation  -  rejecting submitted studies because the underlying data are not available is prohibited under the APA.
Response: EPA has finalized a weighted approach to considering underlying data and models, not a categorical exclusion. EPA has explained that this procedural rule has limitations on its scope in terms of conflicts with other substantive requirements and that, in situations where this rule applies, EPA will give greater consideration to those studies where the underlying data and models are available for independent validation. That means that the agency will consider all relevant studies when taking covered actions.
Comment [159-1415]: Commenter (12430) writes that Agency action violates the APA, 5 U.S.C. section 706, if it is "arbitrary, capricious, an abuse of discretion, or otherwise not in accordance with law." Agency action is arbitrary and capricious when the agency "has relied on factors which Congress has not intended it to consider, entirely failed to consider an important aspect of the problem, offered an explanation for its decision that runs counter to the evidence before the agency, or is so implausible that it could not be ascribed to a difference in view or the product of agency expertise[,]" or otherwise fails to "examine the relevant data and articulate a satisfactory explanation for its action `including a rational connection between the facts found and the choice made.'"
If finalized, the SNPRM would be arbitrary and capricious, and not in accordance with law. Congress has directed U.S. EPA to use the "best available science" or the "latest scientific knowledge" in a multitude of programs. Congress has never directed U.S. EPA to consider the public availability of underlying data and models as the sole or even predominant factor in evaluating the soundness of science. Nevertheless, U.S. EPA is proposing to codify public availability as a threshold requirement for agency consideration or publication of or reference to a scientific study. Codifying this requirement, when public availability is not a factor that Congress intended U.S. EPA to prioritize, would itself be arbitrary and capricious in violation of the APA. U.S. EPA has also manifestly failed to consider important aspects of the ostensible problem, offered explanations for its proposed decision that run counter to the evidence, and failed to articulate any rational explanation for the proposal, as discussed below under, "The SNPRM Fails to State a Reasoned Basis for the Proposed Regulation."
Finalizing the SNPRM would render subsequent U.S. EPA actions arbitrary and capricious as well. The SNPRM would prohibit or restrict U.S. EPA's consideration of relevant and high-quality data, studies, and models that do not meet U.S. EPA's arbitrary and outcome-seeking proposed requirements for public availability. In excluding relevant science from expert consideration, based on a factor that Congress did not intend for the agency to prioritize, U.S. EPA would inevitably fail to consider important aspects of the problems under consideration, issue decisions counter to the evidence before the agency, and commit other basic APA violations with every affected action. Indeed, the D.C. Circuit Court of Appeals rejected the proposed approach decades ago, agreeing that, "[i]f EPA and other governmental agencies could not rely on published studies without conducting an independent analysis of the enormous volume of raw data underlying them, then much plainly relevant scientific information would become unavailable to EPA for use in setting standards to protect public health and the environment." This instability would also create significant liabilities, and hence reliance risks, for States, regulated entities, and the public.
In addition to its substantive prohibition on arbitrary and capricious actions, the APA establishes general procedural requirements for agency rulemakings, including a requirement for agencies to consider the relevant information presented via public comment on proposed rulemakings. By precluding agency experts from considering relevant studies and models raised in public comments on proposed agency actions, the SNPRM would also cause future U.S. EPA actions to violate the APA's procedural requirements.
Response: Regarding notice and comment on subsequent agency actions, please see response to comment 48-502. The SNRPM is not being finalized as is. As discussed in great detail in the final rule preamble, the final rule makes a number of important changes to the SNPRM. Notably, EPA is finalizing a weighting approach to considering underlying data and models that are not publicly available, not a categorical exclusion. EPA has explained that this procedural rule has limitations on its scope in terms of conflicts with other substantive requirements and that, in situations where this rule applies, EPA will give greater consideration to those studies where the underlying data and models are available for independent validation. That means that the agency will consider all relevant studies when taking covered actions.
Comment [159-1417]: Commenter (12430) writes that under the APA, changes in agency policy positions are permissible only when the agency provides a reasoned justification for the change. Further, "[t]he requirement that an agency provide reasoned explanation for its action would ordinarily demand that it display awareness that it is changing position." U.S. EPA fails to acknowledge that the SNPRM and NPRM comprise a profound change in agency policy position, let alone provide a justification. Instead, the agency struggles to situate the proposals within existing federal policies and guidance on public data availability and peer review. This includes referencing a variety of U.S. EPA and OMB guidance documents dating back to 2002 that, as discussed above, bear little relationship to the current proposals. The SNPRM represents a wholesale departure from past and current U.S. EPA policy with powerful and far-reaching impacts for U.S. public health and the environment. U.S. EPA's failure to justify or even acknowledge its proposed policy change falls far short of the APA's requirements.
Response: EPA disagrees that it has failed to acknowledge a change in agency position or to provide a reasoned explanation for that change. Please see the preamble to the final rule at Section III.A.1 for further discussion of these issues.
Comment [159-1451]: Commenter (12430) writes that the NPRM and SNPRM conflict with many of the statutes that U.S. EPA administers, which include requirements to use, for example, the "best available science," the "best available public health information", the "latest scientific knowledge", or "generally accepted scientific principles or laboratory tests, or appropriately designed and conducted epidemiological or other population studies", among others. U.S. EPA tacitly acknowledges these conflicts in the SNPRM by adding proposed section 30.3, providing, "In the event the procedures outlined in this part conflict with statutes that EPA administers, or their implementing regulations, the statutes and regulations will control." Yet neither this proposed provision nor the SNPRM preamble identify statutory or regulatory provisions with which U.S. EPA believes a conflict could emerge. Moreover, the proposed regulatory text does not provide any parameters for determining whether there is a conflict. Rather than improving the legality and workability of the SNPRM, this provision is impermissibly vague, in violation of the APA, and likely to mire the agency in additional litigation.
Response: Conflict analyses will be conducted and discussed as appropriate when EPA develops subsequent substantive rulemakings implementing this final procedural rule. See the final rule preamble discussion at Section I.A.3. If implementing this procedural rule will result in a conflict with an environmental statute that EPA administers, then this procedural rule will yield.
Comment [179-1609]: Commenter (12688) writes that this proposed policy change would lead the EPA away from its mandate -- in its mission and under various environmental statutes -- to use the best scientific evidence in assessing environmental risks, and its obligation under the APA to refrain from arbitrary decision making. The practical effect of this rule would be to limit EPA's ability to protect public health and meet both its statutory obligations and its overall mission.
Response: Please see response to comment 159-1451.
Comment [179-1790]: Commenter (12688) writes that this proposed policy change would lead the EPA away from its mandate -- in its mission and under various environmental statutes -- to use the best scientific evidence in assessing environmental risks, and its obligation under the APA to refrain from arbitrary decision making. The practical effect of this rule would be to limit EPA's ability to protect public health and meet both its statutory obligations and its overall mission.
Response: Please see response to comment 159-1451.
Comment [205-2125]: Commenter (13788) writes that this rule is not procedural. If finalized, it will have substantial effects on EPA decisions that impact regulated entities. For example, under the CAA's NAAQS program, EPA is required to issue air quality criteria for pollutants that "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such pollutant in the ambient air." If EPA were to ignore or diminish the value of the latest scientific studies because they involve human health data that cannot be made publicly available, that would significantly impact the health-protectiveness of the resulting ambient air quality standards, which would have a substantial effect on both regulatory beneficiaries and regulated industries.
In addition to the Chrysler Court's decision, the D.C. Circuit held in Pickus v. United States Bd. Of Parole, that rules of practice or procedure "should not be deemed to include any action which goes beyond formality and substantially affects the rights of those over whom the agency exercises authority," and that "adherence to congressional purpose counsels a construction of this exemption [for procedural rules] that excludes from its operation action which is likely to have considerable impact on ultimate agency decisions." By restricting EPA's consideration of science in regulatory decision-making based only on the public availability of underlying data, this rule would have a clear and substantial effect on ultimate agency decisions, including decisions that affect the rights of regulated entities, and therefore it is not a procedural rule.
Nor is this a proposed "internal rule of agency procedure" that "exclusively pertains to the internal practices of the EPA." According to details provided to staff of the House Committee on Science, Space, and Technology, EPA's proposed tiered access system would not be implemented internally at EPA, but rather the Agency expects that the bulk of the burden of implementing such a system would fall on the researchers who conduct (or conducted, as this relates to retroactive application of this new policy) the study. According to information shared with committee staff, EPA anticipates that researchers would be responsible for managing the logistics of making data and models publicly available in compliance with the rule, for judging the sensitivity of study data and models and what information can or cannot be made publicly available through tiered access, and what tier of access should be designated for different types of information. It is unclear how EPA would ensure compliance with this rule, or whether after this rule is finalized, it will in effect ban all studies based on confidential human health data. However, what is clear is that this rule would have significant impacts beyond the Agency's internal practices and procedures.
Response: As discussed in the final rule preamble at Section II.C, this is a procedural rule governing EPA's internal affairs. It does not regulate or affect the legal rights and obligations of parties outside of the agency, and therefore is not a substantive rule. An internal rule of agency procedure is not substantive merely because it may indirectly affect the voluntary behavior of parties outside of the agency. See American Hosp. Ass'n v. Bowen, 834 F.2d 1037, 1051 (D.C. Cir. 1987) ("[A]gency rules that impose `derivative,' `incidental,' or `mechanical' burdens upon regulated individuals are considered procedural, rather than substantive."). 
This procedural rule does not impose requirements on external scientists. In fact, it exclusively regulates EPA employees. While it is entirely possible that this procedural rule may influence how scientists choose to conduct their studies or present their research to EPA, that potential incidental effect does not render the rule substantive. 
Comment [205-2128]: Commenter (13788) writes that under the APA and CAA, agency actions that are "arbitrary, capricious, an abuse of discretion, or otherwise not in accordance with law" are to be held unlawful and set aside. Agencies are required to "articulate a satisfactory explanation for [the] action including a `rational connection between the facts found and the choice made.'" Here, however, EPA is proposing to exclude or devalue scientific studies based on criteria that are unrelated to scientific value or merit, and that would eliminate the Agency's ability to rely on research that may be the best available science.
The Agency's newly claimed statutory authority under the housekeeping statute does not save this rule, and actually undermines any transparency justification for the rule. Many of the studies EPA relies on are done outside of the Agency, and therefore must be beyond the reach of the housekeeping statute. The Agency's conflation of transparency in EPA's processes with data availability in scientific research outside the Agency cannot be used to provide satisfactory justification for this rule.
EPA seems to believe that by wielding the terms "validation" and "reanalysis" it shows concern about the quality of studies relied on by the Agency. But EPA's real intention in the SNPRM, as in the Proposal, is to limit the Agency's access to public health science, not to ensure the quality of the studies the Agency relies on. That is made abundantly clear by the Agency's statement that if "multiple high-quality studies exist but only two studies have data and models that are available for independent validation and reanalysis, EPA would only include those two studies as pivotal regulatory science and/or pivotal science." By acknowledging that the highest quality studies will be ignored if confidential data and models are not made available for validation and reanalysis, EPA is admitting that this rule is not concerned with improving the quality of studies relied on by the Agency. The SNPRM is instead laser-focused on information availability, even if it comes at the expense of the quality of scientific work. The best studies may not rely on available underlying data -- precisely because they are human health studies that requires the maintenance of the privacy of the study participants  -- and in those situations this rule would clearly prioritize data availability over quality.
Unlike the 2018 Proposal, the SNPRM does not include any acknowledgement that EPA must use the "best available science" for regulatory actions -- but it must. This lack of concern for quality is evident throughout the SNPRM and is particularly alarming in an agency that relies heavily on science -- as it must, given its statutory mandates. Additionally, the suggestion that the rule could apply to studies retroactively if the Administrator does not grant an exemption further illustrates the intention to undermine longstanding reliance on the human health studies that have done the most to advance air quality and public health. The SNPRM states that the "proposal would apply to reviews of data, models, and studies at the time a rule is developed or influential scientific information is finalized, regardless of when the data and models were generated." EPA has provided no reasoning to explain how retroactive application to past studies -- including the Harvard Six Cities and American Cancer Society particulate matter health studies and subsequent work that links exposures to air pollution with declines and damages to human health -- is justified. It certainly has not provided any information to suggest that this rule would encourage the release of more data from those studies. This proposal's focus is clearly on arbitrarily excluding or diminishing the value of quality scientific research rather than any real commitment to transparency.
Response: EPA disagrees that the true intent of this rulemaking is to diminish the value of quality scientific research. As discussed in the final rule preamble at Section I.A, EPA is taking this action to help strengthen the transparency of the data and models underlying EPA's significant regulatory actions and ISI. 
Regarding the scientific standards that EPA is required to apply, this purely procedural rule does not interpret or apply the provisions of any environmental statutes that EPA administers, and such efforts will occur in the subsequent rulemakings under the relevant statutes. If implementing this procedural rule will result in a conflict with an environmental statute that EPA administers, then this procedural rule will yield. See the final rule preamble at Section I.C.
Comment [205-2164]: Commenter (13788) writes that EPA has failed to provide a meaningful opportunity to review and comment on the proposed rule as required by the APA, because many of the provisions in the SNPRM are described only in vague terms, with very little detail provided about the proposed policies and how they would be implemented. This continued failure to address critical details ignores significant problems that will arise on implementation of a final rule. For example, EPA in the SNPRM proposes to apply its data release requirements across the board, without distinguishing between types of studies and the data they rely on, which may or may not include confidential information. Additionally, while the 2018 Proposal was limited to dose-response data and models, the SNPRM expands that to all data and models broadly and without justification. The SAB's report on the 2018 Proposal stated that:
the requirement to make data available for public inspection will be more easily implemented for some datasets than others (e.g., [certain] studies include individual sample data as part of standard reporting). The expectation that data and methods will be available for all endpoints may be unrealistic.
The SAB was clearly concerned that the rule's requirements could not even be put into practice, never mind whether they are lawful or rational or adequately justified. And, although the SNPRM allows the Administrator to grant an exception to the requirements, the SNPRM provides no reason to believe that provision will sufficiently address the concern identified by the SAB. Nor is there any safeguard included to ensure that the EPA Administrator, in exercising his discretion, will not bring extra-statutory political considerations into the decision-making process.
Response: Please see response to comment 25-96. The SNPRM and final rule address these and many other concerns articulated by public commenters. See 85 FR 15396 and final rule Federal Register notice for this action, respectively.
1.7 Other Authorities (NPRM Comments)
Comment [6835-2704]: Commenter (6857) writes that increased scientific transparency can improve the application of the Endangered Species Act (ESA), specifically in the determination of "critical habitat." Proper determination of critical habitat is a very important issue for state and local governments, as well as businesses located in areas impacted by ESA activity. A determination of critical habitat can literally remove hundreds of miles from the possibility of any type of development. Currently, regulatory agencies can even make this designation based on the "historical" presence of a species, years in the past. In the transportation arena, the critical habitat designation is especially relevant as states promulgate transportation plans years, if not decades, in advance. If a regulatory agency summarily declares an area "off limits" through an overly broad critical habitat designation, then it can unnecessarily jeopardize carefully designed plans for economic development. At a minimum, all economic and scientific analysis necessary for a critical habitat determination should be based on the best data available and incorporate an area's planned transportation improvements.
Response: EPA appreciates your comment but critical habitat determinations under the ESA would be made by other agencies and not EPA. See ESA section 4. 16 U.S.C. § 1533
Comment [6427-1069]: Commenter (6449) writes that the Final Information Quality Bulletin implements the IQA (Pub L. No. 106-554, § 515, Dec. 21, 2000) This rule should reference the IQA along with the bulletin.
Response: We cite OMB's bulletin because it interprets terminology in the IQA that Congress left undefined. The IQA directed OMB to issue guidelines for use by federal agencies in developing their own information quality guidelines, which EPA did in 2002. 
Comment [4921-366]: Commenter (4932) recommends that EPA should clarify that nothing in the Transparency Proposal is intended to diminish the Agency's existing legal authority to enhance the transparency of EPA's regulatory science.
The Transparency Proposal omits discussion of EPA's ample existing legal authority to enhance the transparency of the Agency's regulatory science, at least as it relates to public health research sponsored by the federal government. As set forth in the Office of Management and Budget's guidance for financial assistance to non-Federal entities (2 C.F.R. § 200.315), federal agencies have unfettered legal authority to "[o]btain, reproduce, publish, or otherwise use the data produced under a Federal award," or to "[a]uthorize others to receive, reproduce, publish, or otherwise use such data for federal purposes." In addition, any public health research data (a) produced under a Federal award, and (b) used by the Federal Government in developing agency action that has the force and effect of law, must be released to the public if a request for the data is made pursuant to the FOIA.
Because the federal government has sponsored a substantial majority of the public health research conducted in the United States over the past 50 years, the federal government is well positioned, using existing legal authority, to facilitate release of public health research data if it is inclined to do so as a matter of policy. MIG recommends that EPA clarify in the final regulation that the Transparency Proposal's omission of these existing authorities was not intended, nor should it be interpreted in any fashion, to diminish the Agency's existing legal authority to enhance the transparency of EPA's regulatory science. Further, the scope of EPA's authority noted above should be identified in the Transparency Policy as an additional source of support.
Response: EPA agrees that there are existing regulations that govern the ownership and use of data generated with federal funding. Because the final rule is one of internal agency procedure, it does not impact or diminish those authorities. Rather, as EPA states in the preamble, "[t]his rule builds upon prior EPA actions in response to government-wide data access and sharing policies" such as the regulations cited by the commenter. The focus of § 30.5 is how the Agency will handle pivotal science based on the availability for independent validation of the underlying dose-response data. Where the data is available, the pivotal science will be given greater consideration. Where it is not available for reasons that may include confidentiality and privacy protections, the pivotal science will be given lesser consideration unless an Administrator's exemption is granted under § 30.7.
Comment [6169-2818]: Commenter (6179) states that EPA is authorized by Congress to promulgate internal housekeeping regulations like the transparency proposal.
The Transparency Proposal relies upon authority under various environmental statutes, including Clean Air Act sections 103 and 301(a). EPA solicits comment on whether additional or alternative sources of authority are an appropriate basis for the Transparency Proposal.
In addition to those statutes cited in the Transparency Proposal, we note EPA's authority under 5 U.S.C. § 301. More commonly known as the "Federal Housekeeping Statute," this provision authorizes the "head of an Executive department" to "prescribe regulations for the government of his department, the conduct of its employees, the distribution and performance of its business, and the custody, use, and preservation of its records, papers, and property."
The Federal Housekeeping Statute authorizes the Transparency Proposal  -  a regulation for the government of EPA and the performance of its business  -  on a general basis, similar to how the environmental statutes cited by EPA do so specifically. As seen in the Federal Housekeeping Statute, Congress expected that regulations like the Transparency Proposal would be so commonplace within agencies that it provided a general federal government-wide authorization for their promulgation. We recommend EPA to consider the Federal Housekeeping Statute as additional support for the Transparency Proposal.
Response: EPA appreciates your comment in support of the rule. EPA is relying on its authority to promulgate housekeeping regulations as legal authority for this rule and not the Federal Housekeeping Statute. Also see the response to Comment 31-135. 
Comment [6334-3154]: Commenter (6356) expresses that EPA's proposal is consistent with Congressional directives that EPA is to utilize the "best science." In the Notice of Proposed Rulemaking at 18,771/col.2, EPA requests additional authority for the proposed regulation. The federal "Information Quality Act," provides additional authority for the proposed regulation. Pub. L. 106 - 554 (2001), codified at 44 U.S.C. §§ 3504(d)(1), 3516. While numerous other federal agencies have adopted guidelines implementing the 2001 law, EPA is conspicuous because it has not adopted such guidelines. 
Response: EPA disagrees that the Agency hasn't adopted information quality guidelines. EPA issued its guidelines in 2002. See https://www.epa.gov/quality/guidelines-ensuring-and-maximizing-quality-objectivity-utility-and-integrity-information. Also see the response to Comment 6427-1069.
Comment [6353-2324]: Commenter (6375) notes that EPA solicits comment on whether additional or alternative sources of authority are appropriate bases for this proposed regulation. (pg. 18,771). EPA's statement concerning its legal authority could be enhanced by reference to statutory provisions that require it to rely on "accurate," "useful," or "best" data. For example, section 108(a)(2) requires that the Administrator issue air quality criteria that "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health and welfare . . .." 42 U.S.C. § 7408(a)(2). Similarly, section 304 of the CWA requires water quality criteria that "accurately reflect [] the latest scientific knowledge . . .." 33 U.S.C. § 1314(a)(1). The SDWA requires the Administrator to assess risk using "the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices." 42 U.S.C. § 300g1(b)(3)(A). Increasing the transparency of regulatory science and ensuring that it is reproducible helps in evaluating its accuracy and usefulness.
Furthermore, making data and methods underlying regulatory decisions publicly available is not new. Guidelines issued by both the OMB and EPA implementing the IQA, Treasury and General Government Appropriations Act for Fiscal Year 2001, § 515, P.L. No. 106-554, 114 Stat. 2763 (Dec. 21, 2001), emphasize the importance of reproducibility in assessing the quality and utility of data used in making regulatory decisions. It appears the OMB's IQA guidelines recognize that the reproducibility of underlying science helps to ensure the integrity of agency decisions and explain that "[m]aking the data and methods publicly available will assist in determining whether analytic results are reproducible." 67 Fed. Reg. 8452, 8460 (Feb. 22, 2002) (OMB IQA Guidelines).
EPA's IQA guideline states: "A higher degree of transparency about data and methods will facilitate the reproducibility of such information by qualitied third parties, to an acceptable degree of imprecision. . .. It is important that analytic results for influential information have a higher degree of transparency regarding (1) the source of the data used, (2) the various assumptions employed, (3) the analytic methods applied, and (4) the statistical procedures employed."
EPA should also consider replacing some of the citations it provides for its legal authority with others that might be more relevant. For example, the Agency may want to cite 42 U.S.C. § 6981, on research, instead of 42 U.S.C. § 6979, which concerns "Labor standards." In addition, section 115 of CERCLA, 42 U.S.C. § 9615, which authorizes regulations, might be a more appropriate citation than section 116 of that Act, 42 U.S.C. § 9616, which specifies schedules for assessment and evaluation of facilities subject to the Act's requirements.
Response: EPA is not relying on any of the environmental statutes cited in the 2018 and 2020 proposal as legal authority for the rule. EPA acknowledged that there were errors in a few of the citations in the 2018 proposal and corrected them in the 2020 supplemental proposal. EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In the final rule, EPA does not interpret or apply provisions of a particular statute or statutes that it administers. EPA will undertake such efforts in forthcoming substantive actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. 
EPA agrees with the commenter that the IQA provides support for the final rule. That is why the Agency has referenced both the OMB and EPA IQA guidelines, which interpret the IQA, in the rule's preamble.
Comment [6353-2331]: Commenter (6375) notes that EPA solicits comment on whether alternative or additional regulatory or other policy vehicles are appropriate to establish and implement these policies, and whether further regulatory or other policy vehicles at the programmatic or statutory level would be appropriate as alternative or additional steps the agency may take to further the policies articulated in Section II. (pg. 18771 of the NPRM FR notice).
The regulatory language that EPA proposes provides general requirements that apply to the Agency's use of science under the many statutes that it implements. The generality of the proposed regulations may be a function of the diverse purposes of those statutes Specifically, some of these statutes focus on the introduction of new substances into the marketplace (e.g., FIFRA and TSCA), while others primarily concern regulation of emissions or releases to (e.g., CAA and CWA), or removal from (e.g., CERCLA), the environment. The scientific analyses required for these different programs and the sources of the scientific information underlying those analyses differ. The Agency should consider supplementing the general language that it has proposed with statutory specific regulations. A statutory-specific rulemaking would provide greater certainty going forward.
Response: EPA agrees that this rule should be supplemented with statute-specific transparency regulations. EPA intends to undertake such rulemakings beginning in 2021.
Comment [6851-3717]: Commenter (6873) writes that the authority to make data public to the extent envisioned in this proposal, may not lie with the EPA.
This proposal concludes that the EPA has appropriate sources of statutory authority for this regulation; you have asked for comment on whether additional or alternative sources of authority are appropriate. The concern raised by some in our group is not necessarily whether the EPA's authority is appropriate or complete, but whether it is realistic and sufficient to affect the kind of changes you're looking for. It is understood that your proposal is to modify EPA practices, not science publishing practices, but the impact on science publishing -- or at least the collision it sets up with existing publishing practices -- will be significant. As you know, most US research funding comes from NIH, NSF, DoD, DOE and NASA. The researchers who receive funding from these agencies adhere to the publishing policies set forth by these agencies -- which include open and transparency policies -- which in turn work in parallel with the US public access program policies managed by OSTP, and with a wide variety of publishing, open and transparency policies put forth by universities, publishers, private funders, foreign governments, and other groups. It's unclear how the EPA will be able to influence this existing symphony of public and private regulation, at least for the research it doesn't fund. The EPA certainly has a right to try, but without broad agreement amongst other stakeholder groups, it's unlikely that any one stakeholder in this space will be able to promulgate or enforce regulations that will have such a broad impact. 
Response: EPA disagrees that the final rule will impact science publishing practices. This rule governs internal agency procedures and does not obligate or impact any entity besides EPA. The rule governs how EPA will handle pivotal science based on the availability of the underlying dose-response data. This final rule does not regulate the rights and obligations of any party outside of the EPA let alone have legal force and effect on them. Any incidental impacts on voluntary behavior outside of the EPA do not render this a substantive rule.
Comment [6873-1710]: Commenter (6895) writes that EPA incorrectly cites various authorities in an attempt to use an unsupported claim of lack of transparency as a means to justify the proposed rule, when in fact those authorities make no such findings. For example, in American Trucking Associations v. EPA 283 F.3d 355, (DC Cir., 2002), the court found that EPA did not violate procedural rights in relying on scientific studies rather than raw data. EPA itself argued that [i]f EPA and other governmental agencies could not rely on published studies without conducting an independent analysis of the enormous volume of raw data underlying them, then much plainly relevant scientific information would become unavailable to EPA for use in setting standards to protect public health and the environment...[S]uch data are often the property of scientific investigators and are often not readily available because of ...proprietary interests...or because of [confidentiality] arrangements [with study participants].
Response: EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, EPA does not rely on, interpret or apply provisions of a particular statute or statutes that it administers. Nothing in the final rule will prohibit EPA from relying on studies where the underlying dose-response data are not publicly available. Nor does the rule require that independent validation occur before EPA can use any studies.
Comment [6339-1801]: Commenter (6361) writes that the proposed HONEST Act, the Secret Science Reform Act, and this proposed rule all directly contradict a 2002 ruling and a 2010 ruling from the D.C. Circuit Court of Appeals which states "[w]e agree with EPA that requiring agencies to obtain and publicize the data underlying all studies on which they rely 'would be impractical and unnecessary". American Trucking Associations, Inc. v. EPA, 283 F.3d 355, 372 (D.C. Cir. 2002) (citation omitted); see also Coalition of Battery Recyclers Assoc. v. EPA, 604 F.3d 613, 622-23 (D.C. Cir. 2010) (rejecting claim that EPA was arbitrary and capricious for not making public data from its studies).
Response: EPA did not finalize the categorical exclusion approach from the 2018 proposal. EPA will consider all relevant studies when developing significant regulatory decisions and influential scientific information and will use the availability of dose-response data in determining the weight to assign to the subset of studies identified as pivotal science. The Agency will provide the basis for giving studies lesser consideration in the record for the rulemaking or in the documentation for the influential scientific information. This would provide an opportunity for public comment on the rationale for how EPA handled pivotal science. Further, the final rule does not require EPA to obtain and publish data underlying the studies it uses to support significant regulatory actions and influential scientific information. 
Comment [6161-1514]: Commenter (6131) writes that the proposed HONEST Act, the Secret Science Reform Act, and this proposed rule all directly contradict a 2002 ruling and a 2010 ruling from the D.C. Circuit Court of Appeals which stated "[w]e agree with EPA that requiring agencies to obtain and publicize the data underlying all studies on which they rely `would be impractical and unnecessary'". American Trucking Associations, Inc. v. EPA, 283 F.3d 355, 372 (D.C. Cir. 2002) (citation omitted); see also Coalition of Battery Recyclers Assoc. v. EPA, 604 F.3d 613, 622-23 (D.C. Cir. 2010) (rejecting claim that EPA was arbitrary and capricious for not making public data from its studies).
Response: See the response to Comment 6339-1801.
Comment [5154-419]: Commenter (5165) states that the U.S. Court of Appeals for the District Columbia Circuit has affirmed EPA's use of non-public data in support of NAAQS, and in so doing it characterized as "persuasive" EPA's approach to data availability, which the court quoted as follows:
      If EPA and other governmental agencies could not rely on published studies without conducting an independent analysis of the enormous volume of raw data underlying them, then much plainly relevant scientific information would become unavailable to EPA for use in setting standards to protect public health and the environment. [S]uch data are often the property of scientific investigators and are often not readily available because of ... proprietary interests ... or because of [confidentiality] arrangements [with study participants].
Now, EPA indicates that it intends to reverse this policy and adopt one that would expressly preclude it from using studies based on such "non-public data." It is inappropriate for EPA to undertake such a consequential policy change without explaining why it believes the concerns it expressed above are incorrect or no longer valid.
Response: See the response to Comment 6339-1801.
Comment [6163-1116]: Commenter (6133) writes that no case law supports the proposal.
The Proposal "directs EPA to ensure that the regulatory science underlying its actions is publicly available in a manner sufficient for independent validation." 83 Fed. Reg. at 18,773. However, the Proposal fails to identify a single court decision supporting an agency's decision to bar itself from considering relevant studies or information on the grounds that underlying data are not "publicly available in a manner sufficient for independent validation," where such a requirement is not statutorily imposed. Indeed, EPA in the entire Proposal only cites two cases related to this question, and EPA admits, as it must, that both cases "upheld EPA's use (sic) nonpublic data in support of its regulatory actions." Id. at 18,769 n.3 (citing Coal. of Battery Recyclers Ass'n v. EPA, 604 F.3d 613, 623 (D.C. Cir. 2010) & Am. Trucking Ass'ns v. EPA, 283 F.3d 355, 372 (D.C. Cir. 2002)). 
Footnote 3 in the Proposal contains two noteworthy, albeit unintended, indictments of the approach proposed by EPA. First, footnote 3 states that "[h]istorically, EPA has not consistently observed the policies underlying this proposal." Tellingly, EPA does not and cannot identify even one example in which EPA has observed the policies underlying the Proposal. Our research, to the contrary, has identified no instance in which EPA has followed the policies underlying the Proposal, to bar EPA from considering relevant studies or science submitted by the public or gathered by EPA, on the grounds that the underlying data are not "publicly available in a manner sufficient for independent validation." 
Second, footnote 3 implies that there are instances where EPA's use of non-public data in support of its regulatory actions was rejected by a court. See id. ("courts have at times upheld EPA's use (sic) non-public data in support of its regulatory actions.") (emphasis added). Again, the Proposal does not and cannot cite a single court decision that failed to uphold use of nonpublic, relevant science or studies relied on by EPA or any other federal or state agency in support of its regulatory actions. Id. Our research also failed to identify a single instance in which a court failed to uphold an agency's use of non-public, relevant science or studies by an agency, after that practice was challenged by commenters or petitioners in court. 
Of course, in both the cases that the Proposal cites in footnote 3, the D.C. Circuit Court of Appeals refused to prohibit EPA from considering non-public data. In American Trucking, the court declined to "impose a general requirement that EPA obtain and publicize the data underlying published studies on which the Agency relies," holding that the "Clean Air Act imposes no such obligation." 283 F.3d at 372. The court agreed with EPA that "requiring agencies to obtain and publicize the data underlying all studies on which they rely would be impractical and unnecessary." Id. (quoting EPA in Particulate Matter NAAQS, 62 Fed. Reg. at 38,689).
The D.C. Circuit reaffirmed this holding in its 2010 decision, Coalition of Battery Recyclers, in which the court reiterated that requiring publication of all data underlying studies would be impractical and unnecessary, and was not required by the Clean Air Act. 604 F.3d at 623. EPA in the Proposal utterly fails to explain or demonstrate why its proposed, self-imposed restriction would be any less impractical or unnecessary than those it previously opposed on these grounds. This failure to explain, failure to offer any convincing counterproof, and failure to explain the agency's reversal of its positions in American Trucking and Coal. Of Battery Recyclers Ass'n provide independent grounds for finding EPA's Proposal arbitrary and capricious and an abuse of discretion. Similarly, the Proposal does not identify any case law supporting EPA's claimed ability to "exercise its discretionary authority to establish a policy that would preclude it from using such [non-public] data in future regulatory actions." 83 Fed. Reg. at 18,769 n.3. Our research failed to identify any case in which the courts allowed an agency to categorically bind itself from considering relevant, peer-reviewed science, or otherwise valid studies or evidence, because the underlying data was not made publicly available. Cf., e.g., Southwest Airlines Co. v. Tr. Sec. Admin., 554 F.3d 1065, 1074 (D.C. Cir. 2009) (holding TSA was not required to disclose to airline companies the underlying data file used in a GAO report that informed TSA's calculation of security fees given the nature of the decision -- which was industry-wide rather than an adjudicative decision -- and the deference given to agency denials of discovery); Pharm. Research and Mfrs. v. FTC, 790 F.3d 198, 210 - 11 (D.C. Cir. 2015) (holding the FTC was not required to disclose the 66 individual filings underlying its decision to target only the pharmaceutical industry in a new rule because the filings were confidential, were used as a general source of background in the rulemaking process, and were exempted from disclosure by statute); State Corp. Comm'n of Kan. v. FERC, 876 F.3d 332, 335 - 36 (D.C. Cir. 2017) (holding FERC was justified in relying on a study used by the agency to assess the benefits of a power facilities merger, even though the study was objected to by Kansas on the grounds that the study was performed by a third party and its results could not be verified by Kansas. The court rejected Kansas's objections to the study because Kansas had access to a redacted electronic version of the study, though not the underlying data; Kansas did not pinpoint a specific reason to question the study, and the study's assumptions and results had been reviewed for reasonability.) 
Under some circumstances, the D.C. Circuit has upheld an agency's decision to exclude an individual piece of evidence from the decision-making process. In API v. EPA, the D.C. Circuit upheld the EPA's decision to discount a published meta-analysis that ran counter to the rule ultimately adopted. 684 F.3d 1342, 1350 (D.C. Cir. 2012). There, EPA considered the study but discounted its results after "[finding] its methodology wanting." The court found the EPA decision to discount the study was not arbitrary and capricious because EPA had not "entirely failed to consider an important aspect of the problem [or] offered an explanation for its decision that runs counter to the evidence before the agency." Id. (quoting North Carolina v. EPA, 531 F.3d 896, 906 (D.C. Cir. 2008)). Critically, EPA did consider the study (unlike the censorship approach in the instant Proposal). Moreover, following consideration, the agency offered specific reasons for not relying on the study, including its disagreements with the methodology. Id.
Likewise, the D.C. Circuit found in Intercollegiate Broadcasting System v. Copyright Royalty Board, that the Copyright Royalty Board had "properly excluded" from evidence a reference to a survey because the survey itself was not entered into evidence and could not be verified. 796 F.3d 111, 129 (D.C. Cir. 2015). In both cases, the court yielded to an agency's discretion to exclude a particular piece of information where the agency had made an individualized determination about the source. None of these cases support the Proposal's categorical ban on EPA considering relevant data, science, or studies (where data are not "publicly available in a manner sufficient for independent validation"), that have been submitted to the agency and that have not been the subject of any individualized determination that the studies or information are flawed or erroneous.
In its Proposal, EPA proposed to categorically ignore and exclude all peer-reviewed research with non-public underlying data, without individually considering each study or offering specific reasons for not relying on that study. The Proposal, by barring consideration of foundational scientific research premised upon non-public data, would result in EPA "fail[ing] to consider an important aspect of the problem." API, 684 F.3d at 1350. There is no evidence of a court supporting an agency's decision to exclude entire categories of evidence, or studies or information based on categorical prohibitions like the ones in the Proposal, without considering the source and offering specific reasons for not relying on the study. Instead, both EPA and the Courts have indicated already in API and Coalition of Battery Recyclers, that a rule like the one EPA is currently proposing is not required by the Clean Air Act and would be both impractical and unnecessary. This Proposal runs counter to the D.C. Circuit's decision in API and would render EPA's regulatory actions based on the Proposal arbitrary and capricious and an abuse of EPA's discretion. The Proposal's blanket rule would represent a significant and unlawful departure from D.C. Circuit rulings on agencies' limited discretion to choose the sources it will consider and ignore.
Response: See the response to Comment 6339-1801.
Comment [6157-901]: Commenter (6127) writes that this is a rejection of the scientific mandate and approach of the agency as mandated in the Environmental Protection Act, including multiple court precedents in favor of the EPA in the use of non-public data for regulatory purposes including Coalition of Battery Recyclers Assn. v. EPA, 604 F.3d 613, 623 (D.C. Cir. 2010) and American Trucking Assn's v. EPA, 283 F.3d 355, 372 (D.C. Cir. 2002).
Response: EPA has no knowledge of the Environmental Protection Act referenced in this comment. Nothing in this rule prohibits EPA from using non-public data for regulatory purposes. Also see the response to Comment 6339-1801. 
Comment [1425-4482]: Commenter (1440) suggests that the authorities listed for promulgating this regulation do not make sense. (See "Authority" section for proposed 40 CFR part 30 at 83 FR 18773). For example, the proposed regulations cite to 42 U.S.C. 6912(a)(1) and 42 U.S.C. 6979 under the RCRA as the authority for EPA to promulgate them. There appears to be no basis to such a citation.
42 U.S.C. 6912(a)(1) does give the Administrator the authority to promulgate regulations "as are necessary to carry out his functions under this chapter." However, the "functions" this section refers to involve activities "to promote the protection of health and the environment and to conserve valuable material and energy resources." The proposal provides no justification for how the new regulation would accomplish this goal. Depending exactly on how the vaguely worded provisions are implemented, the proposed regulation could in fact be in direct opposition to the goals of the RCRA statute.
Even stranger is the citation to 42 U.S.C. 6979, which discusses labor standards that must be followed for construction grants made under RCRA and does not appear even tangentially related to the proposed regulation.
While in general the goal of increasing transparency of EPA regulatory science is a laudable one, it is not one that Congress has given EPA explicit authority to promulgate administrative rules to carry out. Since administrative rules have the effect and force of law, they must have a clear basis before EPA can promulgate them. The Agency would better accomplish this goal through developing internal policies rather than attempting a change to the regulation.
Response: EPA is not relying on any of the environmental statutes cited in the 2018 and 2020 proposal as legal authority for the rule. EPA acknowledged that there were errors in a few of the citations in the 2018 proposal and corrected them in the 2020 proposal. This rule does not change any existing EPA regulations. It only applies prospectively to significant regulatory actions and influential scientific information, based on dose-response data, that are developed after the effective date of the rule. 
Comment [2393-4702]: Commenter (2408) writes that this effort to exclude important environmental and public health studies while privileging industry-sponsored research is clearly contrary to the public interest as well as to EPA's legislative mandates.
Response: The rule does not require the exclusion of any studies from consideration when the Agency develops significant regulatory actions and influential scientific information. All relevant studies, including industry research will be evaluated using study quality factors in § 30.5. 
Comment [5173-455]: Commenter (5184) notes that the TSCA story provides further evidence that NEPA requires EPA to prepare a thorough EIS before it adopts the proposed regulation.
Response: NEPA does not apply to this rulemaking because the rule is not a "major Federal action significantly affecting the quality of the human environment." NEPA section 102(2)(c); 42 USC § 4332(2)(c). The rule is one of internal agency procedure that describes the consideration the agency will give to studies based on data availability and the factors EPA uses to evaluate studies that may be used for regulatory purposes and influential scientific information and, as such, will not affect the quality of the human environment.
Comment [6112-1612]: Commenter (6137) contends that administrative statutes prohibit the proposed prohibitions.
EPA's Proposed Rule likewise ignores requirements set forth in two administrative statutes that govern its rulemaking: the Regulatory Flexibility Act and the Data Quality Act. Under the Regulatory Flexibility Act, 5 U.S.C. § 601 et seq., federal agencies must consider the impacts their regulations will have on small entities and must consider less burdensome alternatives. In developing a new regulation, an agency must take one of two actions: certify that a proposed rule will not have a significant economic impact on a substantial number of small entities, or prepare an initial regulatory flexibility analysis. EPA must publish its initial regulatory flexibility analysis or a summary of it in the Federal Register along with the proposed rule. Here, EPA includes a certification with the Proposed Rule stating that it will not have a significant economic impact on a substantial number of small entities. However, it provides no support for this certification. 
Pursuant to the DQA, also known as the IQA, Treasury and General Government Appropriation Act for Fiscal Year 2001, Pub. L. No. 106-554, § 515 Appendix C, 114 Stat. 2763A-153 (2000), the OMB issued government-wide guidelines that "provide policy and procedural guidance to Federal agencies for ensuring and maximizing the quality, objectivity, utility, and integrity of information (including statistical information) disseminated by Federal agencies. . . ." The OMB guidelines directed each federal agency to issue its own information quality guidelines to "ensure and maximize the quality, objectivity, utility, and integrity of information, including statistical information, disseminated by the agency." 67 Fed. Reg. 8451, 8459 (Feb. 22, 2002). Following OMB's instructions, EPA issued its own guidelines that apply to information it disseminates to the public. EPA, Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity, or Information Disseminated by the Environmental Protection Agency (Oct. 2002), https://www.epa.gov/sites/production/files/2017-03/documents/epa-info-quality-guidelines.pdf. According to these guidelines, EPA uses a "weight-of-evidence" approach that "considers all relevant information and its quality, consistent with the level of effort and complexity of detail appropriate to a particular risk assessment." EPA Guidelines 6.4 (emphasis added). The Proposed Rule contravenes these requirements, as it will prevent EPA from considering all relevant information by precluding consideration of certain data. 
Moreover, pursuant to EPA's guidelines, EPA must ensure that the information it disseminates "is accurate, reliable and unbiased." EPA Guidelines 6.4(A). To do so, it uses: 
      the best available science and supporting studies conducted in accordance with sound and objective scientific practices, including, when available, peer reviewed science and supporting studies; and (ii) data collected by accepted methods or best available methods (if the reliability of the method and the nature of the decision justifies the use of the data).
Id. Given that the Proposed Rule eliminates from consideration any scientific study where the underlying data cannot be made publicly available, it undoubtedly precludes the use of the best available science in certain situations. Thus, the Proposed Rule is inconsistent with EPA's implementation of the DQA.
Response: EPA disagrees that the final rule is inconsistent with the Regulatory Flexibility Act (RFA) or the DQA (also known as the IQA). The final rule is a rule of internal agency procedure. It does not govern or apply to any entity other than EPA. Therefore, it will not result in impacts to small entities. The final rule does not include the categorical exclusion proposed in 2018. Under § 30.5, EPA will evaluate all relevant studies when developing significant regulatory actions and influential scientific information. Studies considered pivotal science will then be weighed based on the availability of the underlying dose-response data. 
Comment [6112-1620]: Commenter (6137) contends that the proposed rule violates Public Law 95-622 and the common rule for research involving human subjects.
EPA's Proposed Rule also conflicts with Public Law 95-622 and the interagency regulations on testing of human subjects required by that law. In 1978, Congress created a President's Commission for the Study of Ethical Problems in Medicine and Biomedical and Behavioral Research (the "Commission") and directed the Commission to study the "protection of human subjects," defined to include the "health, safety, and privacy of individuals." 42 U.S.C. § 300v - 1(b)(2), (f)(2) (emphasis added). This included a study of and issuance of recommendations concerning, among other subjects, procedures and mechanisms "to safeguard the privacy of human subjects of behavioral and biomedical research, [and] to ensure the confidentiality of individually identifiable patient records." Id. § 300v - 1(a)(1)(E), (a)(4) (emphasis added). 
In response to this charge, in 1981, the Commission recommended that all federal agencies adopt uniform regulations concerning the protection of human research subjects. See 47 Fed. Reg. 13,272 (March 29, 1982). Accordingly, in 1991, EPA and 13 other federal departments and agencies adopted uniform regulations on this issue (the "Common Rule"), see 56 Fed. Reg. 28,003 (June 18, 1991), requiring that all research involving human subjects that is "conducted, supported or otherwise subject to regulation by any Federal department or agency" be reviewed and approved by an institutional review board ("IRB"). 40 C.F.R. § 26.101(a). In order to approve research involving human subjects, an IRB must ensure that "there are adequate provisions to protect the privacy of subjects and to maintain the confidentiality of data." Id. § 26.111(a)(7). 
In particular, the Common Rule prohibits EPA from relying on research that is "deficient relative to the ethical standards prevailing at the time the research was conducted in a way that placed participants at increased risk of harm . . . or impaired their informed consent." Id. §§ 26.1704(b)(2), 26.1705(b). It recognizes that the protection of private information of the subject is central to a subject's ability to provide informed consent. See generally id. § 26.116. And the protection of private information is central to the Common Rule more broadly  -  one of only two situations under which IRBs may waive the requirement to obtain written consent is if the release of the consent form risks linking the subject to the research. Id. § 26.117(c)(1). 
EPA's Proposed Rule is thus antithetical to the confidentiality requirements of the Common Rule. It would ignore relevant health science absent publication of personal data that is required to be kept confidential under the Common Rule. To enable EPA to consider their studies in a relevant rulemaking, IRBs would have to make the underlying data publicly available, which would place them at risk of termination of federal funding, termination of ongoing studies, denial of approval for new studies, or disqualification of the IRB or its parent institution. 40 C.F.R. Part 26 subpart O & §§ 26.103, 26.123, 26.1123.
Response: EPA disagrees that the rule is inconsistent with the Common Rule confidentiality requirements. The final rule does not require disclosure of information subject to statutory or regulatory limitations on disclosure such as those for confidentiality, privacy, and trade secrets. EPA did not finalize the categorical exclusion of studies from the 2018 proposal. All relevant studies will be evaluated and considered by the Agency in developing significant regulatory decisions and influential scientific information.
Comment [6125-2629]: Commenter (6150) agrees that EPA has sufficient legal authority under existing environmental statutes to issue this rulemaking to strengthen the transparency of the science supporting regulatory activity. EPA lays out substantive legal support for this action in the Proposed Rule. Further, EPA's obligation under the DQA is to "maximiz[e] the quality, objectivity, utility, and integrity of information" and, as importantly, "establish administrative mechanisms allowing affected persons to seek and obtain correction of information maintained and disseminated by the agency." These provisions can only be met if the public has access to the data, models, and methodologies used by EPA and underlying research used by EPA to justify new and existing regulations.
Response: EPA agrees that the Agency has legal authority to promulgate this rule. However, EPA decided to rely on its housekeeping authority, and not on any of the substantive environmental statutes, as authority for the rule. 
Comment [6156-1322]: Commenter (6126) states that the burden on researchers is unassessed, but relevant for implementation.
EPA concluded in the proposed rule that "This action does not contain any information collection activities and therefore does not impose an information collection burden under the [Paperwork Reduction Act (PRA)]." However, EPA may have failed to conduct an important prospective analysis of the proposed rule necessary for public commenters to understand the implications on data quality, burden, and privacy implications related to data uses.
The PRA provisions for developing Information Collection Requests (ICRs) and corresponding review and public comment procedures may apply to this draft regulation if EPA intends to collect data from researchers in a consistent way. A key purpose of the PRA is to assess the burden of individuals providing information to the government and to ensure public benefit of that information. PRA requirements apply for "collection of information" which is defined to include "obtaining...facts or opinions by or for an agency, regardless of form or format, calling for...(i) answers to identical questions posed to...ten or more persons..." To the extent EPA anticipates a regulatory action that would routinely encourage voluntary submission of information to EPA as a normal matter with a consistent request established by the proposed rule, the PRA could apply.
Determination about the appropriate application of the PRA is an important issue because the processes of the PRA's ICR reviews provide an assessment of the burden of government information collection. The PRA is also an important tool for transparency about confidentiality and privacy protections, because it offers additional notice and comment periods that inform the public and offer executive branch oversight of agency information collection activities.
Minimally, consideration should be given to burdens the proposed rule would impose on researchers, including those whose research objectives may not have a stated relationship to a regulation or policy issue under consideration. Such analysis of the burden imposed by the regulation could be considered as part of the PRA requirements.
Response: Nothing in the final rule requires the collection of information by EPA or obligates outside parties to take any actions, including providing information to EPA. Therefore, there are no burdens on third parties that EPA should have analyzed. See Section III.J. for a discussion of costs and benefits of the final rule. The final procedural rule describes how EPA will handle studies used to support significant regulatory decisions and influential scientific information. In determining how to weight studies determined to be pivotal science, EPA will assess the availability of dose-response data for those studies. Pivotal science where the underlying dose-response data are available for independent validation will be given greater consideration than pivotal science lacking data availability.
Comment [6156-1326]: Commenter (6126) notes that EPA specifically asked for comment about how the rule can best be implemented in "light of existing law and prior federal policies..." Further review is likely needed to determine how the law would apply in conjunction with the IQA, the Privacy Act of 1974, as amended, and numerous other statutes and policies that encourage the use of data with appropriate privacy and confidentiality protections. For example, IQA's two brief provisions collectively encourage agencies to follow OMB guidance in issuing guidelines for "maximizing the quality, objectivity, utility, and integrity of information...disseminated by the agency." However, EPA's proposed rule does not appear to reference the IQA or suggest its direct relevance, notwithstanding the implication of substantial overlap with the goals of the proposed regulation.
Response: EPA cited OMB's guidelines because it interprets terminology in the IQA that Congress left undefined. EPA also references its own information quality guidelines. EPA's implementation of the final rule will be consistent with the Agency's obligations under the statutes cited by the commenter because the final rule does not require that the Agency disseminate information that is subject to limitations on disclosure such as confidentiality and privacy. 
Comment [6161-1509]: Commenter (6131) writes that it is telling that there have been attempts in Congress to enact legislation regarding the issue of data confidentiality as those legislative efforts demonstrate that the executive branch lacks the authority to undertake a rulemaking to address these issues.
Response: EPA agrees that final rule is similar to several provisions of the HONEST Act and may have be motivated by similar concerns. However, EPA disagrees that it lacks the authority to promulgate this rule in the absence of congressional action. The agency has housekeeping authority to promulgate rules that will govern the internal procedures EPA uses to evaluate the science underlying its significant regulatory actions and influential scientific information. EPA could have revised its internal guidance without notice and comment rulemaking but doing so allowed the agency to inform the public of its efforts to increase transparency of the science the agency relies upon and to help facilitate independent validation of the science. 
Comment [6425-2369]: Commenter (6447) suggests that EPA impermissibly failed to notify or seek input from its SAB with regard to the proposal.
As the SAB noted in its May 12, 2018 memo, EPA staff violated the Environmental Research, Development, and Demonstration Authorization Act by failing to inform the SAB about the current proposal. According to the SAB, it only became aware of the proposal after seeing it in the Federal Register and in the news. This is a flagrant violation of law, and of reasoned decision-making. As the SAB notes, "[t]he proposed rule deals with issues of scientific practice and proposes constraints that the agency may apply to the use of scientific studies in particular contexts. As such, this rule deals with a myriad of scientific issues for which the Agency should seek expert advice from the Science Advisory Board."
The fact that EPA failed to obtain scientific support for its proposal shows that the proposal has no scientific basis, is ideologically motivated, and is arbitrary and capricious.
Response: EPA disagrees that it violated the Environmental Research, Development, and Demonstration Authorization Act (ERDDAA). EPA did seek and obtain SAB review of aspects of the proposed rule. See https://yosemite.epa.gov/sab/sabproduct.nsf/LookupWebProjectsCurrentBOARD/2DB3986BB8390B308525855800630FCB/$File/EPA-SAB-20-005.pdf. 
Comment [6895-3139]: Commenter (6916) writes that to the extent that the Proposal would require release of the underlying confidential data as a prerequisite for EPA to be able to use a federally funded study in making a rulemaking decision, it also is in conflict with the regulations implementing the FOIA. Those regulations, following the 1998 Shelby Amendment to the 1999 Supplemental Appropriations Act, provide access through FOIA to federally supported studies' "research data"  -  so long as its confidentiality is preserved. The Shelby Amendment had directed changes to OMB Circular A-110 and FOIA. The implementing rules state that:
      [i]n response to a Freedom of Information Act (FOIA) request for research data relating to published research findings produced under a Federal award that were used by the Federal government in developing an agency action that has the force and effect of law, the Federal awarding agency must request, and the non-Federal entity must provide, within a reasonable time, the research data so that they can be made available to the public through the procedures established under the FOIA.
But the same rules make clear that "research data" does not include:
      Trade secrets, commercial information, materials necessary to be held confidential by a researcher until they are published, or similar information which is protected under law; and (ii) Personnel and medical information and similar information the disclosure of which would constitute a clearly unwarranted invasion of personal privacy, such as information that could be used to identify a particular person in a research study.
The reasons for this policy are clear: researchers have shown that it takes only two or three factors to permit the identification of a study participant using study data, even if most of the features are made "blind," so that the data set can be shared without exposing confidential information. First, the researchers eliminated as much information as possible from the raw data about a health study's participants, to disguise them. But in order to permit others to meaningfully use the data set, some identifying information had to be preserved. For example, geographic location (the place of death, for example), is required to study the correlation between air quality in that location and deaths. Age is a necessary criterion to account for the effects of normal aging and also the sex of the patient, to factor in any possible sex-related issues. The researchers, who included Douglas Dockery of the Six Cities team, found that even retaining this small number of identifying features meant that the patient could be identified by any researchers wishing to use that data to reproduce a study's results.
Response: Nothing in the final rule would require release of confidential data as a prerequisite for EPA to be able to use a federally funded study when promulgating significant regulatory actions or developing influential scientific information. EPA's implementation of this rule will be consistent with statutory mandates, including FOIA, regarding the disclosure of information. 
Comment [8750-1470]: Commenter (9227) suggests that EPA's proposed rule would violate the IQA. 
EPA's proposed rule is also unlawful because it exceeds EPA's authority under Section 515(a) of the Treasury and General Government Appropriations Act for Fiscal Year 2001 (Public Law 16-554; H.R. 5658), commonly referred to as the IQA. Specifically, the IQA requires EPA promulgate data quality guidelines that are consistent with those promulgated by the OMB. Contrary to EPA's assertion in the preamble to the proposal, the Proposed Rule is not consistent with OMB's data quality regulations.
The OMB Guidelines recognize that data availability is not necessary to high quality science, but is one among many factors. While imposing high standards of quality, objectivity, utility, and integrity of information disseminated by Federal Agencies, the Guidelines recognize the need to implement controls "flexibly, and in a manner appropriate to the nature . . . of the information to be disseminated." As part of ensuring "objectivity" these guidelines encourage agencies that disseminate influential scientific, financial, or statistical information, "to include a high degree of transparency about data and methods to facilitate the reproducibility of such information by qualified third parties." However, they emphasize the need to treat certain data differently, due to privacy and confidentiality concerns. In fact, the OMB Regulations specifically declare that "[w]ith regard to original and supporting data related thereto, agency guidelines shall not require that all disseminated data be subjected to a reproducibility requirement." Rather, the OMB Guidelines instruct that agencies "identify, in consultation with the relevant scientific and technical communities, those particular types of data that can practicable [sic] be subjected to a reproducibility requirement, given ethical, feasibility, or confidentiality constraints." The OMB Regulations further explain that while "[m]aking the data and methods publicly available will assist in determining whether analytic results are reproducible...the objectivity standard does not override other compelling interests such as privacy, trade secrets, intellectual property, and other confidentiality protections. OMB explains that "where public access to data and methods will not occur due to other compelling interests, agencies shall apply especially rigorous robustness checks to analytic results and document what checks were undertaken."
By outright prohibiting EPA from relying on a study to support a significant rulemaking if that study's underlying data and models are not publicly available, EPA's proposed rule departs from OMB's unambiguous language instructing agencies that they "shall not" require that all data and models be subject to the reproducibility requirement, and that "the objectivity standard does not override other compelling interests." The fact that EPA's proposed rule includes a discretionary "exemption" provision does not correct this problem, as that provision would not require the Administrator even to consider whether an exemption is warranted, let alone grant such an exemption under appropriate circumstances.
Because Congress expressly granted OMB the authority to set guidelines for data quality and instructed agencies like EPA to follow OMB's lead, EPA lacks statutory authority to adopt a regulation that is contrary to OMB's guidelines. Accordingly, EPA's proposed regulation violates the IQA and must be withdrawn.
Response: EPA disagrees that the final rule violates with the IQA or is inconsistent with OMB's IQA guidelines. In 2002, EPA issued its information quality guidelines in compliance with the IQA and OMB's guidelines. The final rule does not require that the Agency disseminate information that is subject to statutory limitations on disclosure such as confidentiality, privacy and trade secrets. The final rule also does not require the categorical exclusion of studies which was included in the 2018 proposal. The Agency will consider all studies relevant to the development of a significant regulatory action or influential scientific information but will give greater consideration to the studies identified as pivotal science when the underlying dose-response data are available for independent validation.
Comment [8750-1661]: Commenter (9227) contends that EPA arbitrarily failed to consider the implications of this proposal on interagency coordination.
Additionally, the commenter further contends that the EPA arbitrarily failed to consider the far-reaching implications this Proposal could have on inter-agency coordination and consultation given that other agencies normally rely on research potentially excluded by the Proposal. In the numerous environmental statutes that EPA cites, there are dozens of provisions that require EPA to coordinate or consult with other Federal entities -- especially when implementing research programs and issuing information or guidelines. The Proposal would almost certainly frustrate and impair this coordination and consultation, either by forcing EPA to ignore the science provided by other agencies or by severely restricting the science that EPA itself would be able to share with other agencies in these statutorily required processes. The Proposal arbitrarily ignores these potential impacts.
In addition to the many examples of statutorily required consultation that are identified in Appendix B of the commenter's comment letter, other federal agencies routinely incorporate and rely upon EPA science assessments in their own efforts to carry out their mandates to protect human health and safety. As with statutorily required consultations, the Proposal utterly fails to acknowledge or consider what impacts restricting EPA's own use of dose-response studies would have on the work of these other agencies. Indeed, there is no evidence that these other agencies were even permitted to comment on the Proposal as part of the usual process of interagency review.
Some selected examples of other federal agency programs provided by the commenter that rely on EPA science include:
 The Food and Drug Administration (FDA) enforces tolerances established by EPA for pesticide chemical residues in human and animal foods under the Federal Insecticide, Fungicide, and Rodenticide Act, including through a comprehensive pesticide residue monitoring program that tests for approximately 700 pesticide residues in both imported and domestic commodities. To the extent the Proposal affects EPA's tolerances, the nature and effectiveness of FDA's own work to monitor for violations of those tolerances would be impacted.
 FDA also regulates contaminants in bottled water under the Federal Food, Drug and Cosmetics Act. Section 410 of the Act requires that FDA regulations for bottled water be issued in coordination with the effective date of National Primary Drinking Water Regulations issued under the Safe Drinking Water Act, and be no less protective of public health than those standards. If the Proposal impedes EPA's work to establish drinking water standards, this may affect FDA's own ability to justify protective bottled water standards.
 In certain circumstances, FDA also coordinates with EPA to provide the public with information and advice on environmental contaminants in foods. For example, in 2017 FDA and EPA released a joint advisory on mercury hazards associated with the consumption of fish and shellfish, which was based in part on EPA's assessment of the "reference dose" or level of exposure that a person can experience over a lifetime without a risk of harm. The Proposal could radically alter the science EPA would be permitted to consider in future such initiatives, and frustrate the ability of FDA and other agencies to coordinate effectively with EPA to develop joint advice and information.
 The Department of Housing and Urban Development (HUD) is required by statute to assist EPA in assessing the extent of radon contamination in the United States and developing measures to avoid and reduce radon contamination. HUD has also developed policies to require radon testing at properties receiving federal financing, which incorporate EPA radon standards. To the extent the Proposal affects future EPA assessments of radon risks, the scope, cost and effectiveness of HUD radon programs could be affected as well.
Response: Nothing in the final rule prohibits EPA from using any relevant studies, including studies offered by other federal agencies, in promulgating significant regulatory actions or developing influential scientific information. This rule of internal agency procedures governs how EPA will handle pivotal science based on the availability of underlying dose-response data. This rule will not impact how EPA coordinates and works with other agencies in meeting statutory mandates.
Comment [8750-1765]: Commenter (9227) argues that EPA fails to comply with the PRA.
EPA and the White House OMB must scrutinize the Proposal for its information collection burden, as that concept is defined under the Paperwork Reduction Act (PRA). The only reference to the PRA in the Proposal is EPA's denial that this action "contain[s] any information collection activities" or "impose[s] an information collection burden." But if finalized, the Proposal would significantly increase that burden in the rulemakings to which it applies. EPA and OMB cannot rationally ignore such an entirely foreseeable impact when considering this Proposal.
The PRA institutes procedural safeguards to "minimize the paperwork burden for individuals, small business, educational and nonprofit institutions," and others. It requires that, prior to initiating a "collection of information," agencies must "provide 60-day notice in the Federal Register . . . to solicit comment to," inter alia, "evaluate whether the proposed collection of information is necessary for the proper performance of the functions of the agency," "evaluate the accuracy of the agency's estimate of the burden of the proposed collection of information," and "minimize the burden of the collection of the information on those who are to respond." After evaluating public comments, agencies must submit the proposed collection of information to OMB for additional review and publish a notice in the Federal Register setting forth "an estimate of the burden that shall result from the collection of information" and "notice that comments may be submitted to the agency and [OMB]." Any such collection of information is subject to OMB approval. OMB is required to determine "whether the collection of information . . . is necessary for the proper performance of the functions of the agency." A negative determination precludes the agency from initiating the collection of information.
The requirements that EPA would impose through this Proposal qualify as collections of information under the PRA. The statute defines "collection of information" to include "the obtaining [or] causing to be obtained . . . of facts or opinions by or for an agency, regardless of form or format, calling for . . . answers to identical questions posed to, or identical reporting or recordkeeping requirements imposed on, ten or more persons. . ." OMB regulations emphasize the breadth of this definition, specifying that "[a] Collection of information may be in any form or format, including . . . reporting or recordkeeping requirements; . . . policy statements; . . . rules or regulations; . . . oral communications;" and others. "Any recordkeeping, reporting, or disclosure requirement contained in a rule of general applicability is deemed to involve ten or more persons." The definition of "collection of information" is agnostic as to whether disclosure is "mandatory, voluntary, or required to obtain or retain a benefit," and to whether disclosure is to an agency or "members of the public or the public at large."
The Proposal would impose a burden that falls squarely within the definition of "collection of information." In order to use scientific research, the agency would "obtain[] or caus[e] to be obtained . . . facts." Assuming the requirements are applied consistently, the "questions posed," or "reporting or recordkeeping requirements imposed," would be "identical." As the requirements are "contained in a rule of general applicability" -- i.e., the instant Proposal -- they are "deemed to involve ten or more persons." It makes no difference whether the agency seeks the information through a questionnaire, telephone call, or some other format. Nor does it matter whether the agency directly mandates that entities provide the information, or provides that entities must "voluntary[ily]" provide the information in order for research to be eligible for consideration in important rulemakings.
While EPA has refrained from detailing the mechanics by which entities would provide the information, the agency expressly contemplates that the burden of providing such information would fall at least partly to members of the public whom the PRA exists to protect. For example, proposed regulation 40 C.F.R. § 30.5 provides that, "[w]here data is controlled by third parties, EPA shall work with those parties to endeavor to make the data available in a manner that complies with this section." Moreover, the agency specifically "solicits comment on how to incorporate stronger data and model access requirements in the terms and conditions of cooperative agreements and grants." As noted above, the PRA is implicated when collection of information is "required to obtain or retain a benefit," and OMB guidance has identified grants as a "Federal benefit" for purposes of the PRA.
EPA cannot evade the PRA requirements by narrowly asserting that "this action" imposes no information collection burden and ignoring the action's entirely foreseeable future impacts. The proposal expressly "is intended to apply prospectively," suggesting that it "prospectively" requires burdensome collections of information in future rulemakings. EPA must not ignore the PRA in this rulemaking, only to claim in future rulemakings that this rule moots or constrains the PRA's application by compelling certain collections of information.
In the alternative, if EPA genuinely believes that this Proposal would not burden the public with new collections of information, then EPA's stated basis for this rulemaking is exposed as a farce. EPA claims that the Proposal would "ensure" that certain data "are publicly available" and expresses specific concern for science "developed outside the agency." Collection of information, including from researchers employed outside of the federal government, is central to the purpose -- and essential to the implementation -- of the Proposal. Providing this information would inevitably impose a burden on researchers. If the agency does not actually intend to collect information under this Proposal, it underscores that EPA's true purpose is not to increase transparency, but rather to thwart the development and maintenance of vital public health protections on the grounds that the agency lacks the information it would need to support them.
At a minimum, EPA must acknowledge and describe the information collection burden that this Proposal would impose so that OMB and the public can conduct a proper evaluation and provide responsive comments.
Response: EPA disagrees that the PRA applies to this rule. Nothing in the final rule requires the collection of information by EPA from outside parties nor does it obligate outside parties to provide information to EPA. The final rule describes how EPA will handle studies used to support significant regulatory decisions and influential scientific information. In determining how to weight studies determined to be "pivotal science," EPA will assess the availability of dose-response data for those studies. Pivotal science where the underlying dose-response data are available for independent validation will be given greater consideration than pivotal science lacking data availability. 
Comment [6895-3138]: Commenter (6916) notes that to the extent that the Proposal would require release of the underlying confidential data as a prerequisite for EPA to be able to use a federally-funded study in making a rulemaking decision, it also is in conflict with the regulations implementing the FOIA. Those regulations, following the 1998 Shelby Amendment to the 1999 Supplemental Appropriations Act, provide access through FOIA to federally supported studies' "research data"  -  so long as its confidentiality is preserved. The Shelby Amendment had directed changes to OMB Circular A-110 and FOIA. The implementing rules state that:
      [i]n response to a Freedom of Information Act (FOIA) request for research data relating to published research findings produced under a Federal award that were used by the Federal government in developing an agency action that has the force and effect of law, the Federal awarding agency must request, and the non-Federal entity must provide, within a reasonable time, the research data so that they can be made available to the public through the procedures established under the FOIA.
But the same rules make clear that "research data" does not include:
      Trade secrets, commercial information, materials necessary to be held confidential by a researcher until they are published, or similar information which is protected under law; and (ii) Personnel and medical information and similar information the disclosure of which would constitute a clearly unwarranted invasion of personal privacy, such as information that could be used to identify a particular person in a research study.
The reasons for this policy are clear: researchers have shown that it takes only two or three factors to permit the identification of a study participant using study data, even if most of the features are made "blind," so that the data set can be shared without exposing confidential information. First, the researchers eliminated as much information as possible from the raw data about a health study's participants, to disguise them. But in order to permit others to meaningfully use the data set, some identifying information had to be preserved. For example, geographic location (the place of death, for example), is required to study the correlation between air quality in that location and deaths. Age is a necessary criterion to account for the effects of normal aging and also the sex of the patient, to factor in any possible sex-related issues. The researchers, who included Douglas Dockery of the Six Cities team, found that even retaining this small number of identifying features meant that the patient could be identified by any researchers wishing to use that data to reproduce a study's results.
Response: See response to Comment 6895-3139. 
1.8 Trying to Achieve Administratively What Could Not be Done Legislatively
Comment [550-4263]: Commenter (1563) notes that STRS provides that "When promulgating significant regulatory actions, the Agency shall ensure that dose response data and models underlying pivotal regulatory science are publicly available in a manner sufficient for independent validation." It codifies what was intended in the Secret Science Reform Act of 2015, and the Honest and Open New EPA Science Treatment Act of 2017 (HONEST Act), both of which passed the House but never came up for vote in the Senate.
Response: EPA agrees with the commenter that the final rule is consistent with some of the provisions of the HONEST Act.
Comment [997-3842]: Commenter (1012) writes that this rulemaking bears a remarkable similarity to highly partisan legislation that the Chairman of the Committee on Science, Space, and Technology has attempted to enact for the past three Congresses. In each of the past three Congresses, this legislation has failed to be enacted into law. Committee Chairman Lamar Smith was present and standing next to you when you signed the proposed rulemaking on April 24. During the signing ceremony for this proposed rulemaking you acknowledged that the proposed rulemaking was a "culmination of the work" of Chairman Smith's efforts. Further illustrating the notion that this rulemaking is a direct substitute for Chairman Smith's failed legislative efforts is the Chairman's press statement from April 24, when he stated: 
      Earlier in this Congress, the House passed the HONEST Act with bipartisan support. The legislation requires that scientific information relied upon by EPA for regulations be publicly available for independent analysis. Administrator Pruitt's actions enable us to put the principles of this bill into practice. 
Article I of the Constitution vests the power to legislate solely in the hands of the U.S. Congress. Despite the repeated efforts of Chairman Lamar Smith, Congress has never enacted a "Secret Science" bill into law. The commenter believes that the EPA's efforts to administratively implement the provisions of Chairman Smith's failed "Secret Science" bills oversteps the EPA's authority under the U.S. Constitution. For that reason, I urge you to withdraw the proposed rulemaking.
Response: EPA agrees that final rule is similar to several provisions of the HONEST Act and may have been motivated by similar concerns. However, EPA disagrees that it lacks the authority to promulgate this rule in the absence of congressional action. The agency has housekeeping authority to promulgate rules that will govern the internal procedures EPA uses to evaluate the science underlying its significant regulatory actions and influential scientific information. EPA could have revised its internal guidance without notice and comment rulemaking but doing so allowed the agency to inform the public of its efforts to increase transparency of the science the agency relies upon and to help facilitate independent validation of the science. 
Comment [1370-4426]: Commenter (1385) writes that recent news reports suggest that you plan to adopt restrictions on research similar to those contained in two pieces of proposed legislation (the Secret Science Reform Act and the HONEST Act), which have both repeatedly failed to pass Congress for several years for good reason.
Response: While the final rule is similar to aspects of the legislation cited, the rule will not restrict the use of any scientific studies to support final significant regulatory decisions and influential scientific information. The final rule adopts a weighting approach to evaluating studies rather than the categorical exclusionary approach in the 2018 proposal. 
Comment [1380-4438]: Commenter (1395) writes that Congressman Lamar Smith sponsored a bill, the "Honest Act" which would bar the use of data from such studies, which did not pass. This rule making is an attempt to circumvent Congress and implement such a rule anyway.
Response: See the responses to Comments 997-3842 and 1370-4426.
Comment [1409-4467]: Commenter (1424) writes that the origin of this proposal is the house bill HR1430, sponsored by the anti-science republican chair of the House of Representatives Science, Space and Technology Committee, Lamar Smith. The EPA, under the troubled head, Administrator Pruitt, has transferred this failed bill to the EPA in an attempt to enact it as an agency regulation.
Response: See the responses to Comments 997-3842 and 1370-4426.
Comment [1973-4610]: Commenter (1988) writes that on 24 April 2018, Administrator Scott Pruitt signed the misleadingly named "Strengthening Transparency in Regulatory Science" - a proposal to require the Environmental Protection Agency (EPA) to make the raw data underlying regulatory action publicly available in a way that allows for independent validation. Based on the Secret Science Act and the more recent HONEST Act, this rule would restrict the use of scientific research vital to informed decision-making, such as studies containing private or proprietary information including personal health records, under the guise of transparency and the desire to ensure the robustness of science.
Response: See the responses to Comments 997-3842 and 1370-4426.
Comment [4579-2459]: Commenter (4590) writes that a Congressional bill similar in intent to the EPA's proposed rule  -  the HONEST Act  -  was offered and defeated last year. Members of Congress had serious and valid concerns about the negative impact the bill would have by significantly reducing the amount of scientific evidence that EPA can consider when adopting regulatory standards to protect public health and the environment. Given the requirements of the APA and federal environmental statutes, the agency cannot do by regulation what Congress declined to do through legislation.
Response: See the responses to Comments 997-3842 and 1370-4426.
Comment [4870-2301]: Commenter (4881) writes that the Congress has consistently declined to adopt legislation that would add an explicit requirement to provide public access to raw data. The EPA, in its proposed rule, is attempting to delegate to itself this clearly legislative authority, with no evidence for the allegedly "secret science" raised by a small group of legislators who seek to undermine traditional EPA authority to regulate environmental hazards. That small group of legislators has had a barely concealed focus on several particular studies that have been foundational in establishing the very real connections between exposure to air pollution and adverse public health impacts.
Response: See the responses to Comments 997-3842 and 1370-4426.
Comment [4883-2114]: Commenter (4894) writes that the rule's premise, which was also the premise of the Secret Science Act and the HONEST Act, cannot stand.   
Response: See the responses to Comments 997-3842 and 1370-4426.
Comment [5011-2654]: Commenter (5022) writes that the proposal lacks critical citations and documentation or even an adequate justification for why it was proposed. Rather than furnishing the evidentiary support required for administrative action, the agency has merely adopted a legislative initiative that failed to pass despite support from the energy, chemical, manufacturing, and other key industries. That legislation was proposed by Rep. Lamar Smith (R-TX), Chair of the House of Representatives Committee on Science, Space, and Technology, who questions the science behind climate change and the relationship between air pollution and mortality.
Response: EPA explained the justification for the rule in the preamble of the 2018 proposal. 
      Enhancing the transparency and validity of the scientific information relied upon by EPA strengthens the integrity of EPA's regulatory actions and its obligation to ensure the Agency is not arbitrary in its conclusions. By better informing the public, the Agency in enhancing the public's ability to understand and meaningfully participate in the regulatory process." 83 Fed. Reg. 18769 (April 30, 2018).
Further, EPA cited Executive Orders, OMB memoranda and guidelines, and scientific journal articles, etc., as justification for the rule.
Comment [5153-4081]: Commenter (5164) writes that recent news reports suggest that you plan to adopt restrictions on research similar to those contained in two pieces of proposed legislation (the Secret Science Reform Act and the HONEST Act), which have both repeatedly failed to pass Congress for several years for good reason. Multiple major scientific societies repeatedly came out strongly against the legislation at the time. "We urge caution in setting laws that submerge science beneath politics," they wrote.
Response: See the responses to Comments 997-3842 and 1370-4426.
Comment [6104-614]: Commenter (6114) states that the proposal is strikingly similar to both the proposed Honest and Open New EPA Science Treatment (HONEST) Act of 2017 (H.R.1430-115th Congress, 2017-2018) and the proposed Secret Science Reform Act of 2014 (H.R. 4012-113th Congress, 2013-2014) and 2015 (H.R.1030 - 114111 Congress, 2015-2016), all of which repeatedly failed to pass in the Senate despite being supported by the energy, manufacturing, and chemical industries.
Response: See the responses to Comments 997-3842 and 1370-4426.
Comment [6155-2166]: Commenter (6125) writes that this initiative is clearly based on the so-called HONEST Act, legislation introduced in each of the last two Congresses to restrict EPA's ability to rely on scientific information, but never enacted into law (H.R. 1030 in 2015 and H.R. 1430 in 2017). The CBO in consultation with EPA analyzed the costs that would be imposed by these legislative proposals. The 2015 analysis assumed that EPA would reduce the number of studies it relied on by half, but would still need to expend $250 million/year initially in an effort to determine data availability, and where necessary pay for obtaining and disseminating it. Given EPA's intention to "minimize" such costs in its proposed rule, this estimate may be considered an upper bound of the direct costs. 
The CBO analysis does not tell the whole story, however, because it did not assess the disbenefits to the regulatory process and to public health of being unable to base regulations on numerous influential studies for which data could not be made available. Considering only the science policy value of the information in multiple original studies that are likely to be lost (e.g. the cohort epidemiology studies noted above, which must limit access to protect privacy) it is reasonable to conclude that their loss would almost certainly outweigh the value of any information gained by subsequent re-analyses of a more limited set of studies, which rely on publicly available databases that are often inferior to those that contain more relevant 
information.
In the 2017 legislative analysis, CBO estimated a cost ranging from 1 million to 100 million dollars per year, depending on the approach taken by EPA in assessing studies. They determined that meeting H.R. 1430 requirements would cost EPA an average of $10,000 per study. EPA officials told CBO that the agency would likely greatly reduce the number of studies it relied on and would not take on the cost of disseminating the underlying data.  The proposal reiterates EPA's plan to focus on a more limited number of studies. Under these assumptions, CBO suggested costs could be as low as $1 million/year, but again did not assess the potential implications for existing or future regulations. An unofficial draft response to CBO questions from unidentified EPA staff strongly disagreed with the lower cost estimates, and expressed concern that the legislation would prevent EPA from using the best available science; this response was never forwarded to CBO.
Response: The final rule will not reduce the number of scientific studies that the EPA may use in developing significant regulatory actions or influential scientific information. Further, the rule does not require the EPA to compile, store or disseminate data from outside sources. As discussed in Section III.J. of the preamble, the EPA has identified some incremental costs that the Agency may incur as a result of this rule. The EPA will continue its current practice of conducting extensive review of scientific studies during the development of final significant regulatory actions and influential scientific information. The additional procedures required by this rule only apply to pivotal science, which is a subset of the total number of studies that the EPA would evaluate. Given the costs of the extensive review and documentation and the determination of dose-response data availability is limited to pivotal science underlying final significant regulatory actions and influential scientific information, the EPA anticipates that the incremental costs of this rule related to the additional review of data availability will be small.
Comment [6161-1506]: Commenter (6131) notes that the proposal is strikingly similar to both the Honest and Open New EPA Science Treatment (HONEST) Act of 2017 (H.R. 1430  -  115th Congress, 2017-2018) and the Secret Science Reform Act of 2014 (H.R. 4012  -  113th Congress, 2013-2014) and 2015 (H.R. 1030  -  114th Congress, 2015-2016), all of which repeatedly failed to pass in the Senate despite being supported by the energy, manufacturing, and chemical industries.
Response: See the responses to Comments 997-3842 and 1370-4426.
Comment [6163-589]: Commenter (6133) contends that the governing, harmful conceit of the Proposal -- to censor the best available, peer-reviewed health science that EPA may consider, in order to prevent adoption of protective health and environmental safeguards -- is a thinly disguised version of anti-science legislation that Republican members of Congress have introduced, repeatedly, but have been unable to enact into federal law, repeatedly. The commenter opposed those bills strongly, and still does. The commenter raised many of the identical objections to those bills that they raise to the Proposal in these comments. Indeed, it is striking that one of the primary EPA co-authors of the Proposal was a Committee staff person for the leading congressional co-sponsor of the legislation in question when the failed bill was being shepherded through the House of Representatives. 
Members of Congress understood that new legislation was required to censor EPA consideration of high quality, peer-reviewed science, and yet EPA barreled ahead with a Proposal based on the same legislative approach while pretending, suddenly, that multiple federal laws have authorized that approach, magically, all along. For the reasons discussed in these comments, the Proposal is not authorized by any federal laws. Moreover, the Proposal violates numerous federal laws entrusted to EPA, in addition to being arbitrary and capricious and an abuse of EPA's discretion. 
Response: See the responses to Comments 997-3842 and 1370-4426.
Comment [6339-1797]: Commenter (6361) states that the proposal is strikingly similar to both the Honest and Open New EPA Science Treatment (HONEST) Act of 2017 (H.R. 1430  --  115th Congress, 2017-2018) and the Secret Science Reform Act of 2014 (H.R. 4012  --  113th Congress, 2013-2014) and 2015 (H.R. 1030  --  114th Congress, 2015-2016), all of which repeatedly failed to pass in the Senate despite being supported by the energy, manufacturing, and chemical industries. These nearly interchangeable bills would have required all research data used in agency actions to be made available to the public. Our nation's health privacy laws were enacted to address citizen's concerns about this type of information being made public or getting into the wrong hands.
Response: See the responses to Comments 997-3842 and 1370-4426.
Comment [6857-2946]: Commenter (6879) writes that the proposed regulation is also arbitrary and capricious. This rule has its origins in a failed congressional attempt to limit the science available for use in EPA rulemaking, and Congress rightfully rejected this extreme and harmful proposal. It is therefore shocking that EPA would take this failed effort and, unsupported by facts, reason, history, or scientific evidence, put the proposed regulation on a fast-track for administrative approval and adoption.
Response: See the responses to Comments 997-3842 and 1370-4426.
Comment [6894-3593]: Commenter (6915) writes that this anti-science approach has stalled in Congress and been rejected by the courts; it has no place at EPA.
Response: See the responses to Comments 997-3842 and 1370-4426.
Comment [6897-1882]: Commenter (6918) writes that they support enhanced transparency in EPA rulemakings, but recognize the vital importance of privacy, confidentiality of personal and business information, and national and homeland security as limits on data access. Commenter therefore agrees with EPA's goal of making data and models underlying pivotal regulatory science available while providing a process for protecting data if their release would impair privacy, confidentiality, or security. Commenter recommends that EPA more clearly detail an objective and consistent process that would determine when release of data underlying pivotal studies would be deemed inappropriate because it would impair privacy, confidentiality, and/or national security. 
Response: The EPA appreciates your comment in support of this rule. The EPA intends to develop guidance for Agency employees to ensure that the rule implemented in a consistent manner. The EPA will continue to comply with statutory restrictions on the release of dose-response data.
Comment [6907-3957]: Commenter (6928) writes that the new EPA proposal is the latest in a long-running campaign to let the public and regulated industries sift through the raw data of epidemiologists whose work could affect pollution regulations. 
In the 1990s, members of Congress pressed for legislation requiring scientists to disclose their raw scientific data, partly in response to a Harvard University study finding a correlation between more air pollution and lower life expectancy. Several times in recent years, the House of Representatives passed a bill requiring public disclosure of data from any new studies used by EPA to write regulations, but the proposal never made it out of Congress.
Response: See the responses to Comments 997-3842 and 1370-4426.
Comment [9283-3588]: Commenter (8281) notes that over the last few years, the House Committee on Science, Space, and Technology has considered several iterations of legislation that have many similarities to the proposed rule. Commenter has been a vocal opponent of these bills... Commenter also wants to note that despite repeated efforts by the majority, the so-called secret science legislation has not passed both chambers. Congress has the sole constitutional authority to legislate, and this proposed rule is an administrative attempt to circumvent the legislative process. Commenter strongly urges EPA to withdraw this proposed rule. It will undermine scientific integrity, jeopardize bedrock public health and environmental standards, and endanger the EPA's ability to protect the American people, which is its mission.
Response: See the responses to Comments 997-3842 and 1370-4426.
1.9 Other Comments (NPRM Comments)
Comment [2157-145]: Commenter (2171) states that this proposed rule is counter to the current goal of Cooperative Federalism.
As an example, the MPCA manages facility air toxics emissions through modeling and risk-based assessments, with support from MDH. Specifically, we compare modeled air concentrations to inhalation health benchmarks. ln order to keep our approach systematic and non-biased, the MPCA and MDH pre-select information sources of inhalation health benchmarks in the form of a hierarchy. Approximately 40% of the inhalation health benchmarks that the MPCA uses to manage air toxics emissions are from EPA's IRlS or Provisional Peer-Reviewed Toxicity Values (PPRTV) programs. Our Understanding of this proposed rule leads us to believe that the dose-response information used to develop these values could become mired in redundant and unnecessary review, should the rule be enacted. Any delay in the review of IRlS or PPRTV data greatly compromises the MPCA's ability to assess and appropriately set permit conditions for industries, or to determine clean up requirements for site remediation projects.
Response: This procedural rule only applies to EPA and not to any other party outside of the agency. Any potential implementation issues will be addressed as appropriate in subsequent rulemakings implementing this procedural rule.
Comment [6092-471]: Commenter (6102) suggests that given the practical and legal complexities surrounding this effort, the EPA may want to supplement this rule with statute-specific rulemakings. This would allow EPA to more precisely tailor transparency requirements in the context of individual statutes while providing greater legal certainty.
Response: EPA agrees that statute-specific rulemakings are necessary. As discussed in the final rule preamble at Section III.A, statute-specific transparency regulations or programmatic regulations are forthcoming.
Comment [6119-2471]: Commenter (6144) notes that records released through a FOIA request confirm that EPA staff are aware that the agency does not need to have access to the underlying raw data, codes, etc. to promulgate regulations. In an email sent by Dr. Nancy Beck, the Deputy Assistant Administrator for the Office of Chemical Safety and Pollution Prevention, to her colleagues during the development of this proposed rule, she shares "for awareness" that in the 2002 court decision American Trucking Association v. EPA, it became evident that EPA would not be able to use a significant amount of important scientific literature to protect public health and the environment if it had to access the raw data of peer-reviewed scientific studies. The courts, through that decision and through Coalition of Battery Recyclers Association v. EPA, have made it abundantly clear that EPA is not required to obtain or analyze raw data in order to use independent scientific studies that are published in peer-reviewed journals to regulate.
Response: This final rule does not require EPA or any entity outside of the agency to obtain, analyze, or publicize raw data in order to consider pivotal science to be used in significant regulatory decisions and influential scientific information and does not require EPA to categorically exclude any studies. Therefore, it does not conflict with those decisions.
Comment [6125-1850]: Commenter (6150) writes that increasing transparency will improve the quality of the data used to support regulatory decisions, thereby assuring that EPA relies on the best available science. EPA may want to supplement this rule at a later date with statute-specific rulemakings. This will allow EPA to be more precise in how the transparency requirements apply in the context of individual statutes while providing greater legal certainty. This suggestion should not be justification or causation for not taking steps more generally ahead of addressing any statute-specific rulemakings.
Response: See response to Comment 6092-471.
Comment [6163-5032]: Commenter (6133) notes that EPA identifies no indication under federal laws that Congress intended to create or authorize a lose-lose dynamic, in which EPA could exercise its authority either by excluding the best available, peer-reviewed science to inform health and environmental protections, or force researchers or ordinary Americans to cast aside privacy concerns, as well as legal and ethical restrictions on the sharing of sensitive data. That false choice is entirely a creation of the agency's misguided policy preferences in the Proposal. The rule is arbitrary and capricious and an abuse of EPA discretion, with its selective application of data release requirements and disregard for the quantitative complexities of epidemiologic research. 
Response: The final rule does not require EPA to exclude science or to disclose protected information. Rather, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. See the final rule preamble at Section III.E for further discussion of the finalized approach and EPA's process for considering protected information.
Comment [6356-2427]: Commenter (6378) states that drawing from their experience on multiple EPA panels for CASAC, IRIS, and FIFRA, they are convinced that the proposed rule will severely limit the information that EPA can use to make decisions by requiring that studies used in the scientific reviews in support of science make their data publicly available. This will directly contradict the mandates of many Acts that require that the scientific review draw upon the best available science. 
Response: As explained in Section III.E of the final rule preamble, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are publicly available in a manner sufficient for independent validation, and not categorically exclude pivotal science for which the underlying dose-response data are not available. EPA has explained that this procedural rule has limitations on its scope in terms of conflicts with other substantive requirements and that, in situations where this rule applies, EPA will give greater consideration to those studies where the underlying data and models are available for independent validation. The final rule provides that if implementing the rule results in any conflict between this rule and the environmental statutes that the EPA administers, this rule will yield and the statute will control.
Comment [6357-1865]: Commenter (6379) argues that nothing in the Proposed Rule or its implementation should preclude the use of the best available science. Numerous statutes that EPA is charged with implementing require that EPA rely on the best available science and health information. For example, the Toxic Substances Control Act requires that in conducting tests and risk assessments of toxic substances, "the Administrator shall use scientific information, technical procedures, measures, methods, protocols, methodologies, or models, employed in a manner consistent with the best available science." Similarly, the Safe Water Drinking Act requires that a determination to regulate a contaminant "shall be based on the best available public health information." And, as the Proposed Rule notes, Executive Order 13563 (2011) requires that the "regulatory system" be "based on the best available science."
However, it is unclear whether it is appropriate and legally durable to restrict the use of the best available science by, as the Proposed Rule does, defining "best" to include a requirement that data be "publicly available." Such a requirement may be inconsistent with Court findings and established Federal and EPA requirements. For example, the Eighth Circuit Court of Appeals has stated in the context of other federal regulations that "it is required of the agencies...to seek out and consider all existing scientific evidence relevant to the decision at hand. [Agencies] cannot ignore existing data."
Response: See response to Comment 6356-2427.
Comment [6852-2763]: Commenter (6874) argues that EPA has failed to demonstrate that its current processes for considering science in regulation have suffered from insufficient transparency that has resulted in outcomes contrary to its statutory mandates or executive orders. Given extensive existing procedures used by EPA and the scientific community at large to assure the quality of research, EPA has failed to make a case that additional public access to data is necessary.
Response: This rulemaking is intended to build on open data initiatives in the Federal government and scientific community, and thereby improve the transparency of EPA decision-making and assessments Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to validate original research findings, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Having a better understanding of the underlying data will enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or ISI. EPA further discusses the need for additional transparency in the final rule preamble at Section III.A.1.
Comment [6891-1529]: Commenter (6912) writes that EPA has not engaged in reasoned decision-making in developing this rule.
While judicial review of agency actions is usually ''exceedingly deferential," when, given the totality of the circumstances, the agency appears not to have engaged in reasoned decision-making, a rule should be invalidated.
The scope of review under the "arbitrary and capricious'' standard is narrow and a court is not to substitute its judgment for that of the agency. Nevertheless, the agency must examine the relevant data and articulate a satisfactory explanation for its action including a ''rational connection between the facts found and the choice made." In reviewing that explanation, we must "consider whether the decision was based on a consideration of the relevant factors and whether there has been a dear error of judgement.
This heightened level of scrutiny calls on a court to "intervene not merely in case of procedural inadequacies, or bypassing of the mandate in the legislative charter, but more broadly if the court becomes aware, especially from a combination of danger' signals, that the agency has not really taken a 'hard look' at the salient problems, and has not genuinely engaged in reasoned decision-making.'
In Greater Boston Television Cmp. v FCC, the biggest "danger signal" that caused the court to give an agency's actions a "hard look'' was the fact that the chair of the Federal Communications Commission (FCC) had had potentially improper contacts with an executive at one of companies competing for a broadcast license to be attributed by the agency. Other "danger signals" that courts have held to trigger heightened scrutiny of agency actions include "abrupt shifts in policy," and "where the agency has demonstrated undue bias towards particular private interests." These concerns are epidemic for this proposed rule.
As detailed in Part II A of this comment, the proposed rule is the result of a long, tobacco and fossil fuel industry-led campaign to limit the use of science in rulemaking. This campaign focused on limiting the use of public health studies that demonstrate dramatically negative health consequences of smoking and breathing polluted air. The trick was to take advantage of the private nature of underlying individual health care information (a value worth protecting), and twist that into a faux concern about "secret science."
Part II B lays out the myriad connections between Pruitt, congressional champions of this proposed rule, and the EPA political appointees who developed it, and the fossil fuel and tobacco industries who have campaigned for it for decades. This long and well-documented record is rife with the sort of "danger signals" the courts have found to warrant "hard look" review. Potentially improper contacts between regulators and regulated industries, "abrupt shifts in policy," and ''undue bias towards particular private interests" are all present in this tawdry tail of industry capture. Based on this record, no court could conclude that EPA "genuinely engaged in reasoned decision making" nor did it "articulate a satisfactory explanation for its action including a 'rational connection between the facts found and the choice made. '"
Response: EPA notes that although this is an internal procedural rule not subject to the APA's notice and comment requirements, see 5 U.S.C. § 553(b), EPA nevertheless published a proposed rule and supplemental proposed rule in the Federal Register, solicited public comment on both, and made all relevant documents available for public review. In so doing, EPA complied with the APA's notice and comment requirements even though the rule is not subject to them. Taken together, the proposal and supplemental proposal articulate a rational basis for the final rule and demonstrate that the agency engaged in reasoned decision-making, informed by the over 950,000 public comments received (over 22,400 unique comments). 
Comment [6891-1530]: Commenter (6912) argues that the proposed rule is an illegal delegation of agency rulemaking authority to the industries it regulates. 
Just as an agency rulemaking will be set aside if a court determines that it was arbitrary and capricious, an agency rulemaking should be invalidated if a court finds that the agency delegated its rulemaking authority to one or more private interests; because Congress "cannot delegate regulatory authority to a private entity." "Although objections to delegations are "typically presented in the context of a transfer of legislative authority from the Congress to agencies, [...] the difficulties sparked by such allocations are even more prevalent in the context of agency delegations to private individuals."
While it is clear that an agency may not explicitly delegate its rulemaking authority to private interests, an agency that implicitly delegates its rulemaking authority to private interests raises the same concerns. An agency is effectively captured by the private interests it regulates when its '"regulation is . . . directed away from the public interest and toward the interest of the regulated industry ' by 'intent and action' of industries and their allies."
As described above, the proposed rule was the product of a process that was effectively delegated to industry, with EPA serving as little more than a rubber stamp on an industry -  sought and industry-drafted proposal. The evidence is overwhelming that the proposed rule was conceived by the tobacco industry and then advanced by the tobacco and fossil fuel industries and the network of individuals and front groups they have funded for more than two decades.
Steve Milloy and Christopher Horner, two of the biggest players in industry 's "secret science" campaign, even worked for President Trump's EPA transition team, and one boasted that he was "inside the Administrator 's office" on the day the proposed rule was announced. Richard Yamada, the political appointee drafting the proposed rule, spent most of his Washington career working for Representative Lamar Smith, the primary congressional champion of the "secret science" initiative, who also happens to count the fossil fuel industry as his biggest donors. It was Smith whose meeting with Pruitt apparently triggered the drafting of the proposed rule. Brittany Bolen, Ryan Jackson, and Erik Baptist, the other political appointees involved in drafting the proposed rule, all previously worked for Senator James Inhofe, another fossil fuel industry-funded congressional champion of the rule.
The fossil fuel industry and its long-time "secret science" allies quickly took advantage of Scott Pruitt's ascendency at EPA. At least one company - Chevron - included the "secret science" initiative on its EPA wish list. The fact that Pruitt resigned in disgrace as EPA Administrator after the publication of the proposed rule does not allay these concerns. The man is gone, but his conflicts and the flawed process he oversaw lingers on.
Moreover, the capture of EPA goes deeper than just Scott Pruitt. The case for agency capture survives his departure. Numerous EPA political appointees; all of whom have their own significant ties to the fossil fuel industry, were intimately involved in this rulemaking. Intact current EPA Acting Administrator Andrew Wheeler, who will oversee the rulemaking process going forward, has his own deep ties to the fossil fuel industry.
We are not the only ones to conclude that EPA under the current ·administration has been captured. by industry. A recently published article in the American Journal of Public Health finds that EPA is exhibiting many signs of regulatory capture. The authors of this article examined EPA actions from December 2016 through June 2017 arid they interviewed 45 current and retired EPA employees. Among their findings pointing to regulatory capture:
 Appointees have deep ties with industries."
 "Significant policy changes at the EPA favor businesses and industry, while probably incurring considerable health and environmental "
 ''Pruitt has regularly championed the interests of regulated industries, while rarely affirming environmental and health "
 "Pruitt dismissed many members of the EPA's Science Advisory Board and its Board of Scientific Counselors, created a new rule preventing EPA-funded scientists from serving on those boards, and-for the first time in agency history -  allowed lobbyists on scientific advisory "
 "Pruitt's own meetings and schedule...are almost exclusively with company and trade organizations and rarely with environmental, public health, or citizen groups."
The extreme and well-documented regulatory capture of EPA has effectively delegated its authority to the industry or industries that have captured. There is no substantive difference between an agency explicitly telling a company or industry to write a rule for it, and an agency telling a company or industry that it will write whatever rule the company or industry wants. Like Scott Pruitt' s Devon Energy letter, the substance is all industry, whatever the letterhead, and the public interest is ignored.
Response: This procedural rule outlines the internal processes by which EPA will consider dose-response data underlying pivotal science to be used in its significant regulatory actions and influential scientific information. It does not apply to anyone outside of EPA, and therefore cannot be an impermissible delegation of EPA's authority to industry stakeholders.
Comment [6903-1980]: Commenter (6924) points out that despite citing policies from the NIH and the National Science Foundation (NSF), EPA does not appear to have conducted any inter-agency consultation on the rule. EPA's SAB also notes that EPA is legally mandated to make proposed regulations available for SAB review, which the Agency failed to do.
Response: The supplemental proposal underwent interagency review and the SAB reviewed aspects of the proposed rule. See https://yosemite.epa.gov/sab/sabproduct.nsf/LookupWebProjectsCurrentBOARD/2DB3986BB8390B308525855800630FCB/$File/EPA-SAB-20-005.pdf and response to Comment 6425-2369.
Comment [9078-3499]: Commenter (6194) discusses EPA's failure to address impacts under the Paperwork Reduction Act (PRA).
The PRA requires OMB to review federal rules that impose information collection requirements on "persons," including individuals and corporations. 44 U.S.C. § 3502(10). Such collection of information includes "obtaining, causing to be obtained, soliciting, or requiring the disclosure to third parties or the public, of facts or opinions by or for an agency, regardless of form or format" through the use of "reporting or recordkeeping requirements." 44 U.S.C. § 3502(3)(A)(i). Although EPA asserts that the Proposal is not subject to the PRA, perhaps because it does not include an explicit reference to information collection, the Proposal would create new reporting requirements that would apply to multiple "persons." Specifically, for science to be considered in certain regulatory proceedings, someone (whether the person conducting the science, preparing a report, or utilizing data) would have to "clean" confidential data so that it can be made public. Such reporting and recordkeeping procedures are costly and, in some instances, inconsistent with other federal policies governing information use and dissemination. This aspect of the Proposal must be addressed by OMB as required by the PRA.
Response: This is an internal rule of agency procedure. It does not regulate any party outside of EPA. In any event, this action does not contain any information collection activities and therefore does not impose an information collection burden under the PRA.
Comment [6093-544]: Commenter (6103) notes that the U.S. Court of Appeals for the D.C. Circuit has stated in 2002 and 2010 that the EPA is not legally obligated to obtain and publicize all the data underlying its research when crafting policies and regulations.
Response: See response to Comment 6119-2471.
Comment [6103-518]: Commenter (6113) notes that before EPA approves its proposed regulation, NEPA requires that EPA prepare an EIS. That EIS must include a detailed analysis of each registered pesticide's environmental impacts. Such analysis of each pesticide should include: the quantity of the pesticide used every year, the number of applicators applying the pesticide each year, the human toxicity of the pesticide, the non-human affect of the pesticide on the flora and fauna within wind's reach of the pesticide application area, the pesticide's impact on groundwater underlying the application area. Also in light of the studies relied in the United case, it[']s clear that the EIS must also include a detailed analysis of the pesticide's impact on the family of the applicators and any other people living near the application area. United, slip op at p. 11-12 (discussing internal EPA studies). Given that there are thousands of registered deadly pesticides, EPA should begin work on the EIS immediately. P. Finegan, FIFRA Lite: A Regulatory Solution or Part of the Pesticide Problem?, 6 Pace Environmental Law Review 615, 624 (1989) (noting that there are over fifty thousand registered pesticide products with over 600 active ingredients). 
Response: See response to Comment 20-1294.
Comment [6104-621]: Commenter (6114) argues that the proposed HONEST Act, the Secret Science Reform Act, and this proposed rule all directly contradict a 2002 and a 2010 ruling from the D.C. Circuit Court of Appeals which states "we agree with EPA that requiring agencies to obtain and publicize the data underlying all studies on which they rely 'would be impractical and unnecessary'".
Response: Regarding the HONEST Act and the Secret Science Reform Act, see response to Comment 1370-4426. Regarding the cited court decisions, see response to Comment 6119-2471.
Comment [6133-2172]: Commenter (6158) notes that numerous court cases have weighed in on the best available science standard in the context of numerous different federal laws. For example, courts have found that the agency must use the best science that is available to it at the time of the decision, not the best that might be available to it at some time in the future.
Response: This purely internal procedural rule does not interpret or apply the provisions of any environmental statutes that EPA administers. As discussed in the final rule preamble at Section III.A.3, if implementing this rule would result in conflicts with existing environmental statutes, and their implementing regulations, this rule will yield to the EPA statutes.
Comment [6155-592]: Commenter (6125) writes that numerous courts have indicated that a plaintiff or petitioner can establish a violation of the "best available science" requirement by "point[ing] to any scientific evidence that the agency failed to consider." Safari Club Int'l v. Salazar (In re Polar Bear Endangered Species Act Listing & Section 4(d) Rule Litig. - MDL No. 1993), 709 F.3d 1, 9 (D.C. Cir. 2013). "The best available data requirement ... prohibits [an agency] from disregarding available scientific evidence that is in some way better than the evidence [it] relies on.'" Kern Cnty. Farm Bureau v. Allen, 450 F.3d 1072, 1080 (9th Cir. 2006) (quoting Sw. Ctr. for Biological Diversity v. Babbitt, 215 F.3d 58, 60 (D.C. Cir. 2000)). "Essentially, [the agency] `cannot ignore available ... information.'" San Luis & Delta-Mendota Water Auth. v. Jewell, 747 F.3d 581, 602 (9th Cir. 2014) (quoting Kern Cnty., 450 F.3d at 1080-81 (quoting Conner v. Burford, 848 F.2d 1441, 1454 (9th Cir. 1988))."
Response: See response to Comments 6133-2172 and 6163-5032.
Comment [6172-3286]: Commenter (6182) suggests that to serve the American public's interests, EPA should:
 Codify its findings for "Increasing Consistency and Transparency in Considering Costs and Benefits in Rulemaking Process" as a binding rule in the CFR rather than mere "guidance.
 Internally enforce such a rule and ensure it is judicially enforceable.
Enforcing such a process would also serve the purpose of Executive Order (EO) 12988 to "reduce needless litigation." Enforcement would also serve to comply with stated objectives of Federal "data quality" policies that are still "on the books" from past Administrations.
DOE's Process Rule, while ostensibly well intentioned, was fatally flawed in that:
 It has been generally treated as mere guidance rather than as an enforceable rule (despite its CFR status).
 It was made immune to judicial review and enforcement by the following text:
(c) Judicial review. The procedures, interpretations, and policies stated in this Appendix are not intended to establish any new cause of action or right to judicial review.
Regulatory staffs often tend to resolve issues by promoting more regulation. This tendency should be guarded against if the public interest is to be served. EPA staff, like DOE's, may resist enforceability by arguing that it restricts their regulatory flexibility. By yielding to such arguments, EPA would jeopardize the legitimate objectives of this ANOPR.
DOE/EERE's "process rule" also contains some solid principles regarding transparency and consistency that EPA could adopt. These include:
      Use transparent and robust analytical methods. The Department seeks to use qualitative and quantitative analytical methods that are fully documented for the public and that produce results that can be explained and reproduced, so that the analytical underpinnings for policy decisions on standards are as sound and well-accepted as possible.
Unfortunately, notwithstanding the clear statement in favor of rulemaking transparency, EERE's implementation over the years directly conflicts with the above principles by putting too much emphasis upon outsourced analyses to ostensibly "independent" experts, opaque modelling processes and proprietary data sources. Through the years, such complexity has grown and served to hinder independent verification by the public. The following excerpt is but one example from DOE's Process Rule that engendered this problem:
      Identification of experts and other interested parties for peer review. DOE, in consultation with interested parties, will identify a group of independent experts and other interested parties who can provide expert review of the results of the engineering analysis and the subsequent impact analysis.
The "moral of the story" is that EPA should recognize that ambiguously stated objectives could be misused to justify entrenched or predetermined positions through biased regulatory analyses. DOE's Process Rule provides a case-in-point.
Response: The EPA has finalized the rule "Increasing Consistency in Considering Benefits and Costs in the Clean Air Act Rulemaking Process," and plans to issue similar rules for the other substantive environmental statutes that the Agency administers. These are rules of internal agency procedure and are binding to the EPA.

 Chapter 2: Purpose and Effects
The purpose of the notice of proposed rulemaking (NPRM) and supplemental notice of proposed rulemaking (SNPRM) is to establish a clear policy of transparency of the scientific information used for significant regulations. Specifically, the NPRM preamble states that the proposed rule is designed to increase transparency of the assumptions underlying dose-response models. To meet this purpose, the proposed rule directs EPA to ensure that the regulatory science underlying its actions is publicly available in a manner sufficient for independent validation (40 CFR 30.1). The proposal states that using scientific information that can be independently validated will lead to better outcomes and strengthen public confidence in the health and environmental protections underpinning EPA's regulatory actions. 
Several comments were received regarding the purpose/basis and effects of the NPRM and SNPRM, including the effects of the proposal's requirements on available science, health protections, EPA actions, and research activities under the proposal. In this section the public comments are organized according to the following topics under sections 2.1 (Purpose and Effects (NPRM Comments)) and 2.2 Purpose/Basis and Effects (SNPRM Comments):
Section 2.1: Purpose and Effects (NPRM)
 Change in Culture
 What Problem is the Agency Intending to Fix?
 Proposed Rule is Biased
 Environmental and Health Protections Would be Decreased
 EPA Would be at Odds with Other Parts of the Federal Government
 EPA Would be at Odds with Other National and International Bodies
 Proposed Rule is Flawed and Needs to Be Withdrawn/Redone with Feedback from the Scientific Community/Stakeholders
 Other Effects/Impacts
Section 2.2: Purpose/Basis and Effects (SNPRM)
 Purpose/Basis for the Rule
 Effects
The specific comments provided in this Chapter reflect the comment letters that provide arguments supported by detailed rationale, examples, and references/documentation. The Docket ID numbers for these commenters are cited throughout this Chapter. In addition, some general comment letters may also include comments related to comments included and responded to in this Chapter. These comments are summarized in Appendix B. Note that points and issues raised in these general comment letters may also be raised by the more detailed comments summarized in this Chapter. The responses provided in this Chapter adequately address these more general detailed comments on the purpose and effects of the rulemaking. 
2.1 Purpose and Effects (NPRM Comments)
 2.1.1 Change in Culture
 2.1.1.1 Changing Agency Culture Does Not Occur Through Rulemaking
Comment: Commenter (2171) notes that the EPA states that the proposed rule is designed to "...change agency culture and practices regarding data access so that the scientific justification for regulatory actions is truly available for validation and analysis" (83 FR 18770). The commenter suggests that changing agency culture does not occur through rulemaking, especially if few resources are given to new rule implementation. The commenter explains that agency culture and work practices are changed when agency administrators provide leadership, direction, and support of technical and career staff through adequate funding and, in the case of EPA, partnerships with the health and environmental research communities.
Response: The EPA agrees that agency culture and work practices can be changed through a variety of mechanisms. The promulgation of these internal Agency procedures in this rulemaking provides a more durable, enforceable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data and highlights the Administration's commitment to maximizing transparency in the EPA's most impactful regulatory actions and influential scientific information. The rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made by this rule provides an important step towards prioritizing transparency in particularly impactful rules and assessments and will inform future statute-specific rulemakings.
2.1.1.2 Need to Have "Buy-In" From Publishers and Scientists
Comment: Commenter (6449) provides that a paper cited by EPA in the Proposed Rule, Lutter and Zorn (2016) notes that only 20% of authors contacted to provide data underlying their publication actually provide it. Given this circumstance, the commenter suggests that if EPA wishes to change the culture of data access it seems that they need to receive buy-in from publishers and scientists as well as change the internal EPA culture to ensure requirements for data availability are met by EPA funded laboratory and epidemiological studies.
Response: The findings of Lutter and Zorn (2016) are consistent with findings in other published articles on the current state of data sharing. For this reason, the rule requires the EPA to independently evaluate the availability of dose-response data for pivotal science, rather than relying on publication in a journal or a data availability statement in a published article as evidence of availability. This procedural rulemaking establishes internal Agency procedures and does not intend to change the behavior of external parties, including publishers and scientists. The EPA is also working with scientists through its continued implementation of EPA's Plan to Increase Access to Results of EPA Funded Scientific Research, which seeks to ensure greater public access to the EPA's intramural and extramural research data.
2.1.1.3 Changing Agency Culture is Best Addressed by Guidelines and Not a New Federal Rule
Comment: Commenter (4881) notes that, in order to meet its statutory responsibilities for protecting public health and the environment, the EPA must consider the totality of relevant, reliable, peer-reviewed scientific research. The commenter provides that the Agency's responsibility regarding public access to raw data is already the subject of a regulation requiring EPA, in response to Freedom of Information Act (FOIA) requests, to request and make public "research data relating to published research findings produced under an award" of an EPA grant, subject to privacy and other considerations. The commenter suggests that changing agency culture can be accomplished by a thorough airing of internal guidelines, without resorting to a new federal rule subject to considerable political abuse.
Response: The EPA agrees that it is important to consider the totality of relevant, quality research. Therefore, the EPA did not finalize the 2018 proposal or the 2020 SNPRM proposed option that would have categorically required that the underlying data and models be available for independent validation in order for studies to be considered pivotal science. However, given that transparency is an important aspect of EPA's regulatory actions and assessments, it should be an important consideration in how the Agency considers pivotal science. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations. 
Although these goals might be accomplished through guidelines, the promulgation of these internal Agency procedures in this rulemaking provides a more durable, enforceable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data and highlights the Administration's commitment to increasing transparency in particularly impactful regulatory actions and influential scientific information. The rule will also apply to consideration of research that was not funded by an EPA grant.
2.1.1.4 Political Objective Rather Than Scientific Purpose
Comment: Commenter (4839) asserts that pivotal regulatory science is presented as a cornerstone "to change agency culture and practices" an agenda that embodies a political objective rather than a scientific purpose.
Response: The EPA disagrees that this rulemaking embodies a political objective, as transparency assumes no political ideology. The principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. The broader scientific community has also embraced greater data transparency, as evidenced by data sharing and availability requirements for many high-impact journal publishers and the emergence of organizations, such as the Center for Open Science, and international initiatives like Findable, Accessible, Interoperable, and Reusable (FAIR) data principles; Facilitate Open Science Training in European Research (FOSTER); and Guidelines for Transparency and Openness Promotion (TOP) in Journal Policies and Practices that incentivize greater transparency in research.
2.1.1.5 Lack of Support for Need to Change Culture
Comment: Commenter (4881) provides that footnote number 3 in the Proposal justifies the perceived need to change agency culture by noting that "EPA has not consistently observed the policies underlying this proposal, and courts have at times upheld EPA's use of non-public data in support of its regulatory actions." The commenter notes that in the two court cases highlighted in the footnote, the United States Court of Appeals for the District of Columbia Circuit found unpersuasive the petitioners' claim that the raw data in some of the referenced studies was not publicly accessible. In addition, the commenter adds that the Court dismissed each of the industry petitioners' more technical claims that EPA had imposed "arbitrary and capricious" standards without adequate research support.
Commenter (4881) provides that the stated goal of the Proposal is "to change agency culture and practices regarding data access so that the scientific justification for regulatory actions is truly available for validation and analysis." The commenter suggests that making research results available for validation and analysis is and should be the responsibility of scientists and of the peer-review process overseen by scholarly journals. The commenter notes that scientists have organized workshops to address instances where replicating reported research results has been a problem and that scientific journals have been updating their publication policies to facilitate much more detailed reporting and data archiving.,
Commenter (4848) asserts that the rule promises "to change agency culture and practices regarding data access so that the scientific justification for regulatory actions is truly available for validation and analysis;" however, the new rule in fact just creates new regulatory hurdles by discounting and precluding consideration of long-standing, established scientific practice. Rather than promote the transparency of the scientific information used to create environmental regulations, the commenter suggests that the rule will obscure the democratic process, slow the pace of science and progress, and potentially prevent important health data from being considered by EPA in outlining environmental policy.
Response: As clarified in the preamble to the final rule, because of the potential impact of the EPA's significant regulatory actions and influential scientific information on American lives and livelihoods, the American people deserve environmental decisions and policies that are based on the best information. Only through continuous improvement to its procedures, especially those focused on transparency, can the EPA truly demonstrate that commitment. The EPA's attention on data transparency is also responsive to the broader interest in greater data and model transparency observed in the numerous transparency initiatives in the scientific community and federal government, as well as the criticism the EPA has received from members of the public, scientific community, and Congress on the transparency of the scientific basis for EPA's decisions in many of its previous influential scientific information assessments and regulatory actions. 
Further, this rule exclusively governs the EPA's internal process for handling pivotal science. This rule does not require any researcher or other outside entity to provide data or models to the EPA or the public. The final rule gives greater consideration to pivotal science where the underlying dose-response data are available for independent validation rather than providing for a categorical exclusion of the scientific information. The Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information; therefore, the rule will not prevent important scientific data from being considered by EPA. 
Finally, the EPA disagrees that requirements of this rule will result in any meaningful delay in promulgating regulations. While this rule does require the Agency to evaluate the availability of dose-response data underlying pivotal science, the incremental burden to the Agency to carry out these requirements is expected to be small given (1) the extensive scientific review EPA already conducts regularly and (2) that the requirement is limited to pivotal science (i.e., a small, though highly important, subset of all the studies EPA reviews).
2.1.1.6 Proposal is Counter to EPA's Mission
Comment: Commenters (1433, 2170, 5176, 5179, 6188, 6882, 6884, 6892, 6895, 6915, 8771, 9227) and others that oppose the EPA's proposal contend that the EPA's proposal is counter to the EPA's mission. Comments include:
 If EPA does not use the best available science, it will not be carrying out the mission given to it by Congress to protect Americans and the environment. (1433)
 If EPA wants to change its mission, after nearly 50 years, and advocate for a policy of taking more risks with public health in order to impose fewer costs on industry, that is a policy choice that it should articulate and take comment on. (8771)
 Americans from all walks of life depend on the EPA to create and enforce public safeguards that they trust will protect human health and the environment. The commenter notes that their trust in EPA's work is rooted in the knowledge that agency staff are diligently gathering and reviewing studies of human and environmental risks using their best professional judgement. According to the commenter, a proposal to institute a new system for assessing the validity of scientific evidence undercuts that basis for public trust in EPA and contends that the proposal runs counter to EPA's mission. (6892)
 Commenters (5176, 5179) state that, by promoting an opaque and arbitrary evidence base for policymaking, the proposed rule will erode public trust in the EPA and its regulatory processes. 
 The requirements of the Proposal would lead EPA to adopt less-protective standards across many regulatory programs, which is contrary to EPA's mission to protect human health and the environment. (6915)
 This Proposal is not grounded in a genuine concern for advancing science at EPA and is, in fact, at odds with EPA's mission of protecting human health and the environment. (9227)
Response: The EPA does not agree that this rulemaking is counter to the Agency's mission. The requirements in the final rule advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. As described in the final rule, the EPA will generally use the following factors to assess the quality of studies: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. The EPA will rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies --  based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements or quantitative analyses of an EPA significant regulatory action or influential scientific information will be identified as pivotal science. In compliance with the rule, the EPA shall give greater consideration to evidence, findings, and conclusions of pivotal science where the underlying dose-response data are available in a manner sufficient for independent validation. When the dose-response data underlying pivotal science are not publicly available in a manner sufficient for independent validation, the EPA will give less consideration to the studies' evidence, findings, and conclusions unless the Administrator grants an exemption request. 
The EPA believes this approach will result in significant regulatory actions and influential scientific information based on the highest quality studies that maximize transparency, leading to human health and environmental protections consistent with the statutes the EPA administers.
2.1.2 What Problem is the Agency Intending to Fix?
 2.1.2.1 Question the Purpose of the Rule
Comment: In addition to expressing concerns with the Proposal described elsewhere in this document, commenters (0057, 0569, 0611, 1333, 2171, 2548, 2907, 2909, 3132, 3133, 3644, 4588, 4591, 4594, 4595, 5011, 5021, 5164, 5165, 5170, 5172, 5173, 5176, 5178, 5179, 5420, 6046, 6110, 6111, 6120, 6121, 6122, 6123, 6125, 6127, 6130, 6133, 6135, 6145, 6153, 6155, 6158, 6163, 6166, 6169, 6171, 6185, 6194, 6358, 6373, 6378, 6450, 6872, 6876, 6880, 6881, 6893, 6896, 6898, 6899, 6904, 6907, 6915, 6920, 6924, 6928, 6940, 8272, 8275, 8276, 8281-PH48, 8317, 9225, 9227) and others request that EPA withdraw/rescind the proposed rule. 
Commenters (0572, 0670, 0671, 1006, 1272, 1301, 1311, 1333, 1335, 1375, 1400, 1409, 1433, 1742, 1883, 1905, 1988, 1995, 2003, 2004, 2004, 2159, 2241, 2361, 2408, 2430, 2472, 2492, 2906, 4843, 4872, 4879, 5129, 5131, 5170, 6107, 6109, 6109, 6110, 6113, 6114, 6131, 6139, 6168, 6357, 6367, 6368, 6370, 6373, 6801, 6860, 6876, 6884, 6894, 6905, 6934, 6937, 6944, 8281-PH84, 8317) assert that the proposed rule will not improve transparency.
Commenters (0044, 0671, 1301, 1445, 1589, 2171, 2787, 3135, 4410, 4589, 4595, 4838, 4840, 4845, 4848, 5021, 5022, 5165, 5170, 6111, 6114, 6116, 6119, 6120, 6125, 6127, 6131, 6132, 6133, 6137, 6163, 6164, 6174, 6185, 6188, 6194, 6196, 6351, 6358, 6361, 6373, 6446, 6447, 6450, 6866, 6868, 6874, 6896, 6903, 6904, 6916, 6920, 6925, 6939, 6940, 6944, 8281-PH6, 8281-PH84, 8771, 8801, 8913, 9227) question the problem that the EPA is intending to fix with its proposal. Several of these commenters and other commenters note that the proposed rule is unnecessary (0671, 1445, 1973, 4595, 4844, 4845, 5012, 6110, 6111, 6125, 6132, 6174, 6194, 6196, 6351, 6358, 6446, 6447, 6450, 6696, 6772, 6868, 6872, 6876, 6882, 6895, 6896, 6916, 6920, 6925, 6940, 9227, 8801) and that there are already guidance/mechanisms/requirements (1589, 1973, 4595, 4845, 5012, 5165, 6125, 6137, 6160, 6194, 6195, 6196, 6351, 6358, 6446, 6447, 6450, 6696, 6868, 6872, 6874, 6876, 6895, 6903, 6904, 6916, 6920, 6925, 6940, 8274, 9066, 9227, 8281-PH6, 8281-PH16, 8281-PH84, 8771, 8801) in place and/or being developed to address issues related to transparency/validation and that the rule should be withdrawn. Some specific comments provided to support assertions include:
 Commenters (2171, 2787, 4838, 4848, 5165, 6114, 6116, 6119, 6127, 6131, 6132, 6133, 6137, 6163, 6174, 6194, 6373, 6447, 6866, 6920, 9227) specifically state or infer that the Proposal is a solution in search of a problem. Specific comments include:
 EPA does not explain why the Proposal is preferable to existing recommendations and methods vetted and supported by the scientific community, which endeavor to promote transparency through open science policies and secure data-sharing systems. (6194)
 EPA's Proposal would limit the science the Agency can use to inform decision-making to address an alleged lack of transparency, but there is no crisis or flaw that needs to be addressed. (6174)
 Rulemaking does not identify key gaps in federal environmental policy but rather seeks to obtain transparency in a process that already has multiple checks and balances. (6119)
 Without any significant evidence supporting it, the proposed rule is a solution in search of a problem. The commenter suggests that the Proposal fails to identify specific weaknesses in EPA's current scientific process which is grounded in peer review. The commenter provides that Wendy Wagner, author of two of the studies EPA cites to rationalize the Proposal said in response to the proposed rule: "They don't adopt any of our recommendations, and they go in a direction that's completely opposite, completely different." (2787)
 Strengthening Transparency in Regulatory Science, 83 Fed. Reg. 18768 (Apr. 30, 2018), is a solution in search of a problem. According to the commenter, instead of strengthening ways in which EPA can benefit from advances in scientific studies, the proposed rule limits EPA's access to important studies and hampers the development of regulations needed to protect the public health and welfare of the residents of the District of Columbia and the nation. The proposed rule should be withdrawn. (4838)
 EPA has not provided any evidence that its staff uses "secret science" or "junk science" in decision-making. The commenter suggests that the agenda spearheaded by Congressman Lamar Smith and then former EPA Administrator Scott Pruitt is a solution in search of a problem. For example, the commenter provides that Smith has claimed, with no evidence, that "EPA has withheld data that has been hidden from the American people," There is no evidence of this in the record for the proposal (such as it is), or anywhere else. Similarly, the commenter adds that Pruitt announced this proposal by stating that the "era of secret science is coming to an end." But, according to the commenter, there never was such an era, because "secret science" is a myth. (6447)
 This proposal states that "... EPA solicits comment on this proposal and how it can best be implemented in light of existing law and prior statements of policy that have called for increasing public access to data and influential scientific information use to inform federal regulation. EPA has not previously implemented these policies and guidance in a robust and consistent manner" but the commenters contend that the EPA provides absolutely no evidence to support this statement. The commenters suggest that this appears to be a solution in search of a problem as EPA doesn't offer any evidence that its past decision-making has been inadequate or arbitrary. (6114, 6116, 6131)
 EPA is attempting to undertake these significant changes to its process without identifying particular instances where flawed regulatory science or problems in scientific practice negatively affected EPA action in a manner that the Proposal would prevent in the future. According to the commenter, EPA has failed to cite specific examples of research considered by EPA when promulgating past regulations that were later found to be flawed in a way that the application of the Proposal would have uncovered, or instances where EPA had difficulty verifying the analysis and findings of a study. The commenter suggests that the Proposal appears to be a solution in search of a non-existent problem. The commenter adds that the Proposal does not identify, much less remedy, any flaws in current scientific practice and regulatory process. (6373)
 This rule aims to propose a solution to what some major scientific journals call a replication crisis. The commenter notes that there is no mention in the rule, or its citations, of a study that was shown to be flawed due to a lack of raw data. (6127)
 EPA fails to point to a single example of a case in which, in developing regulations, EPA relied upon a study or studies later found to be questionable or invalid. Having failed to address this foundational question, the commenter suggests that the EPA also misses the questions that would build on that -- even if EPA actually had used invalid science in some instance, EPA would still have to ask whether the underlying data for that study had been made publicly available, and if not, if the problems with the study could have been avoided through having made the data publicly available. (9227)
 The Proposal provides no information supporting the notion that the overarching processes of EPA assessment of relevant scientific studies and subsequent peer review of such assessments, as well risk and policy assessments that EPA has developed and improved over time, are in any way insufficient to address the concerns that are allegedly the main focus of the proposal. (9227)
 EPA has not adequately explained the purpose and rationale for the proposed rule. The commenter provides that the Agency suggests that both the "integrity" and "validity" of its decision making will be strengthened by requiring full public disclosure of the data and models underlying the scientific studies on which it relies. The commenter suggests that the logical implication is that EPA believes those characteristics are currently lacking. The commenter notes that the agency does not explain how it reached that conclusion, or what particular "problems" the rule is intended to solve. The commenter adds that the EPA never explains why, specifically, it believes that existing policies and tools for vetting scientific research are insufficient, why this rule (or any rule) is the best way to address those deficiencies, or why the proposal would better serve and protect the public than its existing policies and practices. (5165)
 There is no indication that EPA science suffers from the so-called "replication crisis" that the Proposal identifies as the principal reason for requiring the public disclosure of underlying data or models for studies used in EPA decisions. The commenter notes that it is telling that the sources EPA cites in support of its claims of a "replication crisis" call into question its existence and in many instances promote solutions that do not involve access to underlying data -- such as looking at cumulative evidence using a variety of methods instead of over-emphasizing the results of a single study. The commenter adds that it is even more telling that the Proposal identifies no EPA actions that have been called into question because the science underlying those actions cannot be validated or replicated. In any event, the commenter notes that the Proposal does not require replication of studies and only limits the cumulative evidence and context in which to interpret any given study -- only hampering EPA's reliance on more robust scientific findings even if such a crisis were to exist. (9227)
 Overarching concern with the Proposal is that it not clear what shortcoming it is designed to address or how EPA would measure the rule's success. While the commenter agrees with the stated objective of increasing "transparency in the preparation, identification, and use of science in policymaking," the commenter suggests that the EPA has not demonstrated that there is a problem that is best solved through a broad, prescriptive mechanism like the Proposal. (6866)
 Although EPA's argument is that there should be a regulation to ensure increased access to dose-response data and models, EPA offers no evidence of an existing problem with dose-response data transparency that needs to be addressed, nor does EPA demonstrate how independent validation outside of existing peer review processes improves upon current data transparency practices and data availability. The commenter adds that EPA offers no evidence that EPA's previous judgement, particularly in the development of dose-response functions, has been inadequate, invalid, or otherwise arbitrary. (2171)
 The EPA provides no basis for its assumption that science or studies for which data are publicly available yield more valid or reliable results than the best available, peer-reviewed, independent, credible science, for which the underlying data are not publicly available. Similarly, the commenter suggests that the Proposal arbitrarily fails to address, much less explain, why prior EPA regulatory actions that relied upon studies, data, or other information did not reflect the "best available science" or why they were otherwise unreliable, despite failing to meet the Proposal's standards. (6133)
 The Proposal represents an unworkable, ill-explained, unjustified, and thoroughly unlawful approach to address a problem that does not exist. EPA does not explain why the data sharing requirements outlined in the Proposal are suddenly so urgent. The commenter suggests that this missing argument is especially significant given the decades of peer-reviewed data and models that EPA has justifiably relied on for regulatory actions. The commenter notes that there is no "crisis in replicability" for the types of data and models that the Proposal purports to address; as an indication of this, and EPA has not cited any sources for its assumptions presented in the Proposal. (6133)
 EPA has presented no scientific reason to prevent use of human health studies simply because the underlying medical records are not available for public inspection and review. (4848)
 Proposal does not show how or why EPA's existing practices fail to adequately achieve these purposes, or how the proposed new procedures would do a better job. In reality, EPA's historic practices in adopting health and welfare standards are extraordinarily transparent, public, and accessible to interested persons. (6137)
 As written, the proposed rule would exclude scientific research if the underlying information is not publicly available in a way that allows for independent validation. However, the notice fails to indicate why such a change is necessary, particularly as the notice acknowledges the existence of laws and policies that promote transparency. While we, the undersigned societies, support transparency and openness of science, we reject the notion that data must be publicly published to be credible and have merit in decision-making. (6163)
 Proposed rule offers very little in the way of examples of non-transparent science. Instead the rule asserts "[a]s a case in point, there is growing empirical evidence of non-linearity in the concentration-response function for specific pollutants and health effects." According to the commenter, that is true, however, it has nothing to do with lack of public access of underlying research data. Instead, the commenter expresses that it calls for increased federal funding of research on non-linearity of pollution health impacts. (6920)
 There is no need for this rule. According to the commenter, in conflict with the oral comments by Steven Milloy, who they state incorrectly suggested in his public EPA testimony that the American Cancer Society (ACS) and the Six Cities Studies' researchers have not been willing to release their data for validation by others, both of these studies have actually already provided their raw data for validation by the independent and unbiased Health Effects Institute (HEI) (HEI, 2008). The commenter asserts that such independent evaluations could easily be applied to any new cases of concern for data validation, without the risks implicit in the EPA's proposed rule. Thus, the commenter states that this dangerous rule seeks to needlessly "solve" a purported problem that does not exist. (6132)
 Several commenters (6133, 6137, 6185, 6772) cite court cases that rejected the notion that the EPA must obtain and disclose underlying studies.
 EPA tries to justify the Proposed Rule as a way to ensure the Agency is not arbitrary and capricious in its conclusions. But the commenter states that the D.C. Circuit has twice rejected the notion that EPA must obtain and disclose data underlying the studies it relies on in National Ambient Air Quality Standards (NAAQS) development. (6137)
 In light of the court's holding in American Trucking Assns. v. EPA [283 F.3d 355 (D.C. Cir. 2002)] that differentiated the substantial difference and administrative hardship between reliance on peer-reviewed scientific studies cited in a rulemaking record rather than on the raw data underlying those studies, and without additional clarity from EPA, we are having difficulty identifying the problem EPA seeks to address with the Proposal. Therefore, the commenter requests that the Agency withdraw it. (6185)
 The Proposal is unnecessary because court cases have produced rulings stating that, while the EPA should be as transparent as possible, the EPA is not required, for its decision making, to obtain or analyze raw data in order to rely on studies that cannot provide access to that underlying information. (6772)
 Commenter (6109) cites Harvard's Environmental Law program's provided analysis: 
                  The failure of Pruitt's proposal to meet these requirements reflects its lack of validity on the merits. While it's true the proposal reflects the longstanding practice that the EPA rely only on impeccably trustworthy science when formulating standards, performing benefit-cost analyses, and evaluating toxic substances, it fails to articulate either (i) that there is a problem with how the EPA has done its work over the past four decades or (ii) what are the requirements, methods, and practices Pruitt believes are necessary to achieve that goal beyond those long since in place at the agency. Instead, it simply asserts that making certain data available to the public in and of itself will increase the reliability of the science the agency incorporates in performing its various actions. Nowhere does it explain why or how the new data availability requirement would enhance the trustworthiness of the science the agency relies on.
                  In fact, . . . it is through the scientific peer-review process that the ostensible goals of the proposal can be, and are being, achieved. Since the EPA already relies almost exclusively on peer-reviewed science, it's essential that the proposal explain exactly what its views are as to the adequacy of peer review and how, if the EPA believes current peer-review is inadequate, the proposal specifically addresses those inadequacies."
 The main provisions of the Proposed Rule are either unnecessary, duplicative, burdensome, and likely to be confusing in their application. (4589)
 According to the commenter, the EPA provides no examples of where and how, in the agency's view, past rulemakings specifically failed to make these efforts, and how EPA would change past practice in this context. (5021)
 Commenter urges the EPA to withdraw the rule unless it can be revised to clarify its intent and purpose, and more clearly explain how it would contribute to protecting human health and the environment. (6120)
 EPA has not established a legitimate need for the proposed rule. The commenter provides that the EPA has made thousands of regulatory decisions over the last 50 years. The commenter notes that the Congressional Budget Office (CBO) estimates that EPA "relies on about 50,000 scientific studies annually to perform its mission." The commenter suggests that the proposed rule fails to identify a single regulatory action based on faulty science. According to the commenter, the rule is not needed or warranted and will do far more harm than good. (6111)
 If EPA has any doubts or concerns regarding the merits of a particular study, it must address those doubts or concerns for that particular study in the context of a given rulemaking, where agency staff, internal scientific experts, scientific advisory committees, or commenters contend that study is relevant. The commenter contends that the EPA has provided no scientific justification for ignoring an entire class of health science simply because the underlying data has not been disclosed. (6137)
 The EPA invokes "strengthening transparency" as a primary driver for the Proposal but fails to describe how a perceived lack of transparency has hampered past rulemakings. The commenter suggests that it provides no examples of where "EPA has not previously implemented these policies and guidance in a robust and consistent manner" nor what are the specific "agency culture and practices regarding data access" that require changing. (5021)
 It would be relatively simple for EPA or a chartered external review group to analyze a sample of EPA's decisions to see if there really is a problem or whether this transparency rule would have made a difference. The commenter asks why this was not done? (6139)
 Based on the lack of meaningful information and articulated or demonstrated need for the proposed rule, EPA has not made the case for a new regulation at 40 CFR Part 30. (4410)
 EPA has advanced no evidence of a crisis of falsification of underlying data on the part of the U.S. research community. (8913) 
 EPA's own Science Advisory Board (SAB) voted to review the Proposal on June 1, 2018. The commenter states that this action is in addition to the vehement reaction that the Proposal has prompted from the most reputable scientists, scientific organizations, and scientific publications in the country directly challenging the need for this proposal -- there is no "secret science." (6174)
 Request that the EPA withdraw this proposed rule as the notice does not provide a clear rationale. (5170)
 There is no indication and no specifics regarding alleged deficiencies in research data relied upon by EPA. EPA seeks to apply this new regulation across the board, with no grounding in law or fact as to where and how specific studies have caused regulatory overreach to the public's detriment. Without a reasonable basis to adopt this broad, sweeping, ill-defined regulation, EPA will likely face multiple lawsuits seeking to strike down this new regulation, distracting the EPA from its true, important mission. (6116, 6131)
 Rather than addressing a legitimate problem, the commenter suggests that the Proposal is a next step in a very transparent effort to discredit and make unavailable certain seminal studies that establish the connection between exposure to air pollution and adverse public health impacts. (6174)
Response: The EPA provides more detail explaining the EPA's rationale for this rule in the preamble for the final rule. In brief, the transparency provisions in this final rule are intended to build upon existing efforts and provide incremental progress toward the Agency's overarching and long-standing goal of greater transparency. The EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. The scientific community has also embraced greater data transparency, as evidenced by data sharing and availability requirements for many high-impact journals and the emergence of several international organizations and initiatives focused on increased transparency and data sharing in science. The EPA supports these efforts and believes that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. The incremental progress made by this rule provides an important step towards prioritizing transparency in particularly impactful rules and assessments and will inform future statute-specific rulemakings.
The EPA also agrees that it is important not to categorically exclude any study because the data underlying a study at the stage required by this rulemaking may not be available for independent validation. Therefore, the EPA is not finalizing the proposal that would have categorically required that for studies to be considered pivotal science the underlying data would need to be available for independent validation. However, given that transparency is an important aspect of EPA's regulatory actions and assessments, it should be an important consideration in how the Agency considers pivotal science. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
Comment: Commenter (0044) states that they do not understand the purpose of this potential rule -- is the scientific acumen and/or honesty of the EPA's professional staff under question? Has there been evidence that any standards have been done using inappropriate science? The commenter (as a citizen) states that they want government scientists, who are disinterested, to be the arbiters of pollution levels, not industrial scientists who get paid by their companies. The commenter also asks whether EPA identified significant data validity problems associated with toxicity study submissions? The commenter provides, of the approximately 810 inspections performed by the EPA for Good Laboratory Practice (GLP) studies, 27 were either rejected by agencies or became the subject of consent degrees, which computes to 31/3%. Assuming that the rejections by authorities not resulting in consent decrees were due to poor quality while only the consent decrees were due to data falsification/fabrication/etc., the commenter states that the actual rate of fraudulent data is much less. According to the commenter, this does not warrant major changes in what data are acceptable for EPA submission, but rather to enhancing regulatory inspections to assure data quality.
Response: This rulemaking is not intended to address the conduct of EPA's professional staff nor to remedy a particular instance in which the EPA relied on a poor-quality study or a study for which the conclusions could not be independently validated. The transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency. Transparency is an important aspect of EPA's regulatory actions and assessments, and it should be an important consideration in how the Agency considers pivotal science.
Comment: Commenter (8801) states that a key point in the proposed regulation is that data from studies used in regulatory decision making be made available to third parties for conducting their own analyses. The commenter contends that this justification fails to recognize that EPA has successfully addressed this issue in the past. For example, the commenter notes that when a series of important studies done in the 1990s by the Harvard School of Public Health and the ACS on effects of fine particle air pollution on human health were questioned by opponents of air pollution regulations, a re-analysis of the data in 2000 by the reputable HEI was undertaken. The commenter states that the independent re-analysis confirmed the findings of the original studies. According to the commenter, this demonstrates that experts independent of the regulators and the regulated can already satisfy one of EPAs primary arguments for the need of the proposed regulation.
Response: Although some mechanisms are in place to allow for data sharing and reanalysis, these mechanisms are not necessarily commonplace. The transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency. The incremental progress made by this rule provides an important step towards prioritizing transparency in particularly impactful rules and assessments and will inform future statute-specific rulemakings. 
2.1.2.2 Existing Mechanisms that Already Address Validity Concerns
Comment: Commenters (1335, 2119, 2908, 4589, 4590, 5170, 5178, 5977, 6111, 6119, 6125, 6137, 6157, 6158, 6160, 6194, 6351, 6373, 6446, 6447, 6872, 6874, 6920, 6924, 6925, 8801, 9227) suggest that current EPA practices are sufficient and provide examples of how existing mechanisms/policies/procedures work to ensure proper review and validation and that the proposal does nothing to improve them. Comments include:
 The EPA already has methods in place, and the proposed rule does nothing to improve upon them while creating unnecessary obstacles for EPA's consideration of the best available evidence for policy and rulemaking. (6160)
 The proposed rule does not acknowledge that the epidemiologic science community ... has been making significant efforts to make data available where possible and to develop studies based on publicly available data where appropriate. (6447)
 EPA's regulatory process already involves peer review, solicitation of expert opinion, steps to ensure data quality, and public comment periods that allow stakeholders to raise questions or concerns and present additional evidence. (5178, 6874) If it had undertaken a similarly thorough process in developing this rule, it could have asked the National Academies of Sciences, Engineering, and Medicine (NASEM) to investigate and report on whether the current level of transparency has resulted in outcomes contrary to EPA's statutory mandates or executive orders; procedures and considerations related to transparency in the scientific community; and the potential results of applying the proposed criteria when regulating. (6874)
 The ultimate safeguard to assure that EPA regulatory action is based on high quality science is the fact that EPA's regulatory action is subject to the notice and comment requirements of the Administrative Procedures Act (APA) (APA, 6 1.J.S.C. § 553) (APA) and ultimately to judicial review. To the extent that scientific information proposed as a basis for regulatory action falls short, the commenter states that the APA provides members of the public, regulated businesses, and other scientists an opportunity to comment to the agency about those shortcomings and to submit contrary studies and information for agency consideration. Following adoption of a regulation, interested parties may seek judicial review to determine whether it is arbitrary, capricious, or lacking in evidentiary support. (5 1.1.S.C. § 706.) According to the commenter, the APA requirements, and the opportunity for judicial review of EPAs regulatory action, protect against the adoption of regulations that lack high-quality scientific support. (6446)
 The proposed rule ignores a host of existing methods and best practices already established -- and adhered to -- by the research community to ensure the transparency, reproducibility, replicability, objectivity, and validity of studies, analyses, models, and reports. (6111)
 The EPA SAB review process already provides assessment of the adequacy of the approaches taken in the studies considered in setting a standard. (1973, 6125, 6446, 6447, 6920, 8801, 9227)
 Congress created the SAB to provide the Administrator with scientific advice. (42 U.S.C. § 4365.) The SAB is required by its authorizing legislation to make every effort to maximize public participation and transparency, "including making the scientific and technical advice of the [SAB] and any investigative panels . . . publicly available in electronic form on the website of the Environmental Protection Agency." The commenter notes that the proposed rule does not mention the SAB, or its important function in providing expert assessments of whether scientific research offered as the basis for regulatory action is, in fact, the best available science. (6446)
 SAB points out multiple ways in which EPA and the scientific community already ensure data quality, obviating the need for any new restrictions. For example, "[t]he proposed rule fails to mention that there are various ways to assess the validity of prior epidemiologic studies without public access to data and analytic methods." The commenter adds that the SAB describes one specific example in which "an unusually rigorous form of peer review and independent reanalysis, coupled with many follow-up studies, has accomplished a measure of confidence in findings without public access to data and analytic methods." In addition, "[t]he proposed rule fails to mention that EPA has mechanisms for vetting science through several expert panels," and that review process "goes beyond the typical journal peer review procedures." (6447)
 As part of the process for the evaluation and publication of scientific research, the commenter notes that all of these studies invariably undergo rigorous peer review by independent experts. Peer review is the world-wide gold standard for evaluating scientific research. The commenter submits that no study is published in any reputable journal without undergoing peer review. Thus, the commenter asserts that any pivotal study used for regulatory decision making has already undergone such review. Moreover, the commenter adds that, in the event that EPA determines that further review is necessary, its own expert panels, such as the SAB, exist to serve such a purpose. Unfortunately, according to the commenter, EPAs intentional crippling of its own SAB, involving the replacement of qualified independent experts by industry 7 employees with obvious conflicts of interest, damages the credibility of the SAB to do its job in an impartial manner. (8801) 
 Commenters (4845, 6110, 6174, 6446, 6872, 6920) infer that the EPA is already transparent in its use of peer-reviewed research in regulatory science. 
 The proposed rule ignores that much of the scientific research considered in the regulatory context, and all research published in reputable journals, has already been subject to extensive peer review. The commenter adds that, while the proposed rule would place judgments regarding the science that EPA may use in the hands of political appointees -- with its suggestion of an ad hoc mechanism for making exceptions to the rule's limitations -- the method that is accepted in the scientific community for assessment of the strength of scientific research is peer review. According to the commenter, the fact that scientific studies are regularly peer reviewed, including peer review by expert panels such as the SAB, renders the proposed rule unnecessary. (6446)
 The commenter adds that situations in which personal data need to be protected are clear and do not compromise the validity of the scientific studies. (6174) 
 It is a standard practice for EPA to conduct comprehensive reviews of the best available scientific research when evaluating air and water quality standards. The commenter states that the EPA relies on peer-reviewed studies that describe the underlying data, methods, assumptions, sensitivity, and uncertainty of the results. These studies are cited and are published and available for public review. The commenter adds that reproducibility and independent validation are critical aspects of the scientific method and have resulted in significant advancements in our understanding of the health effects of air pollutants at different exposures and thresholds. (6920)
 The EPA also relies on independent advisory panels comprised of nationally recognized experts, such as the SAB, and the CASAC, to review and evaluate the state of the research and to provide an additional layer of independent peer-review. (6920)
 EPA science assessments generally include an exhaustive and critical review of relevant studies and a full explanation of how they are being interpreted. Extensive information about each study is typically part of the public record, even if all underlying data may not be included. EPA assessments are normally subject to public comment and independent peer review. And members of the regulatory community are free at any time to replicate studies they deem flawed or to independently seek access to underlying data and reanalyze them. In short, the "problem" that the proposed rule seeks to fix is largely imaginary. (4845)
 EPA already develops and implements policies, guidance, risk assessments, and toxicology assessments using detailed and transparent approaches that include extensive peer review and public input. (6872) 
 There are already Health Insurance Portability and Accountability Act (HIPAA)-compliant and Institutional Review Board (IRB)-approved methods to transfer data between researchers so that studies can be evaluated and verified, and to determine if findings can be replicated. (5012)
 There exist federal guidelines that provide access to and ensure the quality of scientific information used for federal policies and regulation which also emphasize the necessity to maintain needed privacy or assurance of suitable confidentiality. Furthermore, according to the commenter, there are credible procedures for testing results and verifying outcomes with methodologies that do not require access to raw data. There are also procedures for providing protected, limited access to the full raw data to independent researchers for independent validation without making the full data publicly available. These procedures negate the need for this proposed rule, as they respect privacy laws and bona fide practices used by the scientific community. (4595)
 Several aspects of the proposal are already standard Agency practice. For example, the commenter states that the proposed §30.4 would require EPA to "identify all studies (or other regulatory science) relied upon when it takes any final agency action." According to the commenter, the EPA offers no evidence to suggest that the Agency has ever failed to identify studies or science that it relies upon. Similarly, the commenter states that the proposed §30.6 would require the Agency to "describe and document any assumptions and methods used, and [] describe variability and uncertainty." Again, the commenter asserts that this is standard Agency practice, and EPA has not provided any evidence to suggest that its staff has ever failed to follow this practice. (6447)
 EPA already has significant data sharing requirements and does not need more. (0671)
 Key information about studies is already publicly available, including study protocols, recruitment criteria, measurement techniques, and statistical modeling methods including lists of the adjustment variables have all been made publicly available, and peer reviewed. According to the commenter, these protocols and guidelines, such as CONSORT, ARRIVE and STROBE, do not require public access to all study data but still improve the scientific basis of evaluating studies. (1973)
 The Proposal ignores, or presents no argument for rejecting, scientific practices embodied in the host of nongovernmental methods and best practices established and adhered to by the research community to ensure the transparency, objectivity, and validity of studies, analyses, models, and reports, including by reproduction and/or replication where appropriate. (6194)
 The EPA's "Scientific Integrity Policy" provides guidance to Agency employees to ensure that science is used and communicated "with honesty, integrity, and transparency." Part of the policy to enhance transparency includes direction that:
 The Agency will continue to expand and promote access to scientific information by making it available online in open formats in a timely manner, including access to data and non-proprietary models underlying Agency policy decisions. Further, the use of nonproprietary data and models are encouraged, when feasible, to increase transparency. (6903)
Commenter (6446) asserts that the lack of any explanation for a need for the proposed rule is doubtless because procedures are already in place that assure use of the best available science for EPA's regulatory activity. The commenter submits that these procedures include the SAB established by Congress in 1978 to advise on scientific matters (42 U.S.C. § 4365.), existing independent peer review of much of the scientific research relied on for regulatory action, and legal requirements that prohibit adoption of regulations that are arbitrary, capricious, or unsupported by substantial evidence (e.g. 5 U.S.C. §. 706). When consistently implemented, the commenter states that these procedures and requirements, among others, assure that EPA's regulatory actions are based on the best available science and cast substantial doubt that the true purpose of this rulemaking has anything to do with strengthening the validity of regulatory science.
Commenter (9227) suggests that the Proposal neither acknowledges the mechanisms EPA already uses to ensure the integrity of science in decision-making nor establishes that there is a problem that the Proposal is needed to solve. The commenter submits that the reality is that both Congress and EPA have established an array of mechanisms and safeguards over the last five decades to ensure that the Agency's decisions are grounded in best available science. The commenter notes that these mechanisms include review of agency science and decisions by EPA's scientific advisory boards, including the SAB, the CASAC, Board of Scientific Counselors, the Science Advisory Committee on Chemicals, and the Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA) Scientific Advisory Panel -- a process that a work group of the SAB recently described as a "rigorous review process that goes beyond the typical journal peer review procedures," and that the National Research Council (NRC) recognized as playing an "important role in helping EPA to ensure the credibility and quality of . . . science-based decisions." The commenter adds that the Proposal also ignores EPA's use of independent peer review processes to evaluate certain studies used in regulatory decisions; the use of transparent literature surveys that are themselves subject to peer review and public comment, such as the ISA that inform the NAAQS; and independent review of EPA science programs and risk assessment practices by authorities such as the NRC. Lastly, the commenter states that major regulatory decisions -- and the underlying scientific bases for those decisions -- are also subject to public comment and judicial review, which serves as an important check on agency decisions that fail to properly account for the best available science.
Commenter (9227) asserts that thanks to these multiple and overlapping safeguards, the quality of the science underlying EPA decisions is robust. In addition, the commenter provides that numerous independent reviews of EPA's science-based actions by the courts, as well as the consistency with which the Agency has solicited and relied on the advice and approval of its external SAB committees have added to the credibility of EPA's decisions. 
Commenter (6125) states that the EPA's past practice reflects the governing authorities far better than the current proposal. Before 2017, the commenter notes that the EPA over the years had developed and refined its approach to these-case-by case assessments, using a variety of methods that include examination of the data underlying studies where it is available, but without exclusive reliance on that examination, much less making it the foundation of a per se rule. Such examinations typically begin with EPA expert staff identifying and assessing peer reviewed studies and studies published in reputable scientific journals. This includes examining the strengths and weaknesses of individual studies, including factors such as design, the reputation and past work of the researchers, and quality assurance methods and analyses. This is followed by a broader look at the consistency and coherence of the study at issue with similar study types across multiple studies, as well as a more integrated assessment of the weight-of-evidence that considers multiple lines of scientific evidence. The assessments are in turn peer reviewed by panels of EPA's SAB as well as the public.
In conducting such assessments, the commenter (6125) provides that greater weight is given to the results of studies that have been replicated by other investigators using different data sets than to individual studies with limited or no replication. The commenter notes that replication refers to additional studies testing the same hypotheses as the original study, typically conducted by other investigators using different study designs and data sets, to see if the same conclusions are reached. As one of the authors EPA cited in the proposal recognized, replication is "the cornerstone of the scientific method." "Designing and conducting a replication study does not require access to raw data from the original study; this would abrogate the concept of independence." In reviewing the epidemiology studies supporting air quality standards, the commenter states that the EPA has placed greater weight on studies that have been replicated. In cases where one or two new studies appear to break new ground, the commenter notes that reanalysis by EPA or competent third-party investigators can provide some additional credibility, but not as great as that provided by replication.
Response: The EPA agrees that existing mechanisms within the EPA, the federal government, and in the scientific community are increasingly pushing towards greater data sharing and increasing scientific rigor. The EPA supports these efforts and is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. The EPA believes that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. 
As described in the final rule, the EPA will use existing factors (including soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review) to evaluate study quality. When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence (consistent with Agency guidelines on hazard identification and dose-response assessment) to support a relationship between exposure and effect, the EPA will identify a subset of those studies for use in characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect. This will be based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both will be identified as pivotal science. Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. In compliance with the rule, the EPA shall give greater consideration to evidence, findings, and conclusions of pivotal science where the underlying dose-response data are available in a manner sufficient for independent validation. When the dose-response data underlying pivotal science are not publicly available in a manner sufficient for independent validation, the EPA will give less consideration to the studies' evidence, findings, and conclusions unless the Administrator grants an exemption request.
Although current methods and processes do allow for public comment and, where appropriate, peer review in the development of significant regulatory actions and influential scientific information, subject matter experts, peer reviewers (including the SAB and other science advisory panels), and the courts generally have not had access to the underlying data necessary to fully evaluate scientific conclusions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. The EPA is confident that this approach will result in significant regulatory actions and influential scientific information that are based on the highest quality, most relevant studies and ensure adequate human health and environmental protections, while maximizing transparency and maintaining consistency with the statutes the EPA administers. 
Comment: Commenters (6137, 6446, 6920) provide that the process for establishing NAAQS illustrates the scientific review procedures that are already in place for EPA's regulatory actions. 
Commenter (6446) notes that in the NAAQS process, research results that are given the most weight are from the peer-reviewed literature. After EPA staff and their contractors review the literature, the commenter states that the CASAC of the SAB, which consists of internationally recognized experts in their scientific disciplines, reviews the EPA staff reports. The commenter notes that the Committee provides advice to the Administrator regarding the adequacy of current standards and recommendations for revisions, if necessary, for the protection of public health. The commenter adds that EPA staff reports also receive public comment, including from independent and industry scientists, and the Committee review includes consideration of those public comments. The commenter states that EPA NAAQS documents typically receive multiple rounds of expert scientific review before they are finalized.
Similarly, commenter (6920) provides that the NAAQS review process is a model of scientific transparency. Its reviews are renowned for their high quality. In regard to the studies on which they are based, the commenter states that the costs of publication as well as requirements for the privacy of study subjects have prohibited making all the data obtained publicly available. The commenter strongly disagrees that this diminishes the value of such epidemiology and toxicology studies to regulatory science and states that exclusion of such studies would significantly reduce the amount of scientific data available to establish appropriate standards to protect public health and the environment.
Commenter (6137) by way of example provides that the EPA's NAAQS process begins with preparation of an ISA, an extensive review of available science relevant to the development of NAAQS. See EPA, Integrated Science Assessment for Ozone and Related Photochemical Oxidants, EPA 600/R-10/076F, at 1i-1ii (Feb. 2013). The commenter adds that the preparation of the ISA is preceded by a public workshop and call for information. Id. EPA then collects and screens studies (with a heavy focus on studies that have been peer reviewed), prepares an initial characterization of evidence, and then provides a peer review process of the initial draft materials for scientific quality of "building blocks" from scientists from both outside and within EPA. Id. at 1vii. The commenter states that there is then preparation of draft syntheses of the studies and draft conclusions and causal determinations, followed by CASAC input and an opportunity for public comment before preparation of the final ISA. After that, the commenter notes that EPA staff prepares a Policy Assessment (PA) based on integration and interpretation of the findings of the ISA and a separate risk and exposure assessment (REA). The commenter provides that both the PA and REA are themselves subject to separate rounds of public comment and there is yet another round of public comment after EPA proposes its action on the NAAQS. 
The commenter (6137) states that all of the foregoing comprises one of the most open, publicly accessible processes ever devised for the development of health standards. The commenter notes that there are multiple layers of peer review and more than ample opportunity for the public to raise questions about the adequacy and accuracy of the studies and models presented. The commenter expresses that the EPA does not and cannot rationally explain why this system requires yet another layer of complexity to provide adequate transparency, integrity, and public understanding of the process.
Additionally, commenter (6137) notes that the EPA's process for revising ambient water criteria for the protection of human health provides another example of the transparency afforded to its rulemaking, as it provides open access to information for interested persons. In 1998, the commenter provides that the EPA "improved" this process "to provide expanded opportunities for public input, and to make the process more efficient." 63 Fed. Reg. 68,354, 68,355 (Dec. 10, 1998). To revise its ambient water criteria, the commenter states that the EPA must follow a multistep process. First, EPA must "undertake a comprehensive review of available data and information" before developing draft criteria. Id. Second, EPA must "publish a notice in the Federal Register and on the Internet announcing its assessment . . . of the pollutant" which "describe[s] the data available to the Agency," and solicits "scientific views as to the application of the relevant Agency methodology." Id. Third, EPA must "utilize information obtained from both the Agency's literature review and [from] the public [comments] to develop draft recommended water quality criteria." Id. Fourth, and concurrent with the development of the draft criteria, EPA must publish in the Federal Register a notice soliciting the public's "scientific views on the draft criteria," and must "initiate . . . a documented critical review by qualified independent experts." Id. Fifth, EPA must then evaluate and respond on the record to all "[m]ajor scientific issues" (if any) raised during the peer review or public comment period. Id. Finally, EPA must "revise the draft criteria as necessary, and announce the availability of the final water quality criteria in the Federal Register and on the Internet." Id. According to the commenter, like the NAAQS process, this process too is fully open and transparent.
Moreover, the commenter (6137) contends that it would be irrational and impracticable to require EPA to seek out the underlying data for, and demand separate peer review of, all data and models the Agency uses for purposes of characterizing the quantitative relationship between dose or exposure and magnitude of a predicted health or environmental impact. For instance, the commenter notes that in the last ozone NAAQS review, EPA reviewed more than 4,000 studies and references for the ISA, and cited more than 2,200 in the final ISA. See EPA, Health and Environmental Research Online, ISA Ozone (2013) (last updated July 2, 2018), https://hero.epa.gov/hero/index.cfm/project/page/project_id/1628. The commenter asserts that it is neither practicable nor necessary for EPA to demand production of underlying data from thousands or even hundreds of studies, and to require new peer reviews of each and that EPA cannot construe the statute in a way that would render it impossible to complete the review and revision of the NAAQS that Congress mandated.
Response: The EPA agrees that existing mechanisms within the EPA include meaningful opportunities for peer review and public comment. Although the process and justifications for scientific decision making is clear, the public, subject matter experts, and peer reviewers (including the SAB and other science advisory panels) generally have not had access to the underlying data necessary to fully evaluate scientific conclusions. Where underlying dose-response data in pivotal science are available, subject matter experts would be able to reanalyze the data and validate the original research findings, check for errors, test alternative assumptions, and/or better understand and evaluate the implications of the uncertainty. As the EPA better understands the data, the Agency can make stronger, more informed decisions in future rulemakings or in revisions to existing rules or influential scientific information. The incremental progress made by this rule provides an important step towards prioritizing transparency in the EPA's most impactful rules and assessments and will inform future statute-specific rulemakings. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent subject matter experts to validate pivotal science, and as data are better understood through independent reanalysis the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Further, the EPA agrees that determining data availability for all the studies EPA considers in significant regulatory actions and influential scientific information may be infeasible at this time. The EPA modified the scope and the definition of "pivotal science" in the final rule to clarify that the requirements apply to the dose-response data underlying pivotal science and that "pivotal science" is a subset of the studies used in the analysis. The EPA also modified language in 40 CFR 30.6 to clarify that the Agency is not required to peer review to pivotal science that has already undergone adequate peer review. 
The EPA would also like to clarify that this rule does not demand the release of underlying data. As stated in the preamble, this final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science. The EPA is only required to determine if such a mechanism was available, should a subject matter expert decide to use it.
Comment: Commenter (6160) provides that, when assessing scientific evidence for policy and rulemaking, EPA employs a weight‐of‐evidence method that "considers all relevant information and its quality, consistent with the level of effort and complexity of detail appropriate to a particular risk assessment," according to the EPA Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental Protection Agency ("EPA Guidelines"). To insure that the information used in policy and rulemaking is accurate, reliable, and unbiased, the commenter states that the EPA Guidelines require the use of "(i) the best available science and supporting studies conducted in accordance with sound and objective scientific practices, including, when available, peer reviewed science and supporting studies; and (ii) data collected by accepted methods or best available methods ..." Furthermore, the commenter adds that the EPA Guidelines already address the transparency of data and methodology, including "(1) the source of the data used, (2) various assumptions employed, (3) the analytic methods applied, and (4) the statistical procedures employed."
In the General Assessment Factors for Evaluating the Quality of Scientific and Technical Information ("EPA SPC Assessment Factors"), the commenter (6160) provides that the EPA Science Policy Council has established five additional criteria to support the EPA Guidelines. These include:
 Soundness ‐ The extent to which the scientific and technical procedures, measures, methods, or models employed to generate the information are reasonable for, and consistent with, the intended application;
 Applicability and Utility ‐ The extent to which the information is relevant for the Agency's intended use;
 Clarity and Completeness ‐ The degree of clarity and completeness with which the data, assumptions, methods, quality assurance, sponsoring organizations and analyses employed to generate the information are documented;
 Uncertainty and Variability ‐ The extent to which the variability and uncertainty (quantitative and qualitative) in the information or in the procedures, measures, methods, or models are evaluated and characterized; and
 Evaluation and Review ‐ The extent of independent verification, validation, and peer review of the information or of the procedures, measures, methods, or models.
The commenter (6160) notes that the EPA utilizes these assessment factors to evaluate the scientific quality of individual studies that will contribute to the weight‐of‐evidence approach. According to the commenter, this balanced approach that is already in use and is consistent with existing scientific practice is more than sufficient to assess the quality of science that is relied upon for policy and rulemaking.
Commenter (6160) states that existing EPA scientific processes and assessment factors are part of various statutory frameworks, including the guidelines issued by EPA. The statutory basis for this proposed rule is not presented in a manner to support upending the existing EPA scientific processes and assessment factors.
Response: The EPA agrees with this comment in part and has clarified the role of the EPA Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental Protection Agency and assessment factors in the final rule. As described in the preamble to the final rule, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information, consistent with the EPA Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental Protection Agency. The EPA will generally continue to use the following, established factors when assessing study quality: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. The EPA also clarified that it is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. 
Comment: Commenter (6194) asserts that existing laws, regulations and guidance at the federal level (i.e., the Information Quality Act, OMB and EPA's guidelines implementing the Information Quality Act, Circular A-110, and the 2009 Memorandum for the Heads of Executive Departments and Agencies on Scientific Integrity) advance transparency-related goals and make the Proposal redundant.
Response: The EPA agrees that existing mechanisms within the EPA and the federal government are increasingly pushing towards greater data sharing and increasing scientific rigor. The requirements in this final rule build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Although current methods and processes do allow for public comment and, where appropriate, peer review in the development of significant regulatory actions and influential scientific information, subject matter experts and peer reviewers (including the SAB and other science advisory panels) generally have not had access to the underlying data necessary to fully evaluate scientific conclusions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty in the assessment. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. The incremental progress made by this rule provides an important step towards prioritizing transparency in particularly impactful rules and assessments and will inform future statute-specific rulemakings.
Comment: Commenter (6920) notes that EPA also has existing policies and procedures in place to prevent conflicts of interest and ensure that the boards and committees are well-balanced and comprised of independent members with the appropriate expertise. The commenter provides that processes for developing human health assessments and setting and reviewing standards by EPA have been routinely scrutinized by organizations such as the National Academy of Sciences (NAS) and the Government Accountability Office (GAO) and EPA has incorporated their recommendations and improved its approach over time. 
As examples, commenter (6920) notes that several of the landmark studies on the health effects of air pollution, such as the Harvard Six Cities and ACS studies, have been peer-reviewed and re-analyzed by multi-disciplinary expert panels from the HEI. The commenter describes HEI as an independent non-profit research institute that receives funding from both EPA and industry to provide impartial credible science on the health effects of air pollution. In a testimony before a Congressional committee, the commenter notes that the President of HEI stated that "US EPA and other agencies have established procedures to produce and review science for decisions, and in many cases those procedures work to enhance the quality and credibility of the science."
Commenter (6920) expresses that the proposed rule uses the phrase "best available science," but calls into question established processes such as EPA's Integrated Risk Information System (IRIS) review program and NAAQS program. In fact, the commenter states that the EPA routinely makes the `best' use of scientific information in these programs. That includes, according to the commenter, study-by-study evaluation of strengths and weaknesses of all relevant research. Also, the commenter asserts that the EPA more-than-adequately explains its decisions and analyses both in recommending NAAQS revisions and in quantifying chemical toxicities in IRIS.
Commenter (6920) submits that IRIS is not a regulatory program, but it provides essential scientific information for decisions made by Washington State Department of Ecology. In a recent review of the IRIS program, the commenter states that NAS reported that EPA has made "substantial progress" in implementing the recommendations outlined in previous NAS reports, improving the program's overall scientific and technical performance.
Commenter (6920) submits that IRIS uses a rational weight-of-evidence method to assess available research and provide access to a comprehensive source of toxicity data. The commenter notes that it increases the capacity to evaluate chemical hazards, and to quantify risk magnitudes and uncertainties, but it does not inform on how to manage risks. The commenter contends that the proposed exclusion of research lacking all underlying data from IRIS would be an unnecessary waste of information.
Response: The EPA agrees that scientific studies should not be categorically excluded from consideration when promulgating significant regulatory actions and developing influential scientific information and has clarified this in the preamble of the final rule. As described in the preamble to the final rule, the EPA will consider data availability, among other factors, when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available for independent validation. When the dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA will give less consideration to the studies' evidence, findings, and conclusions unless the Administrator grants an exemption request. 
2.1.2.3 Recommend that EPA Continue to Implement Existing Procedures/Policies
Comment: Commenters (2119, 5172, 5178, 5977, 6119, 6160, 6351, 6924, 6925) urge EPA to continue to implement longstanding EPA procedures that promote use of sound science. Comments include:
 It is assumed that EPA's regulations are already and always based on the best science, which regulates itself in proving facts and providing solutions. (5977)
 Commenter urges EPA to follow the current, effective measures in place to ensure the use of robust, uncensored scientific research to protect the health of our patients and our communities. (6361)
 Commenter suggests EPA focus on implementing existing initiatives and guidelines for improving data sharing and transparency at federal agencies. (6924)
 Commenter urges EPA to continue to rely on the existing EPA weight-of-evidence process and assessment factors that are based in scientific standard practices. The commenter does not support a rule that departs from these important EPA-recognized approaches for assessing the quality of science in policymaking. (6160)
 There is an existing and well-established framework in place governing the use of scientific information to ensure that the information informing regulatory actions is the best-available, peer-reviewed science. The commenter provides that this includes EPA's Scientific Integrity Policy, OMB's Guidelines Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information, and Final Information Quality Bulletin for Peer Review. While EPA alludes to instances where EPA's own policies regarding scientific integrity have been followed inconsistently, the commenter contends that the EPA fails to provide a single example where this occurred and was problematic. Even if this were the case, the commenter asserts that it is unclear why this proposed regulatory action would be needed to cure such problems. The commenter submits that the solution instead seems simple  -  enforce the current well-established and well-reasoned framework regarding the use of scientific information  -  either through EPA's own internal procedures, or by OMB during review of regulatory actions. (6925) 
Response: The EPA agrees with this comment in part and has clarified the role of the EPA Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental Protection Agency and assessment factors in the final rule. As described in the preamble to the final rule, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information, consistent with the EPA Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental Protection Agency. The EPA will generally continue to use the following, established factors when assessing study quality: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review.
The transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency. The requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
2.1.2.4 Support for Proposal to Address Transparency Problem and to Reduce Costs to the Regulated Industry
Comment: Commenters (0050, 0555, 0556, 0560, 0563, 0567, 1273, 1302, 1307, 1784, 1938, 1939, 1972, 2121, 2421, 2426, 2978, 3981, 4009, 4430, 4534, 4541, 4932, 5169, 5419, 6045, 6102, 6112, 6115, 6124, 6140, 6141, 6150, 6151, 6182, 6356, 6362, 6364, 6365, 6366, 6369, 6376, 6377, 6448, 6855, 6857, 6878, 6906, 6911, 6913, 6918, 6921, 6927, 6936, 6942, 6948, 9243) express support for the proposed rule. Commenters generally assert that the proposed rule will help ensure that only scientifically sound regulatory initiatives are considered for implementation. Some specific comments include:
 If the taxpayers are funding the research, the data and results should be open access and available to all. (1939)
 Transparency is especially important given that courts provide deferential review of agency decisions involving scientific or technical matters. (1972)
 Commenter supports EPA's full consideration of making its regulations consistently more transparent, timely and objective by balancing both costs and benefits. (6182)
 Transparent data and scientific debate are critical elements for developing objective and legally defensible regulations, in contrast to the cloistered and opaque science that has been practiced in the past. The commenter states that, without such transparency, the scientific or legal bases for rules are not understood by all and industry is often forced to into appellate litigation. The commenter states that non-transparency usually slows down, rather than expedites, proposed regulations from becoming law. (6182)
 Commenter applauds EPA's willingness to interact with stakeholders while developing this critical regulation. The commenter understands that implementation and application of this regulation is not clear cut and commends EPA on proposing such a challenging regulatory program. The commenter states that they are optimistic that through continued involvement with the public and stakeholder community, EPA will be successful in implementing procedures to ensure sound, science-based decisions in a manner that is verifiable and transparent. (6906)
 Scientific data used by the EPA to make regulatory decisions must be fully available to public analysis and scrutiny so that EPA decision making can be fair, democratic, and responsive to those people who use and rely on the multiple use of federal lands. The commenter states that scientific data veiled in secrecy will lead to politicized decisions tailored to a particular agenda and not sound environmental policy. (6878)
 Commenter supports the proposed rule's underlying values which originate from EPA's statutory authority and principles of good government, including the following:
 The NRC concluded that transparent and effective public participation is essential for EPA to issue valid rules that can be effectively implemented. 
 EPA recognizes several reasons why public participation in government actions is important, regardless of statutory basis:
 The dialogue can result in deeper and more practical insights into the issues than if the interested parties acted individually.
 Those affected are far more likely to understand and accept decisions when their concerns have been acknowledged and addressed.
 Citizen participation in government programs is a democratic ideal.
 Pursuant to Executive Order 13563, EPA must base its regulatory decisions on the "best available science," and this goal can be significantly aided by informed public comments. 
 EPA's Public Involvement Policy recognizes the importance of public participation to the Agency's core mission of protecting human health and the environment. (6918)
 The Transparency proposal is a vital and very important policy for the EPA that will have beneficial effects in that it will identify the nature and magnitude of the EPA research misconduct that has been intentionally and viciously put forward as good science for purely partisan ideological purposes, not the pursuit of science that allows the EPA to identify real risks and mitigate the effects of those risks. (4009)
 Commenters (6112, 6115) assert that poor rulemaking, particularly when it is not based on sound science, can have disastrous impacts on the aggregates industry. For example, commenter (6115) notes that the "Waters of the United States" (WOTUS) Rule that was proposed in 2015 lacked sufficient science, research methods, and data to support the stated purpose and revisions. In fact, the commenter states that the proposed revisions, as introduced, would have dramatically impacted aggregate operations in the nation with an increase in costs associated with expanding their operations, with the potential to be barred from mining future reserves that will be needed as our economy grows and our population continues to increase. The commenter states that, while their company prides itself as being environmentally responsible, the broadened scope of this WOTUS Rule would directly impact their operations, with little environmental benefit. In turn, according to the commenter, these proposed changes to the "Waters of the US" will increase the costs of public works projects across our nation. 
 Commenter supports adoption of the proposed rulemaking because many "significant" and "lesser-significant" regulations have imposed (and will continue to impose) costly requirements and choices on the public but have lacked transparency and replicability with regard to their scientific/technical foundation. According to the commenter, not only has data for these decisions been withheld from public review, but the EPA is conspicuous among federal agencies for its reliance on proprietary data and models. The commenter adds that, not only are these agency practices inconsistent with EPA's own Scientific Integrity Guidance, but they also are inconsistent with research and publishing guidelines adopted by hundreds of peer-reviewed scientific journals and universities. The commenter states that their (NEDA/CAP's) members support peer-reviewed research that invites scrutiny from researchers worldwide with respect to the foundation, causality, and conclusions of that research. Most important, the commenter notes that their members support public access to the scientific and technical underpinnings of EPA regulation. (6356)
      As a general matter, the commenter (6356) suggests that the lack of regulatory transparency on the scientific foundation of EPA regulations has led to fevered political debate on EPA regulations that has impugned their integrity and acceptability by politicians, the regulated community, and the public. As a matter of course, the commenter expresses that the EPA has come to rely improperly on judicial decisions such as Chevron USA v. Natural Resources Defense Council, 467 U.S. 837 (1984), which vest the agency with broad discretion in interpreting ambiguous statutory authority and should be distinguished from its obligation to use the best available science in its decision-making. As a consequence, the commenter notes that stakeholders in regulatory development have been asked to trust EPA's conclusions on important health-based initiatives and not given the opportunity to review data, models, and interpolations of dose-response relationships on which they are based. The commenter asserts that, as EPA's regulations become more and more costly in attempts to achieve incrementally smaller reductions in pollution, it is critically important for the agency to take steps to avoid criticism that it regulates in secret.
      In short, the commenter (6356) states that the proposed rule generally tracks components of the scientific method in use by scientists and researchers since the seventeenth century. The commenter submits that it is important that EPA codify these principles to ensure that the data and models underlying scientific studies that are pivotal to regulatory actions are made be available to the public.
Response: The EPA agrees with commenters on the importance of data transparency to scientific scrutiny and debate in creating sound environmental policy. Because of the potential impact of the EPA's significant regulatory actions and influential scientific information on American lives and livelihoods, the American people deserve environmental decisions and policies that are based on the best information. Only through continuous improvement to its procedures, especially those focused on transparency, can the EPA truly demonstrate that commitment. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent subject matter experts to validate pivotal science, and as data are better understood through independent reanalysis the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information. 
Comment: Commenter (0556) suggests that the EPA rule is likely to, according to them, finally establish specific standards for openness. Moreover, the commenter contends that these standards will set a potential precedent for other Federal Agencies, possibly even other Governments, or even for scientific journals.
Commenter (6936) believes that the proposed rule is a major advancement in terms of EPA's compliance with the expectations before regulatory agencies. The commenter states that, by not disclosing relevant data and methodology behind studies used to justify EPA rulemakings, the reviewing public is often left only with the conclusions or findings of the given studies. The commenter provides that a recent Congressional Research Service report regarding judicial review of agency actions notes the following: 
      [C]ourts require agencies to provide the `essential facts upon which the administrative decision was based' and explain what justifies their determinations with actual evidence beyond a `conclusory statement.' An agency's failure to provide an adequate explanation for its decision will typically result in remand or invalidation of its decision.
Commenter (6936) suggests that given the importance of independent verification and replicability of results in terms of confirming scientific conclusions, it is their view that EPA reduces the risk of its future rules being invalidated by the courts by adopting the proposed transparency standards.
Response: The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. The EPA's continued progress toward maximizing transparency is vital to building and maintaining trust with the public and credibility in the Agency's decisions. Although other federal agencies may adopt similar standards, the EPA would like to clarify that this rule only applies to EPA practices.
Comment: Commenter (6140) attached a paper by Dr. Logomasini and comments offered by Marlo Lewis, Jr. The commenter states that the paper and comments provide details that explain their support for the Transparency Rule. That paper and comments are summarized below:
   A summary of the paper:
 The EPA rule on transparency should be viewed as another tool in a larger effort to promote transparency and reproducibility within the scientific community, most of which is happening outside of government. 
 The EPA's transparency rule will help bolster private pro-transparency efforts and should be the beginning of a government-wide effort. 
 Transparency guidelines could also be applied to government research and government-funded research within all agencies, from the EPA to the National Institutes of Health (NIH), the Food and Drug Administration (FDA), and the Consumer Product Safety Commission.
 The rule contains provisions to protect privacy and provide exceptions that allow regulators to consider quality research that is not fully transparent because of privacy concerns. 
 Improving transparency is an achievable goal. 
 Transparency ensures that all aspects of scientific methods and results are available for critique, compliment, or reuse. 
 Researchers often avoid transparency out of fear that their novel findings may eventually be proven wrong when someone tries to replicate their work.
 Many positive associations will occur by mere chance or unintentional biases. 
 The EPA's proposed rule follows principles laid out in 2017 by the bipartisan Commission on Evidence-Based Policymaking --  humility, transparency, privacy, capacity, and rigor -- and moves us toward providing greater access to scientific data while protecting individual privacy. The commission report addresses privacy issues, finding that transparency of data can be achieved without compromising privacy concerns:
 EPA has plenty of funds within its $8 billion budget to pay for transparency. 
 Transparency might reduce costs for businesses and consumers, who ultimately pay the high costs of misguided regulations.
   A summary of the comments:
 Independent validation is difficult if the study's authors refuse to share their data. That is a longstanding problem in air pollution epidemiology.
 EPA is not proposing to require researchers to divulge data in past studies used to support previous regulatory decisions. 
 The rule includes a provision allowing EPA to exempt significant regulatory decisions on a case-by-case basis by the Administrator
 Nothing in the proposal states or implies that EPA will not consider studies or comments referencing studies unless the data are fully transparent or the results are independently validated. 
 EPA is within its authority to determine that transparent and reproducible science is of higher quality than opaque and irreproducible science. 
 The agency is within its authority to accord greater weight to higher-quality science than to lower-quality science.
 If a study is pivotal to significant regulatory action, preference should be given to studies that can be independently validated. Independent validation is not possible unless the public has access to the underlying data and models.
Response: The EPA agrees that this rulemaking is designed to build upon other important transparency initiatives occurring within the federal government and the scientific community. The incremental progress made by this rule provides an important step towards prioritizing transparency in particularly impactful rules and assessments and will inform future statute-specific rulemakings. Although other federal agencies may adopt similar standards, the EPA would like to clarify that this rule only applies to EPA practices.
Comment: Other comments providing specific support and reasons for addressing transparency include:
 Commenters (0555, 0844, 6150) suggest that greater transparency in the studies, models, assumptions, and risk assessment policy choices used in regulatory decisions could encourage more open, constructive debate on those choices.
 Commenters (1974, 4430, 6102, 6172) support peer review and transparency. 
 Commenters (1784, 3971, 4009, 4430, 4541, 5185, 6161, 6176, 6189, 6365, 6376) support the proposed rule and assert that some past, flawed EPA air quality research has been used as the basis of various regulatory actions.
 Commenter (4541) states that, given the agency's own admission that this failure is widespread within the agency due to a culture of disrespect for the scientific method and independent peer review, a new set of rules is desperately needed. Commenter (6161) states that EPA ignores the absence of transparency and findings of irreproducibility when it suits the Agency's desire to regulate. Commenter (6176) asserts that EPA is not immune to seeking preferred policy outcomes and using questionable science to achieve those outcomes; transparency helps to minimize these problems. 
 Commenters (4931, 6102, 6108, 6126, 6140, 6141, 6150, 6448, 6857, 6878) support the statement in the proposal that "the best available science must serve as the foundation of EPA's regulatory actions." 
 Commenter (3971) states that the proposed regulation is responsive to concerns expressed for the past three or four decades by many within the toxicology community. The commenter states that there are numerous examples of EPA risk assessments that would have been much more readily accepted by the regulatory community had they been more transparent. 
 Commenter (5185) states that, for many years EPA has relied upon non-public data to justify its aggressive regulatory agenda. The commenter states that the most egregious but certainly not the only example of this involves two controversial studies undertaken in the 1980s that suggest a linkage between PM2.5 and health outcomes. The commenter states that the data associated with these decades-old studies has never been made public, but EPA has nonetheless used them to monetize regulatory benefit claims that dominate the communications and regulatory marketing associated with nearly all of its major rules. 
 Commenter (6189) attaches articles and letters regarding PM2.5 and states that these are a highly relevant example of the importance of independent access to underlying data and transparency in regulatory science.
Response: The EPA agrees that scientific openness benefits both scientific and government processes. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty in the assessment. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Comment: Commenter (6365) states that the Lanphear case study demonstrates the need for EPA to finalize its regulatory transparency rule in the following ways: 
 First, it shows that the peer review process does not always identify errors in studies that would constitute "pivotal regulatory science" within the meaning of EPA's proposed rule. 
 Second, the absence of this data prevented EPA and stakeholders from discovering errors in "pivotal regulatory science" at the time EPA relied on it. 
 Third, it shows that absent a regulatory transparency rule, it could take stakeholders years of effort and untold expenditures to obtain data that may demonstrate errors in "pivotal regulatory science." 
 Fourth, its shows that making this data available will facilitate the validation and replication of pivotal regulatory science by reducing barriers to obtaining these materials, furthering (not hindering) public health science.
Response: Although this rulemaking is not intended to remedy a particular instance in which the EPA relied on a poor-quality study or a study for which the conclusions could not be independently validated, it does provide important incremental progress toward the Agency's goal of greater transparency. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. In addition, as data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Comment: Commenter (6376) states that, under President Obama's Administration, the EPA pursued an aggressive regulatory agenda based on single sided scientific studies and modeling. The commenter states that, to provide a real-life example of this misapplication of scientific studies one can look at the Boiler Maximum Available Control Technology (MACT) rulemaking. The commenter states that, when setting the emission standards for the best available technology for boilers the Agency "cherry-picked" data from various boilers and used that data to inform the standard for all boilers. The commenter states that the problem is the final standard was achievable for a hypothetical boiler rather than real life boilers. The commenter states that, not only was the result an unattainable standard, but the difference in the cost estimate from EPA and the regulated community was drastic. The commenter states that EPA estimated it would cost $3 billion to come into compliance whereas industry estimates put the cost at $20 billion. 
Response: Although this rulemaking is not intended to remedy a particular issue, the EPA agrees that the potential impact of the EPA's significant regulatory actions and influential scientific information on American lives and livelihoods is great; therefore, the American people deserve environmental decisions and policies that are based on the best information. Only through continuous improvement to its procedures, especially those focused on transparency, can the EPA truly demonstrate that commitment. 
Comment: Commenters (0563, 1486, 1939, 2142, 3134, 6350, 6374, 6878) support the proposal that the science used to inform the making of regulations be publicly available for independent verification. Commenter (6878) asserts that, as an agency of a democratically elected government, the EPA should ensure that the American public has full access to its decision-making processes. Commenters (1486, 2142, 6350) assert that evidence is the bedrock of the Scientific Method, and it must be observable and measurable and available to all who wish to verify, validate, or disprove the hypothesis which the evidence is claimed to support. Commenter (6350) states that scientific conclusions said to be made based on evidence that is neither public, repeatable, nor reproducible, are neither scientific, nor conclusive. 
Response: The EPA agrees that scientific openness benefits both scientific and government processes. Because of the potential impact of the EPA's significant regulatory actions and influential scientific information on American lives and livelihoods, the American people deserve environmental decisions and policies that are based on the best information. Only through continuous improvement to its procedures, especially those focused on transparency, can the EPA truly demonstrate that commitment.
Comment: Commenters (1939, 0062, 4430, 6045, 6140) support the proposal and agree that, regarding medical information, there are methods available to protect the confidentiality of individuals whose data would be used in environmental studies. 
Commenter (6045) states that, while the EPA might agree to redact these data before public posting, they could be given to anyone who signed a confidentiality agreement designed by the agency. The commenter states that all NIH clinical studies can only be approved for funding if the investigators "surrender personally identifiable information  --  as it pertains to statistical reevaluation of the data and corroboration of the methods employed to study such data, which in turn might lead to public regulatory policy." The commenter states that EPA clinical studies should be held to the same high standards as all NIH clinical studies.
Commenter (4430) states that EPA's proposed rule follows principles laid out in 2017 by the bipartisan Commission on Evidence-Based Policymaking  --  humility, transparency, privacy, capacity and rigor  --  and moves us toward providing greater access to scientific data while protecting individual privacy.
Commenter (6140) states that, while some claim that the rule will prevent EPA from using research that relies on confidential data collected from study participants, the rule affords the EPA Administrator considerable leeway to permit regulators to use research in cases where privacy or other concerns limit public availability. 
Response: The EPA agrees that mechanisms exist to share data while maintaining protections for sensitive information. As discussed in the final rule, underlying data or models are considered available if they can be accessed through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Further, the EPA would like to clarify that the rule does not prevent the Agency from using research with confidential data and does not rely on an Administrator's exemption to be able to consider such research. As explained in the preamble for the final rule, the EPA did not finalize the 2018 proposal or the 2020 SNPRM proposed option that would have categorically required that the underlying data be available for independent validation in order for studies to be considered pivotal science. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
Comment: Commenters (0019, 4430, 6102, 6150, 6161, 6910, 6938) support the proposal that scientific studies used by EPA be available in a format that allows other interested parties to replicate or reproduce the results. Commenter (0019) asserts that, if this is not done, EPA could make decisions on research that may be flawed. 
Commenter (6102) states that increased transparency and access to underlying decision-making information encourages replication and validation, key steps in the scientific process. Commenter (6150) states that replication and validation are key tenets to scientific inquiry and promotion of data sharing will allow testing of new ideas on existing data sets. Commenter (6161) states that the reason transparency is paramount in the scientific enterprise is that clear exposition of materials and methods is necessary to allow attempts to replicate studies. Commenter (6910) states that, in order for science to be accepted and utilized in policy making and consumer choices, science has to be reputable, repeatable, and factual. Commenter (6938) states that too often in the past, EPA has relied upon public health studies with non-public data to justify regulatory policy and without access to the non-public data, it is impossible to replicate and validate the conclusions in those studies. 
Response: The EPA agrees with commenters on the importance of data transparency to scientific scrutiny and debate in creating sound environmental policy. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty in the assessment. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Comment: Commenters (0039, 1974, 2114, 4537, 4882, 6112, 6115, 6140, 6162, 6172) support the proposal that regulatory decisions be based on reproducible and transparent research. 
Commenters (2114, 4882, 6140) assert that the scientific method requires research to be transparent and reproducible. Commenter (0039) believes that these reforms will strengthen American science, by prompting researchers to incorporate and make routine in their practices the highest standards of reproducibility.
Commenter (4537) states that larger concerns about reproducibility suggest that this measure should be applied more generally within EPA and across the Federal Government.
Commenters (6112, 6115) state that because rulemaking and other regulatory actions by EPA and other agencies have enormous impacts, they should be able to clearly understand the rationale behind EPA actions. 
Commenter (6162) states that, while EPA has not consistently followed the current EPA Scientific Integrity Guidance that encourages use of non-proprietary data and models, the proposed rule requires increased access for outside researchers to independently reproduce the underlying analyses on which environmental rulemakings are based. 
Response: The EPA agrees with commenters on the importance of data transparency to scientific scrutiny and debate in creating sound environmental policy. Because of the potential impact of the EPA's significant regulatory actions and influential scientific information on American lives and livelihoods, the American people deserve environmental decisions and policies that are based on the best information. Only through continuous improvement to its procedures, especially those focused on transparency, can the EPA truly demonstrate that commitment. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty in the assessment. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Implementation of this rule will also increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information. Although other federal agencies may adopt similar standards, the EPA would like to clarify that this rule only applies to EPA practices.
Comment: Commenters (4541, 6150, 6878) support the proposal and assert it is consistent with statutes and/or Executive Orders. Commenter (6150) states that EPA's proposal to enhance transparency and to facilitate the use of scientific data that are publicly available in a manner sufficient for independent validation is in keeping with its obligations under federal statutes. Commenter (6878) states that the proposed rule will make EPA decision making more consistent with the requirements in the Administrative Procedures Act (APA) that ensure public participation in the rulemaking process. 
Commenter (4541) states that governing statutes and Executive Orders authorize the proposed Rule and, indeed, have long required it. The commenter states that, in not following the proposed rule in the past, EPA has been flouting governing statutes and Executive Orders, particularly in matters of toxicology and epidemiology, and abusing its authority by ignoring and discarding valid objections to EPA adopted science and the derivative policy and regulatory actions.
Response: The Agency agrees that the EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. The requirements in this rule build upon the open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Implementation of this rule will result in significant regulatory actions and influential scientific information that are based on the highest quality, most relevant studies and ensure adequate human health and environmental protections, while maximizing transparency and maintaining consistency with the statutes the EPA administers.
Comment: Commenters (0563, 1784, 1939, 0844, 3971, 6102, 6182) support the proposal because it could reduce costs.
Commenters (0563, 0844, 3971, 6102) support the proposal because it could reduce costs of regulations that might be based on data that are not publicly available. 
Commenter (1784) states that, with the cumulative economic impact of federal regulations reaching nearly $2 trillion per year, research reform is essential.
Commenter (3971) states that, if EPA can gain more support for its proposals, which this regulation will do, the number of legal challenges would be lessened and billions of dollars in legal fees and awards would be avoided. 
Response: The EPA would like to clarify that this is a rule of internal procedure promulgated under the EPA's housekeeping authority. Because this rule does not require the EPA or other parties to disclose or host data, this rule does not impose costs on the EPA or any other party to make such data available. As more fully discussed in the preamble to the final rule, the EPA recognizes there will be some incremental review and administrative costs to the Agency but anticipates these will be small. However, the EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety.
2.1.2.5 Proposed Rule Will Not Deliver Improvements in EPA's Current Regulatory Processes
Comment: Commenters (2119, 4589, 4589, 4590, 5170, 6111, 6157, 6160, 6168, 6872, 6924) assert that the proposed rule will not deliver improvements in EPA's current regulatory processes. Commenters (5176, 5179, 6158) assert that the goal of increasing transparency in science and the public's faith in the EPA is undermined by the proposal. Comments include the following:
 EPA calls for transparency, many features of the proposed rule themselves are not transparent. The commenter contends that ambiguity and lack of detail lend themselves to decision-making that is arbitrary and open to influence, i.e. the opposite of transparent or based on best available science. The commenter notes that the rule never addresses the wide range of measures that would be necessary to achieve true transparency for either scientists wishing to replicate results or for a broader public wishing to better understand and participate in the regulatory process. The rule proposes just a single measure -- making data and models available -- and proposes a false equivalence between "transparency" and (poorly defined) independent validation. The commenter adds that the proposed measures on data and model availability and use facilitate not transparency but the culture of science denialism: dismissing important evidence and dredging data to endlessly produce false uncertainty and delay scientific advances regarding the harmful effects of chemical exposures. The commenter notes that this false uncertainty is used to justify failure to protect environmental and public health. (6168)
 The proposed rule will do little to nothing to improve the transparency. (6872)
 The proposed rule would not improve data sharing, replicability, or transparency in decision-making. (6924)
 The approach advocated in the proposed rule is inconsistent with professional best practices in their respective disciplines for conducting, reviewing, and confirming the results/findings of studies, especially those based on confidential personal health data of study participants. (6111)
 The proposed rule runs counter to the recommendations of leading scientific organizations like the NAS. (6158)
 Commenter urges EPA to abandon its plans to advance this misguided proposal that would undermine sound scientific norms in the rulemaking process. (6157)
 The emphasis of the proposed rule on making data sets publicly available fails to demonstrate any improvement on the existing process and adds nothing to the evaluation of the scientific merit of any piece of evidence. (6160)
 Because of the peer-review process, EPA already has a firm scientific basis for using studies to protect public health. The commenter adds that making the raw data available provides no improvement to the robustness of the studies. (5170)
Response: In response to public comments, the EPA clarified and modified many aspects of the proposed rule in the 2020 supplemental notice of proposed rulemaking and the final rule. As described more fully in the preamble to the final rule, the EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. The provisions of this rule prioritize the transparency of dose-response data in pivotal science, while still ensuring that all quality, relevant literature is considered in the development process.
2.1.3 Proposed Rule is Biased
 2.1.3.1 Takeover of EPA Leadership
Comment: Commenter (0844) cites The Washington Post quote of the Union of Concerned Scientist's (UCS's) Center for Science and Democracy Director Andrew Rosenberg, who said, "[f]irst, they came after the agency's independent science advisers, and now, they're going after the science itself...what is transparent is the unabashed takeover of EPA leadership by individuals who have demonstrated disinterest in helping communities combat pollution by using the best available science."
Response: The EPA disagrees with this commenter. As discussed more fully in the preamble of the final rule, the federal government and the scientific community have been pursuing greater transparency and data sharing for years. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty in an assessment. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. The EPA is very interested in public engagement and, as data are better understood as a result of the increased opportunity for independent experts to validate pivotal science afforded by this rule, the EPA will facilitate more effective comment and engagement with the public during development of future significant regulatory actions and influential scientific information.
2.1.3.2 Implementation Bias
Comment: Commenters (1006, 2787, 6877) note that proposal could lead to implementation bias. Comments include:
 The Proposal seems like a way for the EPA to have say over what gets implemented by what gets approved with any easy excuse of withholding of data. (1006)
 Contrary to its name, the proposed rule would implement an opaque process allowing EPA to selectively suppress scientific evidence without accountability and in the process undermine bedrock environmental laws. (2787) 
 Discriminating what science to use for drafting regulation risks introducing potential bias in the rulemaking process. The commenter suggests that bias is likely to be introduced through the process of identifying what science can and cannot be used because there is not a bright line for what studies this proposed rule would include or exclude. Specifically, the commenter points out that the requirement to make a decision for what constitutes "high quality studies" is problematic because the term is not defined in the proposed rule nor are operational criteria for its definition specified. The commenter provides that recent work in systematic reviewing of research in many areas of science has demonstrated considerable variation in judgments about "quality" by competent scientists unless the criteria for quality are specified precisely. The commenter suggests that this variation in judgement risks arbitrary and capricious decisions as to what evidence to include or exclude. The commenter notes that several good resources for avoiding bias and gaming of the regulatory process in specifying the standard for "high quality" are available. The commenter concludes that the exclusion of otherwise valid and reliable research from consideration would also often result in regulatory analyses that suffer from biased selection and inadequate statistical power. (6877)
Response: The EPA considered these and other similar comments when developing the final rule. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. Where there is convincing and well-substantiated evidence to support a relationship between exposure and effect, the EPA will identify a subset of those studies -- based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- for use in characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both will be identified as pivotal science. Once the EPA has identified pivotal science -- in either final significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of personally identifiable information (PII), confidential business information (CBI), or propriety information, available through restricted access in a manner sufficient for independent validation. Including this review of data availability for pivotal science is critical to the EPA's progress toward increased transparency and providing increased opportunity for scientific reanalysis and scrutiny by independent third parties. This approach will result in final significant regulatory actions and influential scientific information that are based on the highest quality, most relevant studies and ensure adequate human health and environmental protections, while maximizing transparency and maintaining consistency with the statutes the EPA administers. The EPA does not believe that this final approach will lead to bias.
The EPA intends for the final rule to be implemented in a transparent manner. The final rule requires the EPA to document the rationale for exemptions granted by the Administrator to the requirements included in the final rule in the final significant regulatory action or final influential scientific information. Additionally, the EPA expects to identify pivotal science in proposed significant regulatory actions or external review drafts of influential scientific information, which will allow the public, if they so choose, to independently validate the pivotal science and provide comment or other input to the EPA. 
2.1.3.3 Injects Politics into Science
Comment: Commenters (0040, 0055, 1301, 1395, 2427, 2492, 2907, 4881, 6122, 6152, 6158, 6895, 6903, 6909, 6914, 8274, 9225, 9227) suggest that the proposal is biased in ways that will facilitate political interference in science-based decision-making. Comments provided include:
 Asserts that billion-dollar industries make so much profit that they can spend unlimited amounts of money in lobbying to erode environmental protections that are in the civic best interest. The commenter states the proposal throws away merit-based, evidence-based policy decisions and opens up government policy for hire. (0040)
 Commenter (1301) asserts that the proposal is an attempt to use the concept of transparency to support a policy that provides the ability to reject almost any scientific study of human health effects that EPA would like to jettison to meet Administration goals of subverting environmental protection. 
 Asserts the proposal is an abuse of the concept of scientific transparency in the service of political objectives that support increasing pollution limits to serve business interests but will harm people and result in higher long-term public health costs. (2492)
 The proposed regulation would substitute the judgement of political appointees for scientists and technologists, many from Americans leading universities and research organizations. (2907)
 Despite its title, the commenter believes that the EPA's proposal is not an effort strengthen transparency, but to suppress information that exposes corporations of wrongdoing and caters to special interests. (6914)
 Believes that the proposed rule undermines the essential role of science in the regulatory process and would open the door to partisan manipulation of EPA risk assessment science. (4881)
 If finalized, the rule would allow political appointees to reject the best-available science for virtually any reason and delay taking critical, life-saving actions in order to benefit special interests and corporate polluters. (6122)
 The proposed rule would ignore scientific best practice and inject politics into science. (6152)
 While academic science strives to be as disinterested as possible, parties with an interest in regulation could produce a self-serving but flawed analysis of raw data to sow doubt. The commenter contends that the EPA is clearly attempting to divorce itself from the use of real science in order to promote a political agenda. (6895)
 The proposed rule's attempt to exclude perfectly reliable scientific studies from EPA's consideration is a thinly veiled attempt to allow the EPA to ignore sound science at its discretion if that science does not conform with the current Administration's political agenda. (6909)
 Commenter expresses concern that shifting the decision making process for which scientific studies are used or exempted in EPA policy from experts in the field of the respective research to a politically appointed position creates the possibility of a disastrous situation where political ideals replace scientific fact and expert opinion. (8274)
 Rather than strengthen science, the proposal grants the Administrator authority to censor science--that by any scientific definition is the best--simply because it conflicts with this Administration's political goals. (9227)
Response: The EPA disagrees that the rule will facilitate political interference in regulatory decision-making. Transparency assumes not political ideology. The intent of the rule is to build upon existing federal government efforts and provide incremental progress toward the Agency's overarching goal of greater transparency. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
The EPA agrees with commenters that it is important to consider the totality of relevant, quality research. Therefore, the EPA did not finalize the 2018 proposal or the 2020 SNPRM proposed option that would have categorically required that the underlying data be available for independent validation in order for studies to be considered pivotal science. However, given that transparency is an important aspect of EPA's regulatory actions and assessments, it should be an important consideration in how the Agency considers pivotal science. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will continue to evaluate and consider all quality, relevant studies when promulgating final significant regulatory actions and finalizing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
The EPA does not agree with commenters regarding the role of the Administrator in determining whether to grant an exemption under 40 CFR 30.9 and finds that the Administrator is the appropriate decision maker in this context. To ensure that the Administrator's decision is appropriately transparent, the EPA has included a provision in 40 CFR 30.9 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the final significant regulatory action or final influential scientific information.
2.1.3.4 Industry Bias
Comment: Commenters (1905, 2171, 2787, 4840, 4884, 4933, 6121, 6123, 6125, 6195, 6879, 6892, 9227) suggest that the proposed rule provisions introduce a bias against peer-reviewed research in the use of science in regulatory decision making that favors industry. Comments provided include:
 The EPA is trying to discredit scientific findings that threaten powerful corporate interests. (1905)
 Commenter views the Proposal, and its requirement of making publicly available data for "independent validation", simply as a means of providing industries concerned with various studies an opportunity to reanalyze data to reshape or recast conclusions drawn by researchers and subsequent peer reviews, and to allow EPA to censor important studies without justification. (2171)
 The substance of the rule is concerning. The commenter notes that it appears to be targeted at excluding important public health studies while privileging industry-sponsored research and urges the EPA to withdraw the rule. (2787)
 There is every reason to believe this is the plan of the current leadership of the EPA and is a recipe for privileging mercenary studies by industry commissioned to influence regulatory policy, because they will be structured and released in a way acceptable under this regulation. (4840)
 The real purpose of these rules is to reverse regulations on industries who have been harmful to public health. The commenter suggests that science speaks the truth and the EPA should hear from all scientific studies, not just the ones the industry wants you to listen to, adding that "[s]cience is the cross before the corporate devil". (4933)
 The commenter states that business is most often about annual profit and loss while treating a clean environment as a free raw material. The commenter notes that the proposed rule insertion to Title 30 is meant to frustrate and debilitate the legislated missions of the EPA. Rather than the avowed transparent science, the commenter states that it will enable cherry picked industry data and flush valid scientific research. (4884)
 The EPA's proposed rule is not about transparency or strengthening science. Instead, the commenter suggests that it is a wolf of pro-industry bias hiding in the sheep's clothing of transparency in science and should be withdrawn. (5022)
 Deconstructing Regulatory Science," by Professors Rena Steinzor and Wendy Wagner (June 19, 2018) (cited by 6892)
 The only "transparency" in this proposed regulation is its clear intent to censor legitimate and sound science that may be inconvenient to powerful industries that are regulated by EPA. The commenter requests withdrawal of the rule. (6879)
 The only outcome of this rule will be to put industry profits before the mission of the EPA, which is to protect human health and the environment by reducing risks based on the best available science and urges EPA to withdraw the rule. (6195)
Commenter (4881) notes that, in the light of other recent EPA directives, the proposed rule can be fairly judged to favor the industry perspective at the expense of public health protection, in violation of the Court-mandated balance. The commenter provides that the EPA is currently operating under a directive that eliminates EPA-funded scientists from consideration for membership on EPA scientific advisory boards, such as the CASAC mandated by the Clean Air Act (CAA). Even if they are subject experts on research relevant to regulatory considerations, the commenter notes that scientists with some EPA funding are judged too conflicted to provide advice, while industry representatives are judged to have no such automatic conflicts. The commenter asks:
 Is the advice of such now unbalanced scientific advisory boards to be critical in judging which published research is acceptable for EPA to include in formulating standards? 
 Are intellectual property claims of proprietary data by industry to be handled consistently with claims of confidentiality by independent researchers? 
 Will internal industry memos establishing research programs and judging whether results are consistent with industry goals be made publicly accessible, as well as raw data, to strengthen transparency and enable independent judgments on the validity of that research?
Response: The EPA disagrees that the rule will introduce bias in regulatory decision-making. The intent of the rule is to build upon existing federal government efforts and provide incremental progress toward the Agency's overarching goal of greater transparency. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information. The EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public.
The EPA also disagrees that the rule is intended to benefit industry. As stated in 40 CFR 30.3, "The provisions of this part apply to science that serves as the basis for informing a final significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science." The rule requirements also direct the EPA to evaluate the availability of the underlying dose-response data for any subject matter expert, regardless of their affiliation. Further, the final rule dictates that the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or propriety information, available through restricted access in a manner sufficient for independent validation. Restricted access in the context of this rule means that data must be available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. CBI data are not given different or preferential treatment.
The EPA agrees with commenters that it is important to consider the totality of relevant, quality research. Therefore, the EPA did not finalize the 2018 proposal or the 2020 SNPRM proposed option that would have categorically required that the underlying data be available for independent validation in order for studies to be considered pivotal science. However, given that transparency is a very important aspect of EPA's regulatory actions and assessments, it should be an important consideration in how the Agency considers pivotal science. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will continue to evaluate and consider all quality, relevant studies when promulgating final significant regulatory actions and finalizing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
The EPA would like to clarify that this final rule will be implemented prospectively and rules that have already been finalized will not be subject to this rule's requirements. The EPA would also like to clarify that this rule is not related to membership processes for the EPA's scientific advisory boards.
Comment: Commenter (4877) states that their organization and its members support open access to research outputs to democratize access to information, accelerate scientific discovery, and stimulate innovation. However, they express concern about the rule's proposed selective use of "open" as an enabling strategy in the EPA's policy-making process.
Response: In response to public comments, the EPA clarified many aspects of the proposed rule in the 2020 supplemental notice of proposed rulemaking and the final rule. The requirements in the final rule provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
Comment: Commenters (2909, 4596, 4840, 5022, 6046, 6109, 6121, 6123, 6133, 6144, 6157, 6336, 6446, 6895, 6920, 8281-PH34, 9227) point out the inequity/hypocrisy of the proposed rule subjecting health study privacy data to additional scrutiny while allowing continued confidentiality of equally critical and opaque business information. Comments provided include:
 Questions the "double standard" of allowing industry data to be confidential, which favors polluters over those impacted by their pollution. The commenter adds that the only thing transparent about the proposed rule is the outcome it would have compromising science to the detriment of human health and our environment for the benefit of big polluters. (2909)
 EPA's claims of increasing transparency are disingenuous because the Proposal may allow EPA to consider significant amounts of information submitted by industry to the agency, even though the information will be protected as CBI and shielded from the public. According to the commenter, because industry data are associated with CBI claims more so than public health researchers' work, the Proposal reveals its industry bias. (4596)
 The commenter notes that the proposed rule requirements imposed on independent researchers are onerous and burdensome. (4840)
 Proposal would have an effect opposite to its claimed purpose; it would suppress important and relevant science conducted in large part by the best minds in academia and government, thereby unduly restricting the evidence available to EPA and potentially favoring data developed by industry. (5022)
 The Proposal is biased against non-pesticide contaminants while ignoring the very large transparency issue with pesticide evaluations, could result in increased uncertainty in standards and health values, and ignores current measures that try to address scientific credibility of studies. (6046)
 CBI may be exempted from the transparency rule, but not necessarily confidential health information. The commenter provides that the EPA states that it seeks comment on "methodologies and technologies designed to provide protected access to identifiable and sensitive data, such as individual health data" which they believe illustrates EPA's commitment to making health data public. The commenter provides that the EPA sets a different standard for protecting CBI, suggesting CBI may be allowed to remain nonpublic: "EPA seeks comment on how to balance appropriate protections for copyrighted or CBI, including where protected by law ..." The commenter inquires what EPA means by the phrase "balance appropriate protections" for CBI and why is the same phrase not used for protecting private health data? The commenter suggests that the EPA does not appear to place private health data on the same footing as CBI. This, according to the commenter, suggests EPA will be more likely to allow industry to submit nonpublic data than it will health researchers. The commenter adds that it also suggests that if EPA applies the proposed rule retrospectively, the agency will still consider research that contains nonpublic CBI while discounting research that contains nonpublic private health data. (6109)
 "[P]ublic scrutiny" would only serve industry's interest either through eliminating the use of studies that contradict their claims of chemical safety or through attempting to discredit studies and researchers by "reinterpreting" the data in their favor. (6121)
 EPA indicates that the proposed rule is intended to strengthen transparency of EPA regulatory science, however the commenter finds this to be a duplicitous claim. The commenter states that the proposed rule would favor industry data protected as CBI over public peer-reviewed research. (6123)
 Under the Proposal, EPA would not be able to rely on the best available science for its ISAs of air pollution which inform the NAAQS-setting process, while industry-funded research calling into question the air pollution-health link, would not be subject to similar data release requirements, or even peer-review and independent reevaluation. The commenter notes that this approach is asymmetric and favors selective, opaque, and questionable research methods over the consensus of robust peer-reviewed scientific investigation. The rule is arbitrary in its selective application of data release requirements and disregard for the quantitative complexities of epidemiologic research. (6133)
 If EPA is going to implement any version of this proposed rule, it must not, and cannot include a loophole for industry data while requiring that underlying public health data be made available. (6144)
 The proposed rule would subject health studies alone to additional scrutiny by exposing the privacy of participants and illogically raise the standards by which they can be considered. Meanwhile, the commenter states that the rule would allow for the continued confidentiality of equally critical and often much more opaque business information such as economic cost modeling or industry labeled proprietary data, which is rarely subject to verification at all, let alone rigorous peer review. The commenter contends that this approach would only serve to eliminate the strong and well-founded basis for health and environmental regulations millions of Americans depend on while keeping information deemed confidential by for profit industry interests out of the public eye. The commenter adds that the dividing line drawn by this proposal creates two camps: the privileged, industry data kept out of the public eye and often used to avoid protections like pollution controls for heavily emitting sources such as coal-fired power plants and oil refineries; and the epidemiological data prejudiced by EPA's proposed transparency rule that is used to support protections for the benefit of public health and welfare. (6157)
 Industry research might still be protected as CBI under the proposed rule, while academic or public interest research would not be entitled to rely on intellectual property protections. (6446)
 EPA is looking to establish a rule that would require independent public and private institutions to release confidential data behind environmental health studies while at the same time seeking to exempt private industry from being required to do the same. Given that the confidential data held by independent institutions are also protected by law, the commenter states that the EPA should be seeking the same transparency privilege as that afforded the private sector. (6895)
 Commenter finds it hypocritical that EPA's proposal appears to not want to increase transparency for the thousands of chemicals in the millions of products we buy everyday citing the need for business to keep trade secrets confidential while pushing for unsubstantiated and unnecessary increases in transparency in science that could threaten public health. (6920)
 The EPA has excluded administrative proceedings where EPA and industry regularly rely on nondisclosed information and where EPA action in general, and particularly expeditious action, tends to favor industry. The commenter suggests that by limiting the proposal to "significant regulatory actions," the proposed rule would treat exactly the same study differently depending on whether it supports regulation or non-regulation in a particular context. The commenter adds that the proposed rule will tend to exclude evidence when it supports a health-protective regulation that is costly to industry, but the proposed rule will then allow the use of the exact same evidence when the ultimate agency decision avoids regulation or deregulates industry activities or otherwise has low compliance costs. Thus, according to the commenter, the Proposal is clearly shaped to favor industry interests, not to further transparency. For example, the commenter contends that the EPA's exclusion of the new chemicals program clearly favors industry, allowing industry to conceal information and evade regulation. In addition, the commenter asserts that the EPA cannot rationally impose stringent new disclosure requirements that exclude extensive peer-reviewed, high-quality studies in some contexts while simultaneously authorizing the commercial distribution of new chemicals with almost no disclosure and no peer-review. (9227)
Response: The EPA disagrees that the rule would be biased toward industry. As stated in 40 CFR 30.3, "The provisions of this part apply to science that serves as the basis for informing a final significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science." The rule requirements also direct the EPA to evaluate the availability of the underlying dose-response data for any subject matter expert, regardless of their affiliation. Further, the final rule dictates that the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or propriety information, available through restricted access in a manner sufficient for independent validation. Restricted access in the context of this rule means that data must be available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. CBI data are not given different or preferential treatment.
Based on a consideration of these and other similar comments, the EPA did not finalize the 2018 proposal or the 2020 SNPRM proposed option that would have categorically required that the underlying data be available for independent validation in order for studies to be considered pivotal science. However, given that transparency is an important aspect of EPA's regulatory actions and assessments it should be an important consideration in how the Agency considers pivotal science. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will continue to evaluate and consider all quality, relevant studies when promulgating final significant regulatory actions and finalizing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations. 
According to the final rule, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence of a relationship between exposure and effect, the EPA will identify those studies based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both of an EPA significant regulatory action or influential scientific information will be identified as pivotal science. The EPA shall give greater consideration to pivotal science where the underlying dose-response data are publicly available in a manner sufficient for independent validation. The Agency shall give greater consideration to pivotal science based on dose-response data that include CBI, proprietary information or PII if these data are available through a data sharing mechanism sufficient to protect sensitive data, such as through an agreement with the originating author or institution or restricted access in a manner sufficient for independent validation.
Comment: Commenters (6109, 6144, 6915) note that the EPA statement that where "available and appropriate, EPA will use peer-reviewed information, standardized test methods, consistent data evaluation procedures, and good laboratory practices to ensure transparent, understandable and reproducible scientific assessments" displays a bias toward industry-backed research over research from independent and university scientists. 
Commenter (6109) provides that GLP, a protocol that federal regulators apply to industrial laboratories, is infrequently used outside of industry. According to the commenter, this is because GLP is expensive and does not reflect science quality. The commenter notes that GLP regulations mandate methods for archiving data, record keeping, equipment calibration, etc., but they do not influence research design or methodology. Conversely, the commenter notes that academic researchers (who are frequently federally funded), on the other hand, follow scientific processes that are subject to the rigors of peer review and independent replication. According to the commenter, if EPA is to truly use the "best available science," it must focus on studies that use appropriate protocols and techniques, not merely favored recordkeeping processes.
Commenter (6915) adds that this would favor consideration of industry toxicology studies over equally valid peer reviewed studies from other institutions. Under current EPA risk assessment approaches, the commenter states that all relevant scientific data are considered. In contrast, the commenter asserts that this language indicates the proposed rule is significantly more restrictive than current EPA guidance as far as the types of valid peer-reviewed scientific data that can be considered.
According to the commenter (6915), it is critical to note that the phrase GLP referenced by EPA is not a value descriptor. Rather, the commenter explains that it is a technical term referring to a specific category of study conduct and reporting that is intended for specific regulatory purposes. The commenter provides that GLP/standardized test method studies are typically conducted by industrial or contract laboratories, and test for limited parameters in order to meet specific regulatory requirements, such as for registration of pesticides, drugs, and other products. The commenter adds that these protocols often have not been updated to incorporate recent approaches in toxicology, and they may not look at the most sensitive and relevant toxicological effects of the product being studied. In contrast, the commenter notes that other equally scientifically valid studies, typically conducted in research laboratories in academic, industrial, or government institutions, use specialized approaches to evaluate specific toxicological effects of the chemical under study, and may not follow the standardized protocols specified in regulatory requirements. The commenter asserts that the use of GLP protocols does not necessarily mean that the study is of higher quality, and there is no scientific reason that the data generated under the highly circumscribed regulatory requirements for product registration should receive greater weight than any other valid scientific data. Rather, the commenter suggests that all studies be evaluated on their own merits.
Similarly, commenter (6144) notes that the EPA's proposed rule alludes to EPA's use of GLP and standardized test methods where "available and appropriate." The commenter states that this would bias the literature supporting rules toward industry funded GLP studies, even though GLP practices are more about recordkeeping than upholding a high quality of research design. As an example, the commenter notes that a study reviewing the quality of GLP literature on the chemical bisphenol A (BPA) led the authors to conclude that "simply meeting GLP requirements is insufficient to guarantee scientific reliability and validity." While GLP studies should not be excluded from EPA's consideration, the commenter contends that the highest quality assessment that EPA can engage in would be a systematic review approach that considers all relevant data and establishes criteria for study quality evaluation, as recommended by the NAS.
Response: The EPA considered these and other similar comments when developing the final rule. As described in the preamble to the final rule, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information, consistent with existing Agency practice. The Agency will also continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA disagrees that this rule will favor any stakeholder group. The EPA would also like to clarify that the purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information, and does not prescribe conformity with GLP or other standard practices.
Comment: Commenters (5022, 6881) note that the proposal runs counter to advice provided to EPA by the NRC in a way that favors industry.
Commenter (5022) contends that the pro-industry orientation of the proposal is revealed in its assault on the linearity assumption in the dose/concentration-response function which runs counter to the advice provided to EPA by the NRC, which stated, "[t]he committee recommends that cancer and noncancer responses be assumed to be linear as a default." 
Commenter (6881) notes that the "Science and Decisions" committee was clear that EPA's approach to dose response modeling needed to be changed to reflect growing scientific knowledge and the evolving needs of decision makers. The commenter notes that the proposed rule does not reflect the conclusions from the NRC committee (or other standard mechanisms by which changes in modeling and risk assessment practice are typically promulgated), and instead proposes prescriptive steps that, if taken in aggregate, would delay the risk assessment process and confuse risk managers, ultimately leading to paralysis by analysis.
Response: The EPA disagrees that this final rule favors industry. As stated in 40 CFR 30.3, "The provisions of this part apply to science that serves as the basis for informing a final significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science." The rule requirements also direct the EPA to evaluate the availability of the underlying dose-response data for any subject matter expert, regardless of their affiliation. 
In response to public comments, the EPA clarified and modified many aspects of the proposed rule in the 2020 supplemental notice of proposed rulemaking and final rule, including that the rule requirements are now focused on dose-response data and do not apply to dose-response models.
The EPA also disagrees with commenters that requirements of this rule will result in any meaningful delay to its processes. While this rule does require the Agency to evaluate the availability of the underlying dose-response data for pivotal science, the incremental burden to the Agency to carry out these requirements is expected to be small given (1) the extensive scientific review EPA already conducts regularly and (2) that the requirement is limited to pivotal science (i.e., a small, though highly important, subset of all the studies EPA reviews).
Comment: Commenter (4881) notes that further evidence of an industry bias is reflected in the proposed rule itself by the complaint in footnote of the proposed rule preamble (footnote 3) about the Court decision in the case of Coalition of Battery Recyclers Association v. EPA, in which a central issue was precisely the "growing empirical evidence of non-linearity in the concentration-response function" that EPA now claims is its primary concern. The commenter provides that, in Battery Recyclers, the D.C. Circuit Court supported the EPA's current practice with respect to scientific evidence, rather than that proposed in the new rule. In particular, the commenter notes that the Court upheld EPA's use of evidence revealing important non-linearity in dose response to lead concentrations in the atmosphere, in the face of industry arguments opposing the claim of non-linearity. The commenter expresses that the EPA now appears to support the Battery Recyclers' opposition, because the evidence used suggested a non-linearity opposite to that preferred by industry. The commenter provides that the evidence comprised four independent research studies of correlations between blood lead levels and the intelligence quotient (IQ) of young U.S. children. Rather than supporting a threshold effect, the commenter notes that the data indicated that incremental IQ loss actually increased at lower blood lead levels. The Agency thus based its revised standard for atmospheric lead concentrations on analyses of children with blood lead levels closest to the low average levels of children in the United States today. The commenter contends that, having failed to persuade the Circuit Court of its position, industry, it appears, has now found a friendly audience in the current EPA administration. The commenter explains:
      The petitioners in the above case complained that EPA had relied more on actual research data for the most vulnerable population sector than on generic risk assessment models that would have suggested greater uncertainty regarding the non-linearity. The petitioners ignored the general argument that models are useful only to the extent that they maintain consistency with actual research data. The issue of lack of public access to the raw underlying data was raised for only one of the four independent research studies on which EPA relied. Had EPA been required to eliminate that study from consideration, as suggested by the proposed new rule, the Agency would have had a more difficult task to convince a court that they were fulfilling their responsibility "to protect the public health with an adequate margin of safety."
Response: The EPA disagrees that the rule is biased in favor of industry. As explained in the preamble to the final rule, the EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will increase the transparency of the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. The final rule also does not allow for the categorical exclusion of studies based on the availability of their underlying data.
Comment: Commenters (4840, 4884, 5019, 6132, 6137, 6144, 6171, 6351, 6868, 6876, 6912, 6955, 9227) specifically point to the tobacco industry lawyer tactic of attacking science/studies and other industry "secret assertion" tactics that did not benefit the industry as being similar to what they believe is the intent of the proposed rule. Several of these commenters provide background information to support their assertion and also note that these tactics were the basis of a failed attempt at legislation (i.e., HONEST Act) and bias in favor of the use of industry studies/industry. 
Commenter (6132) notes that a past example of abuse by consultants to a vested interest resulted when the State of Georgia set up an Open Records Law. The commenter states that the RJ Reynolds Company used that law to obtain research data to attack study findings that the use of cartoon characters (such as "Joe Camel") in tobacco advertising influences children's product recognition. The commenter provides that the research was later validated in other studies, but the damage was done, and the physician involved left research for private practice. Thus, the commenter notes that this data release approach has already been tried in the past and has been shown to be too easily abused by vested interests. The commenter also points out that there is also a tobacco industry connection to the proposed rule. Just before the 1997 Open Data amendment was presented in the House, the commenter states that a December, 1996 memo from a consultant to the Tobacco industry, laid out a similar strategy to address federal agency science with respect to secondhand smoke, including a now familiar call for science "Transparency". The commenter states that the tobacco industry's past record of abuse of data they obtained under similar rules, in order to inappropriately attack researchers, makes obvious polluting industries' motive for this misguided change in EPA's handling of scientific data for decision-making.
Commenter (9227) adds that the history of the proposal's internal development indicates that certain representatives of regulated industries had a nearly exclusive role in its promulgation, and that industry concerns were given special solicitude by EPA's senior political leadership. Meanwhile, the commenter complains that the scientific community and the EPA's own SAB were neither involved in the evolution of the proposal nor notified of its initiation until after its official publication in the Federal Register. 
Commenter (6351) notes that industries and their allies have been pushing to exclude studies for decades, using the same arguments found in EPA's proposal, targeting research that shows harm to public health from their products or their emissions. The commenter provides that, in 1996, attorneys working for tobacco industry giant R.J. Reynolds recommended a similar approach requiring review of documents "because, at some point in the future, EPA will most likely be ordered to re-examine ETS [Environmental Tobacco Smoke]." The commenter states that the EPA had issued its first report on ETS in 1992, concluding that secondhand smoke was responsible for approximately 3,000 deaths from lung cancer annually in nonsmoking adults. To prepare for the anticipated next report's likely conclusion of even greater harm from the products, the commenter provides that R.J. Reynolds attorneys developed a strategy to cast doubt on the studies while obscuring the company's real purpose. As they explained in the memo:
      Because there is virtually no chance of affecting change on this issue if the focus is ETS, our approach is one of addressing process as opposed to scientific substance, and global applicability to industry rather than focusing on any single industrial sector. Thus the examples of questionable science, to justify these standards. Congress must require those examples serve as the test cases.
The commenter (6351) adds that tobacco attorneys recommended expanding this approach to other industries, which quickly happened. Two of the early industry targets were landmark air pollution studies completed in the 1990s that found solid evidence that particulate matter air pollution could cause premature death., The commenter states that both studies found the particulate matter in the air was linked to increased risk of premature death. The commenter provides that the EPA incorporated these studies into their review of the research, leading to the first national standard for fine particulate matter (PM2.5) in 1997 and were challenged in the 1990s by members of Congress and their industry supporters seeking access to the confidential patient information, arguing that the raw patient data should be public since the research was federally funded. The commenter adds that other scientists argued for more investigation of whether confounding factors, insufficient years of data collection or other limitations might mean that the findings were not as powerful as they appeared to be. According to the commenter, instead of blocking the studies, as this proposal would do, EPA took a logical step and referred both studies to an independent third party, the HEI, for a deep-dive review. There, the commenter notes, autonomous reviewers examined the data and developed a report on the two studies that confirmed their original findings. 
Commenters (6137, 6144, 6912, 9227) provide a historical description of tactics/bias regarding sound science/secret science and the attempted HONEST Act legislation that they contend was the basis for the proposed rule. The commenters also highlight the proposed rule's biased treatment that favors industry studies over studies conducted by independent academic researchers. Some of their comments (which include points indicated by other commenters) are provided in the following paragraphs.
Commenter (6912) asserts that attempting to require the government to use only studies with publicly accessible data in its rulemaking, and thereby eliminating the most credible science based on actual health information, is a well-documented strategy with roots in the tobacco and fossil fuel industries. The commenter notes that during the mid-1990s, the tobacco industry and its lawyers and lobbyists strategized about changing the rules governing the use of science in agency rulemaking. Christopher Horner, then a lobbyist for the law firm Bracewell and Patterson (now Bracewell), wrote a memo in 1996 to his client, R.J. Reynolds, outlining a strategy for how to head off future regulation of secondhand tobacco smoke. In this memo, the commenter states that Horner acknowledged that R.J. Reynolds has "virtually no chance" of stopping such a regulation unless the company is able to exercise "behind the scenes leadership" in "construct[ing] explicit procedural hurdles [EPA] must follow in issuing scientific reports." 
The commenter (6912) expresses that Horner's recommended approach to constructing these procedural hurdles is strikingly similar to EPA's proposed rule. The commenter provides that Horner advised that R.J. Reynolds advocate for what he termed "sound science" criteria in rulemaking, requiring the data underlying any study to be publicly available and that a study's results be reproducible. Horner further recommended that such a rule be implemented via either an EPA rulemaking or new legislation.
Commenter (6912) further explains that Horner suggested that oversight hearings on EPA's rulemakings to limit fine particulate matter (PM2.5) and ozone serve as the congressional entry point for the tobacco industry 's agenda. Fine particulate matter and ozone are both principally generated by the burning of fossil fuels. The commenter adds that, by suggesting that the "sound science" initiative be launched around PM 2.5 and ozone rulemakings, Horner accomplished two goals: hiding the hand of the tobacco industry and helping broaden the coalition of industries supporting the initiative.
In fact, commenter (6912) states that the fossil fuel and electric utility industries were quite concerned about the potential for new regulations limiting PM 2.5 and ozone. The commenter provides that Citizens for a Sound Economy (CSE), a front group founded by Charles and David Koch of Koch Industries, opposed new PM 2.5 regulations. CSE was led by C. Boyden Gray, an heir to the R.J. Reynolds tobacco fortune. The commenter adds that Gray also ran another front group, the Air Quality Standards Coalition, which was created by the National Association of Manufacturers (NAM) to fight EPA air pollution regulations.
Commenter (9227) also asserts that the proposal's justifications regarding the private-sector burden of regulatory costs reiterates concerns and suggestions about EPA's policy for evaluating science that the Agency received from industry itself and cites emails to EPA leadership from May 2014, the National Association of Manufacturers ("NAM") that specifically identified dozens of EPA regulations that were "affecting its members," many of which were chemical, air, and water regulations which were based upon the types of research and studies that would be excluded under EPA's proposed rule.
In response to EPA's 2017 proposed rule, commenter (9227) provides that the Procedures for Prioritization of Chemicals for Risk Evaluations, NAM made recommendations that EPA ensure that Toxic Substances Control Act (TSCA) prioritization relied upon "the best available science" in a process that requires "a heightened level of transparency." The commenter notes that NAM also provided the EPA with materials that called for reform of EPA's "process for evaluating science to improve transparency and better involve the public." This, according to the commenter, parallels NAM's 2014 letter to the House in support of that year's version of Rep. Smith's HONEST Act. (Whom is discussed further below.)
The commenter (9227) adds that the American Petroleum Institute's (API) Senior Director of Regulatory and Scientific Affairs wrote to the EPA: "[t]he science and data used to support a regulation should be reviewed to determine if they are still valid based on scientific integrity, consistent with EPA's Principles of Scientific Integrity and Policy (2012), with meaningful disclosure of all potential areas of bias, guarding against manipulation or misinterpretation." The commenter adds that API also issued a press release on that same day, May 15, 2017, in which the organization summarized its conversations with EPA: "API today urged the EPA to adopt a regulatory system that enhances safety and protects the environment while prioritizing the production and refining of American natural gas and oil."
In contrast, the commenter (9227) states that the EPA's SAB leadership was not notified of the rulemaking activity until it was published in the Federal Register, in contravention of Agency practices for communicating major actions such as the proposed rule. The commenter adds that the EPA also failed to provide the SAB with a description of the proposal. The commenter notes that SAB leadership took note of the HONEST Act's connection to the proposal, stating the rule was "highly controversial" as indicated by the fact that "a similar legislative effort in the House has been stalled in Congress for several years."
Commenter (6912) notes that the fossil fuel industry strongly supported the tobacco industry 's so-called "sound science" initiative. By January 1997, the commenter states that Citizens for a Sound Economy (CSE) organized a protest at a Senate hearing on proposed EPA air pollution regulations. The commenter notes that protestors, dressed in white lab coats, held signs saying "Harvard , release the data!" in reference to a Harvard study demonstrating a link between PM2.5 pollution and increased mortality. The commenter describes that an October 1997 memo obtained from Philip Morris lists the API, the Electric Power Research Institute (EPRI), and the Statistical Assessment Service, a climate denier front group funded by fossil fuel dynasties such as the Kochs and the Scaifes, as three of the five organizations supporting the "sound science" initiative.
In 1998, the commenter (6912) provides that the initiative was rebranded "secret science" by Powell Tate, a lobbying firm working for the tobacco industry. The commenter notes that "secret science" is of course the very phrase Pruitt used when announcing the proposed rule, writing "[t]he era of secret science at EPA is coming to an end." The commenter adds that Powell Tate recommended identifying a ''catalyst organization" to push the initiative, and suggested NAM and the Business Roundtable as two possibilities. Both NAM and the Business Roundtable count many fossil fuel companies among their members.
During the 1990s, commenters (6912, 9227) submit that Philip Morris (today, Altria) and public relations firm APCO partnered to create a front group to support its campaign to make it more difficult for regulatory agencies to use scientific studies in rulemaking: The Advancement of Sound Science Coalition (TASSC). Commenter (6912) notes that, while a creation of big tobacco, the commenter states that TASSC 's donors also included the oil companies Amoco, Chevron, ExxonMobil, and Occidental Petroleum. As such, the commenter adds that TASSC devoted significant efforts to challenging climate science and its executive director, Steve Milloy, helped shape the "Global Climate Science Communications Plan," an API project to develop a strategy to confuse and mislead the public about climate change. Commenter (9227) provides that TASSC led a worldwide publicity campaign in the 1990s to promote "Good Epidemiological Practices" that aimed at undermining U.S. and international regulatory efforts based on epidemiologic studies of passive smoking and lung cancer.
During the same period, commenter (9227) states that Philip Morris made it a strategic priority to pursue legislation and policies to require public disclosure of epidemiological data. A May 1997 planning document advocated for using "existing political and business coalitions" that opposed clean air regulations to promote "legislative solutions to ensure that public policy is based on sound science" and "require epidemiological studies to meet a minimum set of criteria and/or require researchers to make public the underlying data before these studies can be used as a basis for regulations at the state or federal level." In 1998, the commenter adds that Powell Tate  -  a lobbying firm that represented R.J. Reynolds  -  organized a "secret science" working group focused on "requiring the disclosure of taxpayer-funded analytical data upon which federal and state rules and regulations are based, as well as the analytic data underlying health and safety studies funded by the government . . . ."
Commenter (9227) provides that, although TASSC no longer exists, its executive director, Steve Milloy, continues the organization's "sound science" rhetoric against other types of regulation through his website, JunkScience.com. In fact, the commenter adds that Milloy has personally taken credit for EPA's proposal and was one of a select few invited to Pruitt's public announcement of the proposal earlier this year. After the proposed rule was announced, Milloy told reporters, "I look at this as one of my proudest achievements. According to the commenter, the reason this is anywhere is because of Steve Milloy."
Commenter (6912) suggests that Milloy, along with Horner and a small number of lawyers and lobbyists closely tied to the tobacco and/or fossil fuel industries, drove the industries' two-decade-long push to erect "procedural hurdles" to the use of science in rulemaking under the guise of defending "sound science" and defeating "secret science." The commenter notes that after Pruitt announced the proposed rule at an event to which Milloy was invited, Milloy declared that he'd been working on furthering the "secret science" initiative "for twenty years."
Commenter (6144) asserts that the EPA's proposed rule is not a novel idea. The commenter notes that it originates from the tobacco industry, whose lawyers and lobbyists in the 1990s developed a version of this idea in an attempt to prevent the federal government from regulating second-hand smoke. The commenter provides that the goal back then, as it appears to be today, is to create "procedural hurdles" that would hamstring an agency's ability to implement policy that would protect public health. Furthermore, the commenter contends that the records obtained by the Union of Concerned Scientists through a FOIA request clearly illustrates that this proposal has never been about transparency, and definitely not about science. Under the direction of former Administrator Pruitt, the commenter states that the proposed rule was entirely driven by politics and a way to ignore the will of Congress by administratively implementing the misleadingly named HONEST Act of 2017. The commenter notes that one email sent by an EPA political appointee to his colleagues most accurately describes this proposed rule. In an email dated January 16, 2018, the staffer writes that this idea, if implemented, would result in "no regulation (going) into effect unless the scientific data is publicly available for review." It seems clear that in proposing this rule, EPA's intent is to prevent the promulgation of public health and environmental safeguards, and not increase access to information.
Commenters (6868, 6876) add that the tobacco industry lawyer who authored this 1996 document served on the Trump EPA Transition Team and that it appears that the EPA is literally adopting policy developed by the tobacco industry to undermine the public confidence in pivotal scientific studies. Following Donald Trump's election, the commenter (6912) submits that Milloy joined the EPA transition team, as did Horner. The commenter adds that Milloy celebrated EPA's proposed rule limiting science in rulemaking on Twitter, noting that he was "inside the administrator 's office:"
Commenter (6137) notes that industry lobbied Congress in an effort to gut environmental and health laws by attacking the science upon which they are based. For example, the commenter states that both the Secret Science Reform Act of 2014 and the Secret Science Reform Act of 2015 provided that EPA may not take action "unless all scientific and technical information relied on to support such covered action is . . . publicly available online in a manner that is sufficient for independent analysis and substantial reproduction of research results." Two years later, after its prior unsuccessful attempts, the commenter states that the House passed the HONEST Act in March 2017, which again would have limited EPA's ability to perform any assessment or analysis based on science if the public does not have complete access to the underlying data. When these bills did not succeed, the commenter notes that industry looked to a new audience to try put in place what Congress failed to enact: a ban on consideration of science when making regulatory decisions critical to public health and the environment if the underlying data is not made publicly available. 
Commenter (6137) states that industry pitched EPA hard, and found a willing ear, complaining that air pollution limits and toxin tolerances were being set at levels that were too stringent. Given that the strong Congressional mandate expressed in numerous statutes for strict health protections would require EPA to act if the science demonstrated a risk to public health, the commenter contends that industry saw that their best bet was to knock out the science, this time through EPA itself. The commenter provides that industry groups, including the National Association of Manufacturers and American Petroleum Institute, "pitched EPA a Proposal last spring that closely resembled what became Administrator Scott Pruitt's `secret science' plan," according to EPA internal documents. This plan closely tracks the longstanding attacks on EPA science from these groups. According to the commenter, EPA was responsive to the industry pitch and met with industry groups dozens of time, while repeatedly canceling the few scheduled meetings with public health advocates.
Commenter (6137) states that after EPA published its final Risk Assessment for Environmental Tobacco Smoke (secondhand smoke) in 1992, which concluded that secondhand smoke "is a human carcinogen, responsible for approximately 3,000 lung cancer deaths annually in U.S. nonsmokers," the commenter notes that the tobacco industry went on the attack. The commenter states that the Risk Assessment had been based in part on a meta-analysis of "31 epidemiologic studies from 8 different countries" which showed a significant risk of harm. The commenter contends that recognizing that "[v]igorous denial is not a satisfactory defensive strategy" and that "the most significant [secondhand smoke] problem facing the Industry is the result of epidemiological studies which indicate" a risk from exposure, the tobacco industry decided to attack epidemiological science. The commenter adds that industry lawyers candidly noted that "there is virtually no chance of affecting change on this issue if the focus is" secondhand smoke so "our approach is one of addressing process as opposed to scientific substance, and global applicability to industry rather than focusing on any single industrial sector." Shortly thereafter, the commenter states that one of RJ Reynolds' lobbying firms organized a "Secret Science" Work Group to "[f]ocus public attention on the importance of requiring disclosure of taxpayer-funded analytical data."
The commenter (6137) asserts that the Proposed Rule is just another effort to hide evidence of the public health harm of toxic chemicals, given that there is no way to change the research results showing the deleterious effects. And this time, the commenter states that the effort is much broader, as it is not hiding evidence related to just one industry or one product. According to the commenter, EPA should not be permitted to "deliberately misle[a]d the public about the risks of" certain pollutants or other chemicals by hiding evidence of their harms. As several courts have found, the commenter concludes that the best available politics does not equate to the "best available science," so while this Proposed Rule may serve EPA's political ends, it does not meet the mandates of sound science.
Commenter (6912) notes that another key player in the push to limit science in rulemaking is Texas Congressman Lamar Smith. The commenter provides that Smith has been the chairman of the House Committee on Science, Space, and Technology since 2013. According to the commenter, the oil and gas industry is Smith's largest source of campaign donations, contributing nearly $800,000, and Smith has drawn most of his committee staff from the fossil fuel industry. In his role as chairman of the House Science Committee, the commenter suggests that Smith has embraced the "secret science" initiative championed by Milloy, Horner, and the network of tobacco and fossil fuel industry-funded front groups that employ them. According to the commenter, in a 2013 op-ed in the Wall Street Journal , Smith attacked EPA for not releasing the data underlying two important peer-reviewed studies that demonstrated the increased mortality associated with exposure to fine particulate matter. The commenter adds that, following several letters to EPA demanding the underlying data for these studies, Smith issued a subpoena to EPA, notwithstanding the fact that Harvard University and the ACS, not EPA, possessed the confidential public health data at issue.
Commenter (6912) provides that, realizing he could not force EPA to turn over documents it did not possess, Smith introduced legislation to prohibit EPA from considering scientific studies whose underlying data were not public. In 2015 the commenter states that he introduced the Secret Science Reform Act, a bill supported by ExxonMobil, Integrys Energy Group, CMS Energy, Duke Energy, the Edison Electric Institute, the U.S. Chamber of Commerce, Peabody Energy, and NAM. In 2017, Smith introduced the Honest and Open New EPA Science Treatment Act of 2017 (HONEST Act), which, like the Secret Science Reform Act before it, would prohibit EPA from considering scientific studies whose underlying data are not public. Once again, the commenter suggests that energy interests, including ExxonMobil, the U.S. Chamber of Commerce, and NAM all lobbied in support of the bill.
Similarly, commenter (9227) asserts that a close examination of the history of this Proposal confirms that its purpose is not to strengthen science at EPA, but to undermine public health and environmental protections by arbitrarily blinding the agency to vital research. The commenter adds that the proposal resembles proposals advanced by the tobacco industry for the specific purpose of suppressing public health science warnings about the dangers of tobacco smoke. The Proposal also resembles failed legislation in Congress that was similarly advanced by industry interests seeking to undermine public health and environmental protections, and criticized by scientific experts.
Commenter (9227) also provides arguments regarding how the proposal is an outgrowth of the failed HONEST Act congressional bill:
      Internal and external EPA communications illustrate that the HONEST Act served as a precursor to EPA's Proposal. The intertwined history of the HONEST Act and EPA's Proposal cast doubt on the Agency's proffered rationale.
      The HONEST Act
      The HONEST Act is a House bill introduced in 2017 by sponsor Representative Lamar Smith (R-TX), and is the latest manifestation of various bills aimed at undermining EPA regulation through limitations on the types of scientific research the Agency may use.The HONEST Act and these related bills were introduced and passed in the House three times, but each time, failed to progress in the Senate.
      Like the current Proposal, the HONEST Act was touted by its proponents as an effort to enhance the transparency and credibility of regulatory science at EPA. But the HONEST Act --  like the Proposal -- would in fact have had the effect of limiting the scope and quality of science underlying EPA actions. Indeed the HONEST Act was widely criticized and opposed by scientists, scientific organizations, medical organizations and other scientific authorities for precisely this reason. For example, eight public health and medical associations including the American Lung Association, American Public Health Association, National Medical Association, and Physicians for Social Responsibility issued an open letter to Congress in spring 2017 opposing the HONEST Act because it "would limit the kinds of scientific data EPA can use as it develops policy to protect the American public from environmental exposures and permit violation of patient confidentiality." The American Association for the Advancement of Science and twenty-two other leading scientific organizations and research universities likewise sent a letter to House Majority Whip Kevin McCarthy in March 2017 opposing the bill and warning that it could lead to a "situation where the EPA would be prevented from using the best available science and disseminating public information in a timely fashion."As we have noted elsewhere in these comments, the CBO  -  after consulting with EPA staff  -  likewise concluded that the HONEST Act would "significantly reduce the number of studies that the agency relies on when issuing or proposing covered actions."
      That the HONEST Act would suppress rather than promote good science at EPA is not surprising, given that the sponsors of the HONEST Act have a history of rejecting established climate science and strong ties to industries that would benefit from limiting the role of science in EPA rulemakings. Representative Lamar Smith is widely known as an opponent of mainstream climate science and public health and environmental safeguards. In a July 24, 2017 opinion piece, Representative Smith lauded the benefits of increased atmospheric carbon dioxide: "A higher concentration of carbon dioxide in our atmosphere would aid photosynthesis, which in turn contributes to increased plant growth." Smith and the sponsor of the Senate version, Mike Rounds, also receive substantial contributions from the same industries that will benefit from the proposal.
      Representative Smith also has ties to EPA staff who drafted the proposal, underscoring the close connection between his failed legislation and this proposed rule. Dr. Richard Yamada, former professional staff member on Smith's House Committee on Science, Space & Technology now serves as the Deputy Assistant Administrator for EPA's Office of Research and Development. At EPA, Dr. Yamada has participated in the drafting and development of the Agency's version of the proposal.
      The HONEST Act as Predecessor for the Proposal
      As this section details, it is clear that the HONEST Act is a direct predecessor of this proposed rule and that both initiatives share the same purpose: to undermine EPA's use of rigorous science in crafting health and environmental protections. The language used in the proposal shares strong similarities with the HONEST Act. Furthermore, internal and external communications from EPA leadership demonstrate the proposal's origins in the HONEST Act.
      While lengthier than the congressional HONEST Act, EPA's proposal contains parallel language to the bill. One can compare examples from the text of the 2017 HONEST Act as passed in the House, to the text of the proposal from the Final Federal Register Notice:
      The HONEST Act of 2017
      An Act: To prohibit the [EPA] from proposing, finalizing, or disseminating regulations or assessments based upon science that is not transparent or reproducible....
      The Administrator shall not proposed, finalize, or disseminate a covered action unless all scientific and technical information relied on to support such covered action is -- (A) the best available science; (B) specifically identified; and (C) publicly available online in a manner that is sufficient for independent analysis and substantial reproduction of search results....
      Strengthening Transparency in Regulatory Science Proposal: EPA shall clearly identify all studies (or other regulatory science) relied upon when it takes any final action. EPA should make all studies available to the public to the extent practicable . . . When promulgating significant regulatory actions, the Agency shall ensure that dose response data and models underlying pivotal regulatory science are publicly available in a manner sufficient for independent validation.
      The best available science must serve as the foundation of EPA's regulatory actions.
      Responsive records released to the Union of Concerned Scientists ("UCS") make evident that the HONEST Act served a predecessor to the proposal. Administrator Pruitt's schedule reveals that he met with Representative Smith on January 9, 2018, less than four months before the Federal Register announcement of the proposal. Emails from Pruitt and his staff, dated just over a week after that meeting, indicate that Smith was working on a "pitch that EPA internally implement the HONEST Act." Subsequent emails sent between Pruitt's EPA staff in February 2018 demonstrate that EPA officials promptly began drafting the proposal.
      Before Smith's internal EPA `pitch,' Agency leadership commented favorably on the HONEST Act of 2017. Although EPA initially estimated that implementation of the act would cost over $250 million per year, that estimate was never reported to the CBO. As CBO's cost estimate determination indicates, EPA political leadership diverged from the earlier estimate and instead assured CBO that the bill could be implemented "with minimal funding." Several news sources have reported that the Administrator's Office of the EPA became involved in communications with CBO, and decided to respond to CBO directly with the assurance the bill could be implemented at `no cost.'
      Finally, in an exclusive interview with the Daily Caller shortly before the proposal's publication, former Administrator Pruitt promised:
      If we use a third party to engage in scientific review or inquiry, and that's the basis of rulemaking, you and every American citizen across the country deserve to know what's the data, what's the methodology that was used to reach that conclusion that was the underpinning of what  --  rules that were adopted by this agency.
      The Daily Caller directly linked the proposal to the HONEST Act, "Pruitt's pending science transparency policy mirrors Smith's HONEST Act, which passed the House in March 2017."
      Spokeswoman for Chairman Smith's House Committee on Science, Space, and Technology, Thea McDonald, also told the Daily Caller: "[t]he chairman has long worked toward a more open and transparent rule-making process at EPA, and he looks forward to any announcement from Administrator Pruitt that would achieve that goal."
Response: The EPA, the federal government, the international scientific community, and Congress have pursued efforts to increase data sharing and transparency for many years. The EPA is committed to furthering this effort with the implementation of this final rule. This rulemaking is in no way related to any industry, including the tobacco or energy industries, or an EPA review of environmental tobacco smoke. The EPA's attention to data transparency is responsive to the broader interest in greater data transparency observed in the numerous transparency initiatives in the scientific community and federal government, as well as the criticism the EPA has received from members of the public, scientific community, and Congress on the transparency of the scientific basis for EPA's decisions in previous influential scientific information assessments and regulatory actions. Although these criticisms helped shape the rulemaking, the actual text of the rulemaking was initiated and revised by EPA personnel and was not overly influenced by any particular stakeholder. 
The EPA provides more detail explaining its rationale for this rule in the preamble for the final rule. In brief, the transparency provisions in this final rule are intended to build upon existing efforts and provide incremental progress toward the Agency's overarching goal of greater transparency. The EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a good government principle that is inherently valuable to the public. The scientific community has also embraced greater data transparency, as evidenced by data sharing and availability requirements for many high-impact journal publishers and the emergence of several international organizations and initiatives focused on increased transparency and data sharing in science. The EPA supports these efforts and believes that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation provides additional certainty and increases the transparency of the EPA's scientific analyses and resulting regulatory actions. The incremental progress made by this rule provides an important step towards prioritizing transparency in the EPA's most impactful rules and assessments and will inform future statute-specific rulemakings.
In consideration of public comments on this rulemaking, the EPA did not finalize the 2018 proposal or the 2020 SNPRM proposed option that would have categorically required that the underlying data be available for independent validation in order for studies to be considered pivotal science. However, given that transparency is a very important aspect of EPA's regulatory actions and assessments it should be an important consideration in how the Agency considers pivotal science. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating final significant regulatory actions and finalizing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
The EPA disagrees with commenters that requirements of this rule will result in procedural hurdles resulting in any meaningful delay in promulgating regulations. While this rule does require the Agency to evaluate the availability of underlying dose-response data for pivotal science, the incremental burden to the Agency to carry out these requirements is expected to be small given (1) the extensive scientific review EPA already conducts regularly and (2) that the requirement is limited to pivotal science (i.e., a small, though highly important, subset of all the studies EPA reviews).
Comment: Commenters (6125, 6144, 6912, 6944, 9227) suggest that the pre-proposal correspondence between senior political officials at EPA, the Deputy Assistant Administrator for Chemical Safety and Pollution Prevention and industry, and the resulting proposal infer an industry bias.
Commenter (6944) suggests that the manner in which this rule was initiated began the process by clearly indicating whose input the EPA Administration would be considering and provided a clear example of what a predetermined (or desired) outcome would be as applied under this rule and what the Administrator and his staff considered as acceptable "peers" for review purposes. The commenter notes that conducting meetings with, and accepting input from, private sector business and industry that seek to reduce the regulatory impact and authority of the EPA, while essentially shutting out various environmental parties does not provide a good example of what will be accepted by the Agency Administrator and Staff as "transparent, understandable, and reproducible scientific assessments"
In early 2018, having been unsuccessful at moving his HONEST Act through Congress, the commenter (6912) asserts that Texas Congressman Lamar Smith (Rep.Smith) took his case directly to Administrator Pruitt (as noted previously). On January 16, the commenter states that Pruitt met with Smith to discuss implementing the HONEST Act at EPA. The commenter adds that, just a few days prior to Pruitt's meeting with Smith, internal EPA emails reveal that political appointees indicated that they had identified "a path forward" on the "secret science" initiative. These emails were sent between Brittany Bolen, EPA Senior Deputy Associate Administrator for Policy and former counsel for Senator Jim Inhofe, Richard Yamada, EPA Research and Technology Policy Advisor and former aide to Congressman Lamar Smith, Erik Baptist, EPA Senior Deputy General Counsel and former senior counsel for API, and Ryan Jackson, EPA Chief of Staff and former chief of staff for Senator Inhofe.
Commenter (6144) contends that EPA also appears to be more concerned with protecting industry data and CBI than with protecting public health information. According to the commenter, this disparity is exemplified by the fact that the agency requests that commenters provide feedback on how to balance protections for copyrighted and CBI but does not seek similar advice on how to protect patient health data. Furthermore, the commenter notes that emails obtained by Union of Concerned Scientists (UCS) through a FOIA request show that top EPA officials responsible for drafting the proposed rule, including Dr. Beck and her colleague Dr. Richard Yamada, the Deputy Assistant Administrator for the Office of Research and Development (ORD), struggled with how to incorporate exemptions and limit the impact on industry data. The commenter adds that comments raised by Dr. Beck on how to protect CBI are also very similar to those in her testimony in front of a Senate Homeland Security and Government Affairs subcommittee, where she represented her former employer, the American Chemistry Council, whose membership has a significant financial interest in EPA regulatory matters.
Commenter (9227) provides that EPA documents released in response to FOIA requests relating to the Proposal show that Trump Administration appointees deliberately tailored the scope of the Proposal in order to promote industry interests. Commenter (6125, 6912) notes that emails released pursuant to FOIA requests reveal that senior political officials at EPA did not know that pesticide registrations depend on information which is protected from disclosure to the public. The commenter states that when this was pointed out to them by Nancy Beck, Deputy Assistant Administrator for Chemical Safety and Pollution Prevention and a former senior director of regulatory science policy at the American Chemistry Council raised concerns that requiring underlying data be publicly available would create a problem for the chemical industry, the draft proposal was revised so that it would not cover licensing actions like pesticide registrations and other activities providing a benefit to industry. Commenter (9227) adds that this industry considers chemical testing data to be CBI. Several commenters (6144, 6912, 9227) state that FOIA requests indicate that Yamada assured Beck that he would address her concerns, responding he "didn't know about the intricacies of CBI - ok, we will need to thread this one real tight!" 
The commenter (6144) expresses that they believe the emails also show the lack of engagement with the independent scientific community, let alone EPA's own scientists. In addition, the commenter adds that the emails show that EPA not once considered the impact of this proposed rule on its ability to protect human health and the environment.
Commenter (6144) contends that the EPA is seeking an opportunity to protect data for the industries it is supposed to regulate at the expense of science-based public health safeguards. The commenter asserts that the focus on protections for industry over the public health research community illustrates that the agency is not interested in seeking actual transparency in decisionmaking through this proposed rule. The commenter asserts that the focus on protections for industry over the public health research community illustrates that the agency is not interested in seeking actual transparency in decisionmaking through this proposed rule. Commenter (9227) states that this distinction between using studies when they confer a benefit on industry but disallowing them when used to influence public health protection is not only arbitrary but insidious. Commenter (6144) states that the proposal is irremediably tainted at its inception as a result. Commenter notes that this bias is affirmed by Dr. Yamada's response to an email from Dr. Beck, where he writes that he "didn't know about the intricacies of [confidential business information]" and that the agency would have to "thread this one real tight!" 
Commenter (6912) asserts that, although it would be blatantly biased to treat industry studies differently than ones conducted by independent academic researchers, EPA found a way to "thread this one tight." The commenter submits that the proposed rule allows the Administrator to waive the data availability requirement for any study if it is not feasible to make the underlying data publicly available "in a manner sufficient for independent validation, in a fashion that is consistent with law, protects privacy, confidentiality, confidential business information, and is sensitive to national and homeland security." (Emphasis added.)
Response: The EPA's attention to data transparency is responsive to the broader interest in greater data transparency observed in the numerous transparency initiatives in the scientific community and federal government, as well as the criticism the EPA has received from members of the public, scientific community, and Congress on the transparency of the scientific basis for EPA's decisions in previous influential scientific information assessments and regulatory actions. Although these criticisms helped shape the rulemaking, the actual text of the rulemaking was initiated and revised by EPA personnel and was not overly influenced by any particular stakeholder. As described more fully in the preamble to the final rule, the transparency provisions are intended to build upon existing efforts and provide incremental progress toward the Agency's overarching goal of greater transparency. The EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a good government principle that is inherently valuable to the public. The scientific community has also embraced greater data transparency, as evidenced by data sharing and availability requirements for many high-impact journal publishers and the emergence of several international organizations and initiatives focused on increased transparency and data sharing in science. The EPA supports these efforts and believes that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation provides additional certainty and increases the transparency of the EPA's scientific analyses and resulting regulatory actions. The incremental progress made by this rule provides an important step towards prioritizing transparency in the EPA's most impactful rules and assessments and will inform future statute-specific rulemakings.
The EPA disagrees that the rulemaking was heavily influenced by or designed to benefit industry or provide preferential treatment to industry data. The EPA seeks to appropriately protect all sensitive information (i.e., PII, CBI, proprietary information) in this rule by providing equal consideration of studies with data that is publicly available or available through restricted access or, in the case of data that cannot be appropriately safeguarded, consideration of technical factors in 40 CFR 30.5(e) or an Administrator's exemption under 40 CFR 30.7 that would also allow greater consideration to be given in the absence of data availability. Further, the Agency treats all valid claims of CBI equally under the statutes that it administers, regardless of the entity that generates it. The treatment of CBI in this rule or under the statutes that the EPA administers does not give preferential treatment to industry. In both the NPRM and SNRPM the Agency asked the public to comment on all forms of confidential information, including PII and how best to protect that information while still facilitating the transparency intentions of the rule. 
The EPA also wishes to clarify that public input was an important part of shaping the final rulemaking. The EPA accepted public comment on the proposed rule from April 30, 2018 to August 16, 2018, and held a public hearing on July 17, 2018. In response to public comments, the EPA released a supplemental notice of proposed rulemaking and took public comment from March 18, 2020 to May 18, 2020. All of these comments were considered in developing the final rule.
Finally, the final rule maintains language in 40 CFR 30.3 stating that the statutes and regulations will control in the event of any conflicts, and clarifies in the preamble that the requirements in this final rule set the overarching structure and principles for transparency in the EPA's final significant regulatory actions and influential scientific information. The EPA intends to develop internal implementing guidance and promulgate either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule, which will clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes that the EPA administers.
2.1.3.5 Deliberate Attempt to Limit Science Used for Decision-Making
Comment: Commenters (1272, 1390, 1589, 1997, 2428, 2495, 2501, 2905, 4840, 5016, 5019, 6107, 6132, 6446, 6915, 8281-PH5, 9227) suggest that the Proposal is a deliberate/intentional attempt to limit science used for decision-making. Comments include:
 Commenter (1272) states that applying the proposed rule to decision making would decimate the amount of research policy makers had access to and would unfairly skew the information available to those parties. Commenter (1272) states that applying the proposed rule to decision making would decimate the amount of research policy makers had access to and would unfairly skew the information available to those parties. 
 Commenter states that scientific methods are based upon healthy skepticism. According to the commenter, determining that scientific findings will be ignored or considered, when forming public policies, based upon conditions that exceed the criterion for peer-reviewed presentations in scientific meetings and scientific publications is not important for transparency. The commenter asserts that the EPA Administrator, EPA, and other government agencies should not be developing or considering any definitions - about what science is and is not worthy for informing public policy --- that are incongruent with the consensus of scientific community. (1390)
 According to the commenter, the entirety of the proposed rule is steeped in purposefully obtuse and misleading language and the logic of the rule is dubious and built on unfounded suppositions, misrepresented data, and spurious conclusions, within the guise of increasing "transparency." (1589)
 Commenter (1997) states that the proposed rule is designed to make it so that the EPA can't use most of the best research on human health or cognitive outcomes (e.g. test scores), since most studies of human health and cognitive outcomes use restricted data sets. (1997)
 The Proposal is nothing more than an attempt to remove valid, relevant and peer-reviewed scientific data from the rule-making process without having provided the necessary scientific input to back up this proposal and suggests that the proposal be rejected. (2428)
 Commenter asserts that they believe that the purpose of this proposal is not to achieve "transparency," but rather to remove valid and relevant scientific evidence from the rule-making process to bias the outcome. (2495) 
 Commenter questions whether the EPA wants to regulate scientific research by making it available to the public is honestly for openness and transparency. Instead, the commenter suspects that the purpose is to derail science by subjecting it to scrutiny by people who don't understand or oppose it. Could the proposed regulation's intent be to someone's financial or ideological benefit? Or to slow scientific communication to a snail's pace? (2501)
 The proposed rule is designed to slow down the scientific process by requiring EPA to provide all raw data, including CONFIDENTIAL medical and business information, which would both jeopardize the studies and tie up EPA employees in a tangle of sifting, redacting, and plowing through unnecessary hurdles to providing scientific information. Rather than providing transparency, the commenter contends that it is a transparent attempt by this Administration to undermine science and sabotage the work of the EPA. (2905)
 Certainly, according to the commenter, calls for transparency and reproducibility sound reasonable, but exposing poorly conducted studies is not what this effort is about. Instead, the commenter contends that the proposal is designed to impose onerous and burdensome requirements on independent researchers, and would result in much environmental health science, particularly epidemiologic studies, being excluded from the evidence base and becoming irrelevant to efforts by EPA to protect the public and the environment. The commenter states that valuable research findings in areas like climate change, lead exposure and particulate pollution would be ignored. (4840)
 The commenter asks:
 Why would the EPA's political leadership not want to add to the studies? 
 I know they are on the "drill baby drill" ilk, but why would they want to expose themselves and the public to possible adverse health effects of these new pollutants?
 Additionally, why is the EPA curtailing the activities of their staff scientists who are so experienced? (5016)
 The proposal is intentionally misleading and being sold by EPA leadership as an effort to increase "transparency." The commenter contends that the facts suggest that the real motivation is simply to sweep under the rug the scientific evidence disfavored by polluting companies. The commenter notes that the flimsy proposal was designed without adequate input from the scientific community, according to members of EPA's own SAB. The commenter adds that it was rushed through the regulatory process and was originally proposed with a gallingly short public comment period, suggesting an intention of casting less light on the rulemaking process, not more. (5019)
 Forcing EPA to ignore such private (and appropriately non-public) data, the proposed anti-environment rule is fiendishly calculated to block the EPA staff from considering the best data available. The commenter notes that this scheme has at least two devastating impacts: 
 Would nonsensically bar EPA from considering the best scientific data available disabling EPA rulemaking. 
 Would violate the agency's common sense and statutory duty to consider the best data. (6107)
 The abuse of research data to undermine science credibility is likely the most dangerous aspect of this proposal. (6132)
 The fact that the proposed rule is at odds with the manner in which the scientific community evaluates research -- underscored by the large number of comments submitted in opposition to the rule by scientists, scientific organizations, and scientific journals -- and also at odds with methods EPA uses and is required to use to evaluate scientific information suggests that the proposed rule's true purpose is other than to ensure that EPA relies on the "best available science." Instead, the commenter contends that the true purpose appears to be to exclude from consideration scientific research that might support a need for rigorous regulatory action. The commenter recommends that EPA continue to use of the comprehensive criteria used by the scientific community (and historically used by EPA) to evaluate scientific research. (6446)
 The EPA's failure to consult with its own internal science experts when developing the proposal is, at best, gross malfeasance and, at worst, a conscious effort to subvert the Agency's statutorily mandated practice of using the best available science. (6915)
 The proposed rule has nothing to do with transparency and is a thinly veiled campaign to limit serious and highly credible scientific research that supports critical regulatory action. The commenter inquires: Why would a science-driver public agency undertake such a radical departure from existing and widely accepted scientific standards? They note that they have yet to hear a credible answer to this question that is not rooted in favors to industry polluters. (8281-PH5)
 While the Proposal suggests that its aim is to improve transparency by increasing public availability of data, in actuality it proposes none of the steps that a proposal seriously aimed at that goal would propose, such as increasing funding for EPA grantees to undertake this effort, or proposing solutions to real concerns about patient confidentiality. Instead, the commenter contends that the heart of the proposal is a bar on considering science simply because the underlying data is not publicly available, regardless of whether the science has been peer reviewed, reproduced, or contains other hallmarks of scientific quality. The commenter adds that the agency's recent communication to the CBO that a similar Congressional proposal could be implemented at "no cost" proves the point: EPA's aim here is not to make more data available (which costs money), but to rely on less science in decisionmaking. The commenter adds that the agency's recent communication to the CBO that a similar Congressional proposal could be implemented at "no cost" proves the point: EPA's aim here is not to make more data available (which costs money), but to rely on less science in decisionmaking. That, according to the commenter, appears to be the current Administration's goal. (9227)
Response: The EPA disagrees that the intent of this rulemaking is to limit the science used for decision-making. As described more fully in the preamble to the final rule, the transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency. The EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. The scientific community has also embraced greater data transparency, as evidenced by data sharing and availability requirements for many high-impact journals and the emergence of several international organizations and initiatives that incentivize greater transparency in research. The EPA supports these efforts and believes that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. The EPA's continued progress toward maximizing transparency is vital to building and maintaining trust with the public and credibility in the Agency's decisions. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments.
The EPA agrees that it is important to consider the totality of relevant, quality research. Therefore, the EPA did not finalize the 2018 proposal or the 2020 SNPRM proposed option that would have categorically required that the underlying data and models be available for independent validation in order for studies to be considered pivotal science. However, given that transparency is an important aspect of EPA's regulatory actions and assessments it should be an important consideration in how the Agency considers pivotal science. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
Consistent with existing Agency practice, the EPA will use existing factors (including soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review) to evaluate study quality. When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence (consistent with Agency guidelines on hazard identification and dose-response assessment) to support a relationship between exposure and effect, the EPA will identify a subset of those studies for use in characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect. This will be based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both will be identified as pivotal science. Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. In compliance with the rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or propriety information, available through restricted access in a manner sufficient for independent validation. Including this review of data availability for pivotal science is critical to the EPA's progress toward increased transparency and providing increased opportunity for scientific reanalysis and scrutiny by independent third parties. This approach will result in final significant regulatory actions and influential scientific information that are based on the highest quality, most relevant studies and ensure adequate human health and environmental protections, while maximizing transparency and maintaining consistency with the statutes the EPA administers. When the dose-response data underlying pivotal science are not publicly available in a manner sufficient for independent validation, the EPA will give less consideration to the studies' evidence, findings, and conclusions unless the Administrator grants an exemption request.
The EPA disagrees that this final rule is biased in favor of industry. As stated in CFR 30.3, "The provisions of this part apply to science that serves as the basis for informing a final significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science." Further, the rule requirements direct the EPA to evaluate the availability of the underlying dose-response data for any subject matter expert, regardless of their affiliation.
The EPA also wishes to clarify that public input was an important part of shaping the final rulemaking. The EPA accepted public comment on the proposed rule from April 30, 2018 to August 16, 2018, and held a public hearing on July 17, 2018. In response to public comments, the EPA released a supplemental notice of proposed rulemaking and took public comment from March 18, 2020 to May 18, 2020. All of these comments were considered in developing the final rule.
Finally, the EPA disagrees that requirements of this rule will result in any burdensome requirements on independent researchers, a curtailing of staff scientist activities, or any meaningful delay in promulgating regulations. This is a rule of internal procedure that does not impose requirements on any party other than the EPA. This rule imposes no costs on researchers or their institutions, and the EPA will continue to consider and evaluate all high-quality, relevant science in its significant regulatory actions and influential scientific information. While this rule does require the Agency's staff scientists to evaluate the availability of underlying dose-response data for pivotal science, the incremental burden to the Agency's staff scientists to carry out these requirements is expected to be small given (1) the extensive scientific review EPA already conducts regularly and (2) that the requirement is limited to pivotal science (i.e., a small, though highly important, subset of all the studies EPA reviews). 
2.1.3.6 Deliberately Prohibiting Epidemiological Studies
Comment: Commenters (1409, 1905, 2171, 2408, 4410, 4840, 6137) suggest that the Proposal appears to intentionally limit EPA's use of epidemiological health studies. Comments include:
 The requirement that data be publicly available may sound "transparent," but in fact seems to (deliberately) prohibit epidemiological studies in which the underlying data is subject to confidentiality requirements. (1409)
 Commenter contends that the rule targets long-term epidemiological studies that linked air pollution to shorter lives and were used to justify air-quality regulations. The commenter asserts that the rule could keep that and other high-quality evidence from being used to shape regulations, even if there are legitimate reasons, such as patient privacy, why some data cannot be made public. (1905)
 The proposed rule was clearly designed to undermine and disparage the important epidemiological studies that support public health protection from all pollutants, be they in the air, water, or soil. Simply stated, according to the commenters, the proposal was written with the intent to cast doubt on EPA's prior judgement of and dependence on, health research - and to create suspicion significant enough to deter future use of health-based studies in regulatory decision-making. The commenters assert that the EPA's proposal flagrantly ignores the reasons for the privacy of health data used for epidemiological studies. (2171, 4410)
 The rule as designed would help polluters avoid regulations that protect public health and would intentionally limit EPA's use of epidemiological health studies because they contain confidential medical information that cannot be shared with the public. (2408)
 Much environmental science, particularly epidemiologic studies, would be excluded from the evidence base and would become irrelevant to efforts by EPA to protect the public and the environment. (4840)
Response: The EPA disagrees that the intent of this rulemaking is to undermine or otherwise limit the EPA's use of epidemiological studies in its decision-making. The transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency. The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety.
The EPA agrees that it is important to consider the totality of relevant, quality research. Therefore, the EPA did not finalize the 2018 proposal or the 2020 SNPRM proposed option that would have categorically required that the underlying data and models be available for independent validation in order for studies to be considered pivotal science. However, given that transparency is an important aspect of EPA's regulatory actions and assessments, it should be an important consideration in how the Agency considers pivotal science. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating final significant regulatory actions and finalizing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
2.1.3.7 Reliability of a Study Should Be Determined by Experts in the Field
Comment: Commenters (0671, 1481, 1973, 2907, 4933, 8274) assert that how studies are conducted, reproduced and used should be determined by experts in the field and not by the EPA's political appointees. Comments include:
 The decision to make data public (and which data to make public) should stay between the researcher, the IRB, and journal editors and not through waivers granted according to politically biased and often scientifically illiterate views of Administrators. (0671)
 It is unclear why an EPA Administrator rather than experts in the field should be given final authority on the reliability of the study. The commenter notes that peer reviewed publications already exist to address concerns over poor statistical analysis and scientifically unethical policies by subjecting studies to other experts in the field prior to publication. The commenter states that giving the EPA Administrator say in which of the studies did or did not meet the transparency criteria is equivalent to allowing them to discard expert accredited research at whim. The commenter contends that the proposed policy has grave potential for the capricious disregard of facts and subsequent policies which are ignorant of reality and harmful to the public at large. (1481)
 The EPA's attempt to prescribe how a scientist should assess the shape of concentration-response relationships seems designed more to tie EPA's hands and prevent it from considering studies most scientists think are relevant, rather than to serve any public health purpose. (1973) 
 Commenter asserts that the country's public health protections could be hampered by allowing the agency's political leadership to select studies that benefit its agenda and ignore those that don't, opening the door to industry interests and secrecy. The commenter adds that, with limited access to the most recent scientific research, how will the EPA make the best decisions on behalf of the American people consistent with its regulations? (2907)
 While the goal of transparency in how studies are conducted and the ability to reproduce scientific results are important, the commenter notes that it can offer a politically motivated Administration a convenient excuse for eliminating or ignoring scientific studies that may go against the wishes of a powerful industry group. The commenter adds that all one has to do is demand the data sets be handed over for "further scrutiny" or demand that the study be repeated before basing a regulation on the study in question. The commenter notes that since the founding of the EPA, independent scientific research has been the foundational basis of your mission. The commenter suggests that the science be allowed to speak the truth and that the EPA should hear from all scientific studies, not just the ones the industry wants. (4933)
 It is the scientific community, with public comments, that are best able to determine a study's value to the EPA's policy making process. (8274)
Response: The EPA disagrees that the EPA's political appointees will determine how studies are conducted, reproduced, or used. As described in the preamble to the final rule, EPA staff will continue to use best practices to review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information. The EPA will use existing factors (including soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review) to evaluate study quality. When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence (consistent with Agency guidelines on hazard identification and dose-response assessment) to support a relationship between exposure and effect, the EPA will identify a subset of those studies for use in characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect. This will be based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both will be identified as pivotal science. Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. In compliance with the rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or propriety information, available through restricted access in a manner sufficient for independent validation. Including this review of data availability for pivotal science is critical to the EPA's progress toward increased transparency and providing increased opportunity for scientific reanalysis and scrutiny by independent third parties. This approach will result in final significant regulatory actions and influential scientific information that are based on the highest quality, most relevant studies and ensure adequate human health and environmental protections, while maximizing transparency and maintaining consistency with the statutes the EPA administers. When the dose-response data underlying pivotal science are not publicly available in a manner sufficient for independent validation, the EPA will give less consideration to the studies' evidence, findings, and conclusions, unless the Administrator grants an exemption request. The rule also provides factors for consideration that balance some of the important scientific considerations an EPA analyst may want to consider in addition to data availability and that are particularly relevant to the stage of the analysis where dose-response data are used. EPA analysts will continue to play an important role in evaluating all relevant scientific studies and determining their quality using existing guidelines. 
The EPA also does not agree with commenters regarding the role of the Administrator in determining whether to grant an exemption under 40 CFR 30.9 and finds that the Administrator is the appropriate decision maker in this context. To ensure that the Administrator's decision is appropriately transparent, the EPA has included a provision in 40 CFR 30.9 of the final rule that requires the Agency to document the rationale for any exemptions granted by the Administrator in the final significant regulatory action or final influential scientific information.
The EPA agrees with commenters about the importance of data sharing and analysis by subject matter experts. When data are widely available, researchers can validate research results and help identify and correct unintended errors, as well as reanalyze the data for new and different purposes, examine novel questions, provide new scientific insights, and improve model development.
In response to public comments, the EPA clarified and modified many aspects of the proposed rule in the 2020 supplemental notice of proposed rulemaking and final rule, including that the rule requirements are now focused on dose-response data and do not apply to dose-response models. The EPA would also like to clarify that the final rule does not require any researcher or other outside entity to provide data or models to the EPA, nor does it require the EPA to disclose or host data. Instead, it governs internal agency procedures for determining the consideration to afford to pivotal science according to factors that include the availability of underlying dose-response data. 
2.1.3.8 History of Suppressing Science Puts the Credibility of the Goal of Transparency in Doubt
Comment: Commenter (9227) suggests that EPA, under the Trump Administration, has a history of suppressing science and transparency and purported justifications for the proposal. Similarly, commenter (6937) suggests that the current EPA upper Administration (and industry as well) is not usually a champion of transparency. Therefore, the commenter contends that the call for transparency regulation is hypocritical and lends credibility to those that doubt whether transparency and replication are the true goals of the proposal. Transparency, according to the commenter, so lauded in this regulatory proposal, has been decimated by the current Administration  -  with extensive restrictions and changes to climate change websites, extreme efforts at secrecy and avoiding public scrutiny,,, and efforts to end disclosure of chemicals used in hydraulic fracturing.
Commenter (9227) provides that a FOIA request submitted by E&E News uncovered a document emailed by former EPA official David Schnare laying out a strategy to overturn the 2009 Greenhouse Gas Endangerment Finding. In the document, the commenter notes that one of the steps contemplated as part of the reconsideration included EPA only relying "on information, data and studies where the original data upon which assessment is based is available to the public. . . . EPA would not rely on any study whose authors refuse to provide the underlying data, including computer code used to evaluate and analyze the data." This, according to the commenter, is just one example among numerous others that this proceeding is not intended to increase transparency, but rather aimed at weakening EPA standards that the current Administration disapproves of, despite their grounding in robust scientific evidence.
The commenter (9227) contends that EPA's non-transparent approach to this rulemaking, as well as other Agency actions, underscore that the proposal was not offered in good faith. The commenter provides that the Agency has removed thousands of webpages from its website, limited public and press access to Agency events, and withheld key data underlying rulemakings and proceedings. According to the commenter, these practices cast doubt on EPA's proffered justifications of transparency and accountability. The commenter cites examples related to the stay of the Oil and Natural Gas Sector: Emissions Standards for New, Reconstructed, and Modified Sources rule, and the removal and modification of climate related and Clean Power Plan materials, webpages and files. The commenter states that the Administration has also not rigorously pursued its purported goal of transparency in other contexts by limiting public and press access to Agency events and withholding key data underlying several recent rulemaking proceedings. For example, the commenter notes that, at the event where former Administrator Pruitt announced the proposal, reporters were not invited to attend. The commenter notes that documents received in response to a Sierra Club FOIA request to the EPA reveal that the Administrator had requested press access and advertisement to the public be limited for other events (and cites examples):
 For his speaking engagement at a Federalist Society event in March 2017, Pruitt's scheduling director asked that organizers not advertise to press directly and directed organized to tell media that the event "is not open to press and is off the record."
 Emails also demonstrate that the Agency worked with a public relations firm to devise a plan to promote positive comments and censor negative comments on media from the Administrator's facility visits.
According to the commenter (6125), EPA has not even applied the principles of its proposal to rulemaking when that would get in the way of its central agenda of providing relief to industry. The commenter notes that the EPA's commitment to "transparency" seems to be a recent invention that it has not followed in recent rulemaking proposals. The commenter states that just a few months before issuing this proposal, when EPA proposed to rescind the rule restricting glider trucks (trucks with old diesel engines placed in new truck chassis), the agency uncritically summarized purported emission test results of glider trucks. The commenter provides that the study it used was not peer reviewed; did not disclose its methodology (it turned out the critical observation was done by `visual observation', i.e. eyeballing!), and EPA did not examine it for any indication of reliability. See 82 FR 53444 (Nov. 16, 2017). In contrast, the commenter adds that the EPA's National Vehicle and Fuel Emissions Laboratory (NVFEL) laboratory conducted its own study using, "standardized test methods, consistent data evaluation procedures, and good laboratory practices to ensure transparent, understandable, and reproducible scientific assessments." 83 FR 18770. The commenter notes that the EPA did not even consider the NVFEL study in its proposal. Similarly, commenter (9227) states that the EPA has failed to provide the public with access to data in key rulemakings and proceedings. For example, in EPA's rulemaking to repeal emissions requirements for glider vehicles, engines, and kits, commenced in November 2017, the Agency failed to release the underlying reports and data before the public comment period closed. At this date, EPA still has not released data used in a key study cited in the Agency's proposal. Commenter (2427) expresses concern that, given EPA's recent proposals to restrict the use of certain scientific studies in regulatory decision-making and to allow the widespread sale of "glider kit" trucks which lack modern emission controls shows the Agency's disregard for objective information and the scientific process, and its move to rely on analysis that supports particular outcomes. 
Commenter (9227) adds that, in EPA's stay of the Oil and Natural Gas Sector: Emissions Standards for New, Reconstructed, and Modified Sources, EPA failed to disclose directly relevant evidence for the basis of revision of the standards consisting of industry compliance reports. Despite the fact that these compliance reports were in the agency's possession and comprised of public documents containing factual data that should have been available for public inspection, EPA has to date still not released all of the compliance reports in its possession. The commenter provides that, in August 2017, EDF received information pursuant a FOIA request revealing that more than 1,900 climate-related webpages and files on EPA's website were removed or modified. Many of the removed and modified pages were related to climate change science and impacts, such as "Climate Impact on Health Through Life Stages," "Climate Change Science," and "Methane and Black Carbon Impacts on the Arctic: Communicating the Science."
In January 2018, commenter (9227) states that the EDF received additional responsive records to another FOIA request demonstrating that former Administrator Pruitt directed the removal of many climate change science, impacts, and resources pages as well as all material related to the Clean Power Plan on EPA.gov. At the same time, the commenter states that the EPA was soliciting comments on its proposal to repeal the Clean Power Plan. The commenter suggests that the removal of webpages related to climate and Clean Power Plan topics from the EPA website restricted the public's ability to formulate informed comments throughout the rulemaking process. Thus, according to the commenter, the public lacked the same "access to data and influential scientific information used to inform federal regulation" which EPA claims to observe in its proposal.
Commenter (9227) asserts that, in the words of the proposal, EPA acted in contravention of its goals of "better informing the public," "enhancing the public's ability to understand and meaningfully participate in the regulatory process," and "ensur[ing] that its decision-making is marked by independence, transparency, clarity, and reproducibility" as it proceeded through rulemakings that "will affect the public" and where "the public is likely to bear the cost of compliance."
Response: This rule does not target specific EPA rulemakings and only applies prospectively. The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information. 
As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider the availability of underlying dose-response data when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available for independent validation. Including this review of dose-response data availability for pivotal science is critical to the EPA's progress toward increased transparency and providing increased opportunity for scientific reanalysis and scrutiny by independent third parties. The approach in this rule will result in significant regulatory actions and influential scientific information that are based on the highest quality, most relevant studies and ensure adequate human health and environmental protections, while maximizing transparency and maintaining consistency with the statutes the EPA administers.
2.1.3.9 Societal Bias
Comment: Commenter (0567) states that pivotal regulatory science as currently performed often includes societal objectives. The commenter states that some scientific assessments state that to be "protective," authors use "conservative" assumptions. The commenter states that the inclusion of societal objectives ideology, religious, or other beliefs, or any other non-scientific goal would cause damage to scientific assessment. The commenter states that to be protective is the responsibility of policy makers rather than of scientists. 
Response: Recognizing that assumptions are an inherent part of scientific assessments, this rule seeks to provide greater transparency into the critical assumptions made in an assessment and the impact of those assumptions on the assessment conclusions. As stated in 40 CFR 30.5(e), "The EPA shall also describe critical assumptions and methods used in its dose-response assessment and shall characterize the variability and uncertainty of the assessment. The EPA shall evaluate the appropriateness of using default assumptions on a case-by-case basis. The EPA shall clearly explain the scientific basis for critical assumptions used in the dose-response assessment that the EPA relied on for the significant regulatory action or influential scientific information."
2.1.3.10 Science and Transparency of Decision-making is the Most Unbiased Way to Collect Data
Comment: Commenter (1559) contends that science and transparency of decision-making is the most unbiased way to collect data and to analyze it in order to make decisions and to regulate so that the health of Americans is properly protected as the government is obligated to do by the Constitution's right "to life, liberty, and the pursuit of happiness." 
Response: The EPA agrees that transparency is beneficial to scientific assessments and the public. The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety.
2.1.4 Environmental and Health Protections Would be Decreased
Comment: Commenters (0017, 0055, 0559, 0945, 1012, 1346, 1461, 1476, 1481, 1512, 1536, 1554, 1559, 1753, 1793, 1813, 2002, 2171, 2425, 2495, 2787, 2902, 2907, 2910, 2977, 3131, 3132, 4542, 4596, 4828, 4838, 4839, 4840, 4842, 4843, 4844, 4872, 4877, 4878, 4881, 4933, 5011, 5020, 5021, 5041, 5113, 5115, 5120, 5122, 5172, 5178, 5183, 5184, 5418, 5979, 6046, 6103, 6109, 6111, 6114, 6116, 6118, 6121, 6122, 6123, 6125, 6127, 6130, 6133, 6137, 6144, 6145, 6153, 6156, 6158, 6159, 6166, 6168, 6169, 6173, 6174, 6181, 6185, 6188, 6351, 6355, 6358, 6368, 6373, 6446, 6450, 6464, 6696, 6860, 6868, 6869, 6871, 6874, 6876, 6877, 6879, 6880, 6882, 6884, 6893, 6894, 6897, 6898, 6903, 6915, 6919, 6920, 6924, 6928, 6932, 6939, 6940, 8274, 8276, 8277, 8281-PH5, 8281-PH10, 8281-PH15, 8281-PH20, 8281-PH24, 8281-PH29, 8281-PH34, 8281-PH63, 8281-PH84, 8317, 9225, 9227) express concern that the proposed rule/policy to limit the research and data that EPA can use to make informed policy decisions under major public health and environmental laws will reduce environmental and health protections. Commenter (1346) requests that EPA please rewrite this rule, to make it clear that the health and well-being of people and the environment comes first in these decisions. Several of these commenters also express concern that the negative health impacts of inadequate environmental regulation will fall disproportionately on children, the elderly, pregnant women/fetuses, and marginalized populations, particularly low-income black and Latino communities. Several of the commenters provide specific examples of health protections that could be rolled-back (e.g., water quality criteria, air toxic regulations, drinking water standards, pesticide regulations, chemical use bans/restrictions/control, criteria pollutant standards) or not be pursued or possible (e.g., fracking health exposure/effects studies, pesticide and Parkinson's disease research, greenhouse gas emissions effects on climate change and public health) as a result of the proposal and that the proposal should be withdrawn. Specific comments include:
 Commenters (2002, 6932) assert that, because the proposed rule is retroactive, it is an attempt to reverse much-needed protections. Commenter (2002) states that, since the proposed rule is retroactive, it could lead to the dismantling of many of EPA's most important and effective health protections, including the CAA, rules that set limits on lead in drinking water, and bans on some uses of neurotoxic pesticides, as they all rely on epidemiological studies.
 Commenter (6879) states that, although the proposed regulation has a particular emphasis on undermining the application of science in regulations focused on human health, it sets a dangerous precedent for ignoring and discounting legitimate and sound science in all other areas of EPA's responsibility, including the protection and restoration of natural habitats vital for wildlife conservation.
 While EPA's previous science-based regulatory process has saved countless lives, prevented chronic diseases, and improved the quality of the air pregnant women and children breathe, the water they drink, and the world in which they will grow, the proposed rule will instead prevent evidence-based policies from being enacted and will contravene EPA's mission to ensure that American pregnant women, children, and families have clean air, land and water, and we strongly urge you not to move forward with it. (2425)
 EPA's efforts to reduce lead and methylmercury exposure for women, pregnant people, and children might not have been as successful if the agency had used the overly stringent definition of transparency it now proposes. The commenter states that programs that might not exist, or exist in less-effective forms, include those to reduce lead exposure from paint, gasoline, and drinking water; limit mercury emissions; and advise pregnant people to steer away from consuming fish species that often contain high levels of methylmercury. (6893)
 Commenter (6915) asserts that the requirements of the proposed rule would lead EPA to adopt less-protective standards across many regulatory programs. The commenter states that the proposed rule would allow for the use of less protective dose response models and assumptions in human health risk assessment. The commenter states that it would also preclude consideration of scientifically valid human and animal studies reporting sensitive and relevant toxic effects based on unjustified requirements for public availability of data, and instead favor consideration of studies that do not assess the most sensitive and relevant health effects endpoints. The commenter states that, for example, while the NAAQS review process builds on the administrative record from prior rulemakings, under the proposed rule, EPA may refuse to consider these studies and others because they rely on data pertaining to the personal medical histories of participants that cannot, by the studies' terms or by law, be divulged. The commenter states that restricting the use of such studies would significantly undermine current and future NAAQS reviews.
Commenter (9227) provides that leading experts in public health, science, and environmental policy agree that the proposed rule would have far-reaching, detrimental impacts on public health. The commenter contends that, by limiting the scientific studies that EPA may consider, the proposed rule would lead to less effective environmental policies and weaker public health protections. The commenter notes that experts have said the following:
 "[The proposed rule] will threaten the lives of real people."  -  Commissioners of the Minnesota Pollution Control Agency and Department of Health.
 "If the proposed rule is approved, science will be practically eliminated from all decisionmaking processes. Regulation would then depend uniquely on opinion and whim."  -  John P. A. Ioannidis, C.F. Rehnborg Chair in Disease Prevention at Stanford University.
 "It does not strengthen policies based on scientific evidence to limit the scientific evidence that can inform them. . . . Excluding relevant studies simply because they do not meet rigid transparency standards will adversely affect decision-making processes."  -  Editors of science family of journals, Nature, Public Library of Science journals, Proceedings of the National Academic of Sciences, and Cell.
 "Without access to the restricted data, regulatory programs could become more or less stringent than they otherwise would be, with consequences for both regulatory costs and benefits. . . . [the proposed rule] could have the effect of removing legal, ethical, and peer-reviewed studies of health effects as sources to support the agency's regulatory efforts."  -  Members of the SAB.
 "[The proposed rule] would prevent the best science from informing policy decisions and result in weaker health safeguards."  -  Harold P. Wimmer, National President and CEO of the American Lung Association.
 "If [the proposed rule] had been in effect 20 years ago, the nation might have forgone programs that are preventing over 50,000 premature deaths each year."  -  Environmental Protection Network.
 "[The proposed rule] would greatly weaken EPA's ability to comprehensively consider the scientific evidence across the full array of health effects studies. This would negatively impact EPA public protections that reduce levels of lead, harmful chemicals, and fine particle pollution, among others."  -  985 scientists in a joint letter to Administrator Pruitt.
 "[The proposed rule] would severely hamstring the agency when it comes to developing and enforcing public health rules by limiting the kinds of research the EPA can use in crafting rules."  -  Union of Concerned Scientists.
 "[Administrator] Pruitt is moving to rid the EPA of the science needed for effective regulation. . . . Its potential impact goes well beyond the EPA's regulatory effectiveness to the underlying role of science in American society."  -  Dr. Bernard Goldstein, Professor Emeritus of Environmental and Occupational Health at the University of Pittsburgh and former EPA Assistant Administrator for Research and Development. 
Additionally, commenter (9227) states that when the U.S. House of Representatives passed similar legislation in 2017, H.R. 1430, numerous professional organizations raised concerns about the implications of the proposed legislation. The commenter adds that the Environmental Data & Governance Institute (EDGI) found that:
      A bill that provided genuine provisions for public data access and usability, and did not focus on mandating the reproducibility of studies and on prohibiting the use of any data that could not be divulged to the general public in its entirety, would not be expected to hamper the EPA in a significant way. EDGI's analysis of H.R. 1430 shows that it does not achieve its stated goals. Instead, our research shows that H.R. 1430 would not promote transparency and that its passage would instead block the EPA from using the data it needs to fulfill its mission of protecting public health and the environment.
Commenter (6137) provides several examples where the Proposed Rule's restriction on the science could and would potentially undercut the support provided for environmental programs and resulting adverse consequences to public health protections. The commenter notes that, in a world in which EPA cannot consider critical studies demonstrating the deleterious impacts of toxic chemicals, pollutants, and pesticides when developing rules governing exposure levels, acceptable uses, and safety measures, there will be little to no available evidence to support the imposition of public health protections. Absent such evidence, according to the commenter, the EPA will be unable to implement rules that protect our air and water from harmful pollution, our farmworkers from toxic pesticides, and the public from overall exposure to chemicals, as EPA will have no science to point to as justification for such measures. Simply put, the commenter asserts that the removal of this science from consideration in the rulemaking process will cause the very foundation upon which many of our public health standards depend to crumble.
Commenter (6137) notes that not only is EPA's elimination of its use of critical human health research and studies that serve to protect and promote public health and the environment unlawful, but it is also discriminatory. The commenter asserts that the Proposed Rule has a disproportionately deleterious impact on low-income communities and minority communities  -  the overburdened populations that benefit most from the epidemiological studies used to set limits on air and water pollutants and to set safe exposure levels for pesticides and other toxics. According to the commenter, by removing this critical category of studies from consideration when setting limits on pollutant and toxic exposure as well as standards for safe air and water, EPA is turning a blind eye to accessible and illuminating evidence that provides much-needed safeguards to the vulnerable communities most impacted by the laws EPA is charged with enforcing. These communities will disproportionately suffer as a result.
Commenter (6137) provides several examples of public health studies that indicate that communities of color and economically disadvantaged populations are disproportionately impacted by pollution. The commenter provides that since the 1987 landmark report Toxic Wastes and Race in the United States, studies have confirmed again and again that communities of color and also economically disadvantaged populations are disproportionately located near toxic waste sites and other sources of pollution. And research has also found that overall air pollution exposure is more strongly concentrated in communities of color. The commenter adds that pesticide exposure likewise disproportionately impacts low-income and minority communities: farmworkers tend to be poor, and the vast majority are of Latin American origin, and they have a higher incidence of pesticide poisoning than other workers. For example, the commenter notes that in California, the counties with the greatest use of the highly toxic pesticide chlorpyrifos are the counties with the highest poverty levels and largest Latino populations. The commenter states that, in April 2014, the California Department of Public Health issued a report showing that thousands of children, disproportionately people of color, attend school in close proximity to pesticide use. Overall, according to the commenter, the disproportionate burden of environmental exposure among these low-income communities and communities of color cannot be disputed. 
Commenter (6137) expresses that, fundamental to the research revealing the disproportionate impact of environmental harms on low-income and minority communities and the need to set appropriate standards is epidemiological data. The commenter states that this category of science that EPA wants to eliminate has been foundational to establishing disparate harms from a variety of toxics in air, water, pesticides, and other environmental sources. For example:
 Epidemiological studies have shown that there is a persistent disparity in the blood lead levels measured in children of color compared to white children. These studies revealed that in 2011-2012, the mean level was almost 40 percent higher in black children 1-5 years old than in white children of the same age.
 While studies have shown that the mean blood lead level in black children is much lower than it was several decades ago, epidemiological studies have shown that even low levels of lead in the blood are harmful.
 A 2011 peer-reviewed epidemiological study found that urine samples of children ages 6 to 24 months were more likely to have six organophosphate metabolites the closer the child lived to a pesticide application site.
 Longitudinal cohort epidemiologic studies have shown that even low levels of exposure to the highly toxic pesticide chlorpyrifos can disrupt brain development in prenatally exposed children, leading to developmental delays, lower IQ, learning disabilities, and ADHD-like behaviors. 
 Epidemiological studies demonstrate that farmworkers have a higher rate of pesticide poisoning than any other workers.
Thus, the commenter (6137) contends that it is clear that epidemiological studies have provided a consistent source of reliable data that has been critical to demonstrating disproportionate exposure to toxic chemicals. The commenter states that these studies in turn have been pivotal to setting air, water, and pesticide standards necessary to protect low-income and minority populations from harmful levels of exposure.
The commenter (6137) asserts that the EPA's proposal to eliminate use of epidemiological studies will have a disparate impact on the most overburdened and vulnerable communities, eliminating the very source of data relied upon to provide protections crucial to their health and wellbeing. The commenter contends that this Proposed Rule will perpetuate the environmental injustices that low-income and minority communities already face, as it will remove the primary tool used to study and address the inequitable environmental harms suffered by these populations.
Commenter (6919) expresses concern that the proposed EPA rule leads to the questioning of studies that reveal negative effects of environmental exposures and allow policymakers to take protective action on behalf of those at greatest risk. The commenter notes that, by undermining existing evidence, the EPA would set back the clock on fighting environmental injustices that are taking place in Colorado. The commenter provides that, without evidence from studies like those presently being conducted, in which personal health data is gathered and protected when the data is aggregated, we cannot protect those who are at greatest risk and have lived with the burden of environmental exposures for decades while the current evidence base was built up. For example, the commenter states that a Colorado School of Public Health study shows the negative health effects of exposures on populations living close to oil and gas operations, adding that, in parts of Colorado these are some of the only affordable places to live.
Commenter (6939) asserts that the United States has a positive obligation to ensure equality of rights.
Commenters (1481, 6168) note that decreased health protections will lead to increased health costs to the public. Commenter (2907) also notes that there could also be a loss of productivity due to air pollution created illnesses, which, according to the commenter, shows the folly of focusing only on business cost savings in the energy sector when considering environmental regulation. The commenter states that all Baltimore City area employers suffer financial consequences when productive workers are absent due to illnesses caused by air pollution. The commenter adds that future generations suffer when they are under-educated due to missed days of school.
Commenter (5977) asserts that the main consideration for all EPA's regulations should be the costs on human and non-human communities in the United States and all over the planet and not the costs of compliance for businesses, real or perceived. The commenter states that the very need for EPA's regulations is caused by the negative externalities generated by businesses. 
Response: The EPA does not agree that this rulemaking is counter to the Agency's mission or that the implementation of the rule would result in regulatory actions that are less protective. The EPA is committed to its mission of protecting human health and the environment through sound policy decisions that are informed by robust scientific and technical research. Because of the potential impact of the EPA's significant regulatory actions and influential scientific information on American lives and livelihoods, the American people deserve environmental decisions and policies that are based on the best scientific information. Only through continuous improvement to its procedures, especially those focused on transparency, can the EPA fully demonstrate that commitment. The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
The EPA agrees that it is important to consider the totality of relevant, quality research. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider the availability of underlying dose-response data when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available for independent validation. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating final significant regulatory actions and finalizing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations. Including this review of dose-response data availability for pivotal science is critical to the EPA's progress toward increased transparency and providing increased opportunity for scientific reanalysis and review by independent third parties. This approach will result in significant regulatory actions and influential scientific information that are based on the highest quality, most relevant studies and ensure adequate human health and environmental protections, while maximizing transparency and maintaining consistency with the statutes the EPA administers.
The EPA would like to clarify that this final rule will be implemented prospectively and rules that have already been finalized will not be subject to this rule's requirements.
2.1.5 EPA Would be at Odds with Other Parts of the Federal Government
 2.1.5.1 Impact on Other Agencies by Limiting the Use of Best Available Science
Comment: Commenters (1493, 2335, 2430, 2909, 5022, 6111, 6194, 6915, 9227) express concern that the proposed rule could negatively impact other federal agencies. Commenter (9227) provides that other federal agencies routinely incorporate and rely upon EPA science assessments in their own efforts to carry out their mandates to protect human health and safety. As with statutorily required consultations, the commenter asserts that the Proposal utterly fails to acknowledge or consider what impacts restricting EPA's own use of dose-response studies would have on the work of these other agencies.
Commenter (2335) states that other government agencies use studies that rely on data from individual health records to develop policy and regulations. The commenter is concerned that moving forward with this proposed rule could undermine decisions made by other federal agencies based on similar studies and could invite challenges to other federal agencies' policies. 
Commenter (6915) states that, if EPA adopts a deficient standard due to the arbitrary exclusion of available scientific information, other federal agencies relying on EPA standards as a basis for action would be affected, as would be the states that interface with those federal programs.
Commenters (2430, 2909, 5022, 6194, 9227) state that EPA's scientific research policies and findings influence not only EPA's work, but also the work of other agencies charged with protecting our health and environment. The commenters further states that the National Park Service (NPS), Occupational Safety and Health Administration (OSHA), Department of Housing and Urban Development (HUD), and FDA all rely on the data and findings of the EPA to fulfill their duties. The commenters contend that these agencies cannot effectively protect our health and environment if EPA unreasonably limits the scientific record, as the proposal would. Commenter (2430) states that moving forward with this proposed rule would undermine, and invite challenges to, decisions made by other federal agencies based on similar kinds of studies.
Commenter (6194) states that, in order for OSHA to promulgate standards under OSHA, the agency is required to demonstrate the existence of significant risks to employees using "best available evidence." The commenter states that, currently, OSHA fulfills this obligation, in part, with information provided by EPA's TSCA Section 9(a) reports. The commenter expresses concern that, if the Proposal results in the exclusion of confidential data, then OSHA will likely be forced to either gather supplementary evidence or ignore the reports altogether. The commenter contends that such an outcome will not only impede OSHA in its regulatory functions but could also burden EPA with additional responsibilities. 
Commenter (9227) states that HUD is required by statute to assist EPA in assessing the extent of radon contamination in the United States and developing measures to avoid and reduce radon contamination. The commenter states that HUD has also developed policies to require radon testing at properties receiving federal financing, which incorporate EPA radon standards. The commenter contends that, to the extent the proposed rule affects future EPA assessments of radon risks, the scope, cost and effectiveness of HUD radon programs could be affected as well.
Commenter (9227) states that FDA enforces tolerances established by EPA for pesticide chemical residues in human and animal foods under the FIFRA, including through a comprehensive pesticide residue monitoring program that tests for approximately 700 pesticide residues in both imported and domestic commodities. The commenter states that to the extent the proposed rule affects EPA's tolerances, the nature and effectiveness of FDA's own work to monitor for violations of those tolerances would be impacted. The commenter further states that FDA also regulates contaminants in bottled water under the Federal Food, Drug and Cosmetics Act. The commenter states that Section 410 of the Act requires that FDA regulations for bottled water be issued in coordination with the effective date of National Primary Drinking Water Regulations issued under the SDWA and be no less protective of public health than those standards. According to the commenter, if the proposed rule impedes EPA's work to establish drinking water standards, this may affect FDA's own ability to justify protective bottled water standards. The commenter states that, in certain circumstances, FDA also coordinates with EPA to provide the public with information and advice on environmental contaminants in foods, for example, in 2017 FDA and EPA released a joint advisory on mercury hazards associated with the consumption of fish and shellfish, which was based in part on EPA's assessment of the "reference dose" or level of exposure that a person can experience over a lifetime without a risk of harm. The commenter contends that the proposed rule could radically alter the science EPA would be permitted to consider in future such initiatives and frustrate the ability of FDA and other agencies to coordinate effectively with EPA to develop joint advice and information.
Response: This is a rule of internal procedure promulgated under the EPA's housekeeping authority. The rule does not impose requirements on parties external to the EPA, including state governments and other agencies within the federal government. External users of EPA scientific assessments can still use these assessments. There are also provisions in the final rule (e.g., requirements for the EPA to document critical assumptions and methods used in its dose-response assessment, studies considered pivotal science, the rationale for the level of consideration for studies in the significant regulatory action or influential scientific information, and any Administrator exemption) that will make it clear how the provisions of this rule may have impacted the significant regulatory action or influential scientific information and will help external users determine whether the product is fit for their purpose. The final rule also provides that the EPA will give greater consideration to pivotal science where the underlying dose-response data are available for independent validation, rather than providing for a categorical exclusion, so the EPA will not be limiting the scientific record. Federal agencies were  given the opportunity to provide comment on this rulemaking through the interagency review process associated with Executive Order 12866, but comments on that deliberative process are outside the scope of this rulemaking.
2.1.5.2 Coordination Among Agencies
Comment: Commenters (6194, 9227) express concern that the proposed rule would compromise coordination among federal agencies.
Commenter (6194) asserts that the proposed rule would compromise coordination requirements and data sharing agreements between the EPA and the Department of Agriculture, the Food & Drug Administration, and Agency for Toxic Substances and Disease Registry (ATSDR). The commenter asserts that such a blow to inter-agency information sharing would be devastating to regulatory progress throughout the federal government.
Commenter (9227) argues that the EPA arbitrarily failed to consider the implications of this proposal on interagency coordination and consultation given that other agencies normally rely on research potentially excluded by the Proposal. The commenter states that, in the numerous environmental statutes that EPA cites, there are dozens of provisions that require EPA to coordinate or consult with other Federal entities -- especially when implementing research programs and issuing information or guidelines. The commenter contends that the proposed rule could frustrate and impair this coordination and consultation, either by forcing EPA to ignore the science provided by other agencies or by severely restricting the science that EPA itself would be able to share with other agencies in these statutorily required processes. 
Commenter (9227) states that, in addition to statutorily required consultations, other federal agencies routinely incorporate and rely upon EPA science assessments in their own efforts to meet mandates to protect human health and safety. The commenter contends that, as with statutorily required consultations, the proposed rule fails to acknowledge or consider what impacts restricting EPA's own use of dose-response studies would have on the work of these other agencies. 
Response: The EPA disagrees that this rule would compromise interagency coordination. This is a rule of internal procedure promulgated under the EPA's housekeeping authority. The rule does not impose requirements on parties external to the EPA, including other agencies within the federal government. External users of EPA scientific assessments can still use these assessments. There are also provisions in the final rule (e.g., requirements for the EPA to document critical assumptions and methods used in its dose-response assessment, studies considered pivotal science, the rationale for the level of consideration for studies in the significant regulatory action or influential scientific information, and any Administrator exemption) that will make it clear how the provisions of this rule may have impacted the significant regulatory action or influential scientific information and will help external users determine whether the product is fit for their purpose. The EPA also disagrees that implementation of this rule would limit the scientific record. The final rule does not categorically exclude any studies from EPA's consideration, so the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations. Federal agencies were given the opportunity to provide comment on this rulemaking through the interagency review process associated with Executive Order 12866, but comments on that deliberative process are outside the scope of this rulemaking.
2.1.5.3 Counter to Policies of Other Agencies
Comment: Commenters (0671, 1973, 1796, 2746, 5170, 6117, 6125, 6152, 6158, 6896) suggest that the EPA proposal runs counter to policies that are currently being implemented by other agencies.
Commenter (0671) states that no other federal agency has such a broad standard as the one proposed by EPA. The commenter states that the EPA knows that IRBs will not allow such studies' data to be made public, so to require public data from these studies will make them invisible to the rulemaking process.
Commenters (1973, 1796, 5170, 6125) state the proposal would put EPA in direct opposition to the standards currently implemented by the Department of Health and Human Services (HHS). This includes the evidence considered on the health effects of smoking, high cholesterol, etc. by the Centers for Disease Control (CDC), the studies considered for issuing guidelines generated by the NIH, and the FDA's reliance on non-publicly available studies to approve drugs for use by people. HHS has demonstrated it is possible to reconcile the principles of access to, reliability of, and reproducibility of data while respecting the confidentiality of individual patient data.
Commenters (5170, 6117, 6125, 6158) state that the NIH and numerous other federal agencies have required public access for their funded research and they promote data sharing among researchers. The commenters state that none of these agencies require full public disclosure of confidential personal data from health studies and, consequently, EPA's proposed rule would not be consistent with other agencies. 
Commenters (6152, 6158) state that the proposed rule is not consistent with the principles of open science as articulated and adopted by leading U.S. scientific bodies including the NAS, NIH, and the CDC. The commenters contend that these bodies recommend that the agency implement existing, broadly accepted tools, including "systematic review methods" rather than arbitrarily excluding science based on rigid data disclosure requirements. 
Commenter (2746) expresses concern that that the proposed rule would limit EPA from basing policy on the best available science even though similar data restrictions do not apply to other areas of public policy, such as diplomacy, economy, housing, etc.
Commenter (6896) states that, while each federal agency has different requirements and protocols to ensure sensitive data held under their agency is appropriately protected, the proposed rule does not address these interagency complications. Commenter states that these include data collected and provided by government agencies under specific conditions that vary by agency and dataset and are meant to ensure fair, ethical, and scientific usage. Commenter provides examples from the Bureau of Labor Statistics (BLS), OSHA, the Department of Energy (DOE), the National Institute of Occupational Safety and Health (NIOSH), and CDC's National Center for Health Statistics (NCHS)'s Research Data Center.
Commenter (6125) states that the CDC and other Federal Agencies involved in reducing lead risk, would not be subject to the restrictions on use of science imposed by the proposal, and thus would continue to use the older studies as evidence, creating an unwarranted science practice and policy schism in the federal government. The commenter provides the following examples:
 The CDC has acknowledged that no safe level of lead in blood has been identified and has identified a "reference level" to define especially high-risk populations and geographic areas most in need of primary prevention. Even some of the studies that the Federal government's most respected lead advisory group (CDC's Advisory Committee on Childhood Lead Poisoning Prevention, or ACCLPP) used to develop background justification for the reference level could be precluded from EPA use under this policy. If some of those 92 plus references and studies fail to meet this policy, EPA could not rely on them in developing regulations, and thus could issue regulations that fail adequately to address CDC's high-risk exposures. Further, the additional requirements in the proposal could require that EPA re-evaluate the appropriateness of the CDC dose-response relationship and give "explicit consideration" to other dose-response models even if, as in this case, they have been vetted and rejected by CDC and its advisory committee. 
 Both EPA and the U.S. Department of Housing and Urban Development (HUD) regulate similar residential sources of lead (paint, soil, and dust) - EPA in private housing, and HUD in Federally supported housing, including Federally supported units in private buildings. This policy shift increases the potential for disparate treatment and public confusion stemming from those situations. The identical risks addressed by the two programs should be analyzed consistently. HUD has already acted to incorporate CDC's statement into its programs by making its policy on dust-lead clearance levels more stringent; is still considering its response. But under this proposal, EPA might not be able to rely on as wide a range of studies as would be available to the rest of the federal family, which could result in different risk assessments and regulatory outcomes for similar, and possibly even adjacent, dwellings. 
Response: As discussed in the preamble of the final rule, the transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency. The EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. For example, in 2002 the OMB released its Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies, which includes discussion of the importance of the reproducibility of analyses underlying influential information. The EPA's 2016 Plan to Increase Access to Results of EPA-Funded Scientific Research noted that "transparency is a core EPA value" and that increased availability of research data would accelerate scientific breakthroughs that support the Agency's mission and policymaking efforts. The EPA's Open Government Plan 5.0 also details the EPA's progress in implementing the tenets of the numerous data transparency initiatives in the federal government prior to 2018, including OMB M-10-06 , the Office of Science and Technology Policy Memorandum of February 22, 2013, and OMB M-13-13. In 2019, Congress passed the Foundations for Evidence-Based Policymaking Act of 2018 (or OPEN Government Data Act, Pub. L. 115-435) into law, which included requirements for federal agencies to prioritize making their data available to the public, and OMB has released additional guidance for implementing the act.
Further, as discussed in the final rule, underlying dose-response data are considered available if they can be accessed through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The EPA agrees with commenters that it is important to consider the totality of relevant, quality research. Therefore, the EPA did not finalize the 2018 proposal or the 2020 SNPRM proposed option that would have categorically required that the underlying data and models be available for independent validation in order for studies to be considered pivotal science. However, given that transparency is an important aspect of EPA's regulatory actions and assessments it should be an important consideration in how the Agency considers pivotal science. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science is unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
2.1.5.4 Different Federal Definitions of Science
Comment: Commenters (6125, 8913) express concern that the proposed rule would create different federal definitions of science.
Commenter (6125) states that the proposal on its face does not apply to other federal agencies or to states and tribes (or for that matter, even to EPA outside of major rulemaking). The commenter provides that the proposed rule does not acknowledge that this would result in two different federal definitions of science, nor does it address the potential confusion this would create. The commenter explains one of the definitions is applicable only to major rulemaking and the other for everything else; in other words there would be two different EPA definitions of science for actions addressing the same subject, the risk from the conduct, under the same statute. The commenter asserts that this is the very definition of arbitrary.The commenter contends that EPA has not attempted to explain how the different federal definitions of science would work, nor has it identified it as a source of potential problems for EPA or other agencies. 
Commenter (6125) asks: How would EPA address such a situation? The commenter notes that the proposal says nothing about this, despite the obvious potential for confusion. For example, the commenter asks, would a decision whether to enforce a regulation based on the constricted new rule on science be based on all relevant science, including data not considered in developing the rule in the first place? What about permitting, licensing and registration decisions? What about evaluating a grant proposal under traditional science, for activities governed by a regulation using censored science? The commenter states that none of these serious concerns are addressed in the proposal. The commenter suggests that, if the agency insists on attempting to promulgate this flawed rule, there should be no distinction made between application of its provisions to major rules and to all other regulatory actions with regard to the consideration of the science in the development of such decisions. The commenter states that, if it is appropriate for one category of rules, then it is appropriate for all of them.
Commenter (6125) states that both EPA and HUD regulate similar residential sources of lead (paint, soil, and dust) - EPA in private housing, and HUD in federally supported housing, including federally supported units in private buildings. The commenter asserts that the EPA's proposed policy shift increases the potential for disparate treatment and public confusion stemming from those situations. According to the commenter, the identical risks addressed by the two programs should be analyzed consistently. According to the commenter, HUD has already acted to incorporate CDC's statement into its programs by making its policy on dust-lead clearance levels more stringent. The commenter states that, under the proposed rule, EPA might not be able to rely on as wide a range of studies as would be available to the rest of the federal family, which could result in different risk assessments and regulatory outcomes for similar, and possibly even adjacent, dwellings. 
Commenter (8913) states that an ill-considered consequence of the proposal is the degree to which the rule would sow chaos in government and research domains. The commenter states that governmental record-keeping/information-sharing would be fractured and thrown into confusion. The commenter states that many other regulatory bodies would continue to follow law and rely on best available science. For example, according to the commenter, the FDA, NIH and CDC would be funding research, evaluating medical findings, and focusing on public health values quite differently from EPA. The commenter contends that state and local officials acting to protect the air and water of their regions and the health of the families and children in their communities would no longer be able to rely on EPA standards for guidance.
Commenter (1973) contends that the proposal would put EPA in direct opposition to the standards for considering evidence of health effects of smoking, high cholesterol, etc. by the CDC, of the studies considered for issuing guidelines generated by the NIH, and of the FDA reliance on non-publicly available studies to approve drugs for use by people.
Response: As discussed in the preamble of the final rule, the transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency. The EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. For example, in 2002 the OMB released its Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies, which includes discussion of the importance of the reproducibility of analyses underlying influential information. The EPA's 2016 Plan to Increase Access to Results of EPA-Funded Scientific Research noted that "transparency is a core EPA value" and that increased availability of research data would accelerate scientific breakthroughs that support the Agency's mission and policymaking efforts. The EPA's Open Government Plan 5.0 also details the EPA's progress in implementing the tenets of the numerous data transparency initiatives in the federal government prior to 2018, including OMB M-10-06 , the Office of Science and Technology Policy Memorandum of February 22, 2013, and OMB M-13-13. In 2019, Congress passed the Foundations for Evidence-Based Policymaking Act of 2018 (or OPEN Government Data Act, Pub. L. 115-435) into law, which included requirements for federal agencies to prioritize making their data available to the public, and OMB has released additional guidance for implementing the act.
This is a rule of internal procedure promulgated under the EPA's housekeeping authority. The rule does not impose requirements on parties external to the EPA, including other agencies within the federal government. External users of EPA scientific assessments can still use these assessments. There are also provisions in the final rule (e.g., requirements for the EPA to document critical assumptions and methods used in its dose-response assessment, studies considered pivotal science, the rationale for the level of consideration for studies in the significant regulatory action or influential scientific information, and any Administrator exemption) that will make it clear how the provisions of this rule may have impacted the significant regulatory action or influential scientific information and will help external users determine whether the product is fit for their purpose. The EPA also disagrees that implementation of this rule would limit the scientific record. The final rule does not categorically exclude any studies from EPA's consideration, so the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
2.1.6 EPA Would be at Odds with Other National and International Bodies
Comment: Commenters (1973, 4827, 4878, 6906) contend that the proposal would create an approach to the use of scientific evidence on the adverse health effects of air pollution by EPA that would be at odds with the World Health Organization (WHO), the Royal Society for Medicine, and other national review bodies and law. Several commenters (4840, 4878, 6906) express concern that this could result in the EPA being unable to use beneficial studies conducted in compliance with other national and international bodies. Specific comments include:
Commenter (4840) states that some of the findings on which our public health protections are based are the result of studies done across the globe. The commenter suggests that it is unlikely that Canadian or European scientists would turn over their raw data to a U.S. agency for "independent validation."
Commenter (4878) states that there is little currently known about air pollution and its impacts on the brain. The commenter provides that recent studies have linked particulate exposures to Parkinson's disease including a large study done in Denmark. The commenter states that this study used several thousand people with and without a current diagnosis of Parkinson's disease. Using extremely specific (within 5-50 meters of the front door) geo-coding to estimate participant's exposure to contaminants, the commenter notes that the study estimated that ambient air pollution from traffic increased risk of developing Parkinson's disease by 9 percent. The commenter adds that researchers found an increased risk of Parkinson's disease after exposure to particulate matter in studies from Taiwan and South Korea, as well. 
The commenter (4878) expresses that, in addition to concerns about the usefulness of data if enough information is redacted to protect privacy, these studies raise additional challenges because they were performed internationally. The commenter provides that, in the Danish study participants are protected by European Union (EU) law. Going forward, the commenter notes that an EU study's compliance with the proposed rule will need to be reconciled with the new General Data Protection Regulation, which is seen as more restrictive than HIPAA.The privacy directive raises many issues for science not discussed here, but it is clear that many studies that involve people located in the EU will have a difficult time both complying with the new directive and providing enough information to EPA to be considered.
The commenter (4878) contends that studies coming from other countries are vital to health determinations in the U.S. because people in other countries are exposed to chemicals at different rates than in the U.S. The commenter states that the ability to compare and contrast exposure to health outcome can be enlightening; average particulate matter concentrations in South Korea and China are several times higher than in the United States, making relatively subtle effects stand out more easily. The commenter asserts that studies done in other countries can also help researchers tease out whether an effect is dose or length of exposure dependent and the inability to review and use international research in determinations will virtually guarantee EPA is missing major findings and important data.
Commenter (6906) also adds that recognition of internationally available data reduces EPA's burden in performing evaluations and allows for the consideration of hazard and risk assessments from other jurisdictions for Risk Evaluations under TSCA (§§ 4 and 6). According to the commenter, it also reduces industry's burden by allowing industry to rely on and/or provide data across countries. The commenter asserts that this promotes a uniform international market by coming to a common understanding of risk.
Specifically, commenter (6906) states that Effects and Environmental Fate of Chemicals Access to hazard characterizations, exposure assessments and environmental fate analysis in the EU, Canada and other jurisdictions assists with risk evaluation and new chemical review as required under the TSCA (§§ 4 and 6). The commenter is concerned that the new rule could limit recognition of analysis conducted and information available in foreign jurisdictions.
Commenter (6906) notes that many manufacturers of paint, coatings, sealants and adhesives operate at an international scale, importing raw materials and exporting products to foreign jurisdictions. The commenter states that the U.S. exports of paint and coatings totaled $2.3 billion in 2017, the second-highest level in more than a decade. An analysis of exports to 20 EU member countries, out of 28 EU countries, including Germany, France, United Kingdom, Spain and Portugal shows exports of 86.8 million to EU members. Being the U.S.' most important trade partner, Canada consumed the largest amount of U.S. paint and coatings exports at 1.034 billion, accounting for 45% of all U.S. exports globally. 
Response: As discussed in the preamble of the final rule, the rule requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings. 
As discussed further in the preamble of the final rule, the EPA will continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies --  based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements or quantitative analyses of an EPA significant regulatory action or influential scientific information will be identified as pivotal science. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science is unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. The EPA provides these factors in 40 CFR 30.5(d) and they are discussed more fully in the preamble of the final rule.
Further, as discussed in the final rule, underlying dose-response data are considered available if they can be accessed through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. The EPA would also like to clarify that the rulemaking does not require the EPA to collect, store, or publicly disseminate data and models underlying pivotal science.
Comment: Commenter (6873) notes that, not only are there well-established requirements and procedures for protecting the privacy rights of research participants (45 CFR, Pub. L. 93-348, etc.), but that these rights are also encoded in the international Helsinki Declaration of the World Medical Association. The commenter contends that, coming up with new guidelines for releasing private data, even if the intent is to mask this data and make it available only to government scientists, will require review and approval by the U.S. (the FDA and other agencies) and the global research community.
Response: The EPA would like to clarify that this is a rule of internal procedures and does not direct or require any outside entity or EPA to establish data sharing mechanisms or create new guidelines for releasing data. Further, the rulemaking does not require the EPA to collect, store, or publicly disseminate data and models underlying pivotal science.
2.1.7 Proposed Rule is Flawed and Needs to Be Withdrawn/Redone with Feedback from the Scientific Community/Stakeholders
 2.1.7.1 Feedback from the Scientific Community/Stakeholders
Comment: Commenters (0055, 0057, 1742, 2907, 3644, 3971, 4594, 4831, 4894, 5165, 5181, 6115, 6119, 6125, 6126, 6144, 6373, 6894, 6904, 6924, 8272) urge the EPA to not implement the proposed rule and instead to consult with the stakeholders before issuing the rule, including scientists, medical researchers and health professionals, universities, hospitals, peer-reviewed journals/publishers, science advisory panels, and communities who participate in research studies. Commenters (2472, 2904, 4595, 4839, 5011, 5165, 6117, 6119, 6144, 6163, 6595, 6909, 6915, 8272, 9225) states that the EPA has failed to get adequate input from the scientific community in developing this proposed rule. Commenter (1006) suggests that EPA "take down" the proposal and come back with something that takes in mind the countless feedback that the scientific community has given in response to the Proposed Rule. Commenter (6158) states that the proposed rule appears to have been written by government officials who do not understand the scientific process and thus there is a glaring lack of analysis and rationale in the Federal Register announcement. In particular, the commenter adds that no formal scientific argument or policy rationale is made that would justify the need for this rule. 
Commenter (6117) asserts that it is inappropriate to establish major changes in research priorities through insertion of additional language in a proposed regulation ostensibly on another topic, as was done here, rather than to use the normal channels of consultation with stakeholders, advisory boards, and reference to prior commissioned studies from the NAS. 
Commenter (6144) states that it is clear both from the number of questions posed by the agency during its comment period about how the rule might be implemented and from the outcry of the scientific community that the agency did not consult with scientists or respected national scientist organizations as it crafted this rule. 
Commenter (6126) states that the EPA's proposed rule raises complex and multifaceted issues, and its implementation would likely have meaningful consequences for environmental regulation and, therefore, public health and the environment. The commenter expresses that they (BPC) are disappointed EPA did not engage in a more deliberate and robust effort to engage stakeholders while initially developing the proposed regulation. While the commenter appreciates EPA's efforts since publication of the proposed rule to gather public input, the lack of foresight to engage in meaningful public discourse about the proposed activities prior to publication is troubling. According to the commenter, the now partisan debate over this rule's motives and meaning could have been diminished if EPA had taken time to more meaningfully and transparently engage the public.
Commenter (6126) notes that, as EPA determines whether or how to move forward in developing this regulation, they encourage a much more robust and thorough consultative process. The commenter states that this process should include direct consultation and dialogue with key stakeholders and experts likely to be impacted by the issues of the regulation, including scientists, industry, academic institutions, states, other federal agencies, and EPA's relevant advisory boards. The commenter contends that such a process will likely necessitate a new notice of proposed rulemaking for public comment prior to developing a final rule. The commenter suggests that EPA engage in much more extensive public and interagency dialogue, then subsequently issue a new Notice of Proposed Rulemaking on data sharing and use issues. The commenter suggests that the EPA could either (1) withdraw the rule and issue a new Notice of Proposed Rulemaking informed by a robust dialogue or (2) issue a new Notice of Proposed Rulemaking that supersedes the proposed rule published on April 30, 2018. 
Commenters (4595, 5165) recommend that EPA withdraw this proposed rule until a more thorough examination by an independent scientific organization/scientific community can be completed. Commenter (4595) urges EPA to consider carefully and deliberately, in concert with the scientific community and other stakeholder communities, any policy changes that could diminish the important role of scientific evidence in helping to make decisions that impact the health of Americans. Commenter (5165) suggests that a regulation with such significant ramifications for EPA's science-based decision making should be thoroughly vetted by the scientific community and other key stakeholders, including the state and local air agencies that rely on the scientific integrity of EPA's regulations to protect public health and the environment from the harmful effects of air pollution. Commenter (5165) adds that EPA should also consult from the earliest stages with the state and local agencies that are responsible for implementing our nation's environmental laws.
Commenter (5181) expresses that, given the broad scope and potential effect of the proposed rule, they recommend that an additional step be added for EPA to hold discussions with impacted states and tribes and other stakeholders to provide supplemental information to the current proposal. The commenter also requests that EPA provide a supplement to the rule to explain more fully how the proposed rule would increase transparency without delaying decision-making or excluding consideration of traditionally acceptable data and publications, given the numerous federal provisions already in place to achieve the goal of ensuring transparency.
Commenter (3971) states that the proposed approach, if carefully designed with input from all stakeholders (which will require a series of science advisory panels), can be brought into rulemaking. The commenter states that possible benefits of bringing to the table both classic toxicologists (who embrace historical in-vivo, whole animal, testing), as well as "modern era" toxicologists and physicians (who are well versed in predictive structure activity models, in-silico models, and low-dose modeling) could take rulemaking to a new level of sophistication. The commenter states that doing so would allow all reasonable scientific approaches to be given serious consideration when working toward a final recommendation for a particular conclusion. The commenter states that NAS and numerous respected scientific bodies, as well as trade associations, have made many proposals as to how this approach could be carried out, and these are well documented within EPA files. The commenter states that one unfortunate aspect of this proposed regulation is that the possible published papers that may help guide the Agency are not listed in the appendix. 
Typically, according to the commenter (6125), these actions rely heavily on work done by the career staff in ORD and/or program offices and consultation with the SAB, the FIFRA Science Advisory Panel (SAP), and/or the NAS and others before proposing a regulation. The commenter notes that none of this was done here. The commenter states that multiple reports based on emails and other documents obtained by the Union of Concerned Scientists (UCS) indicate that the idea of developing a policy memorandum or a rule based on legislation developed by Congressman Lamar Smith of the House Science, Space, and Technology Committee began in January 2018. Serious work on drafts took place during February under some deadline pressure, as evidenced by email traffic mainly involving EPA political staff. The commenter provides that the Administrator's first public mention of the effort appeared in a March 19 story in the Daily Caller, which was linked in an official EPA news release.
Commenter (6125) contends that the agency was in such a rush to judgment that it did not provide a role for its own career scientific and science/policy experts in crafting the policy/regulation or assessing potential unanticipated impacts, never included the rule in its regulatory agenda, did not follow its guidance for how to develop agency rules, did not notify or consult with or request a review of the proposal by its SAB, SAP, or the Department of Agriculture (USDA) as required by law, did not solicit the advice of the NAS on provisions that would change dose-response models used in risk assessment from those previously recommended by the NAS.
The commenter (6125) notes that the EPA initially allowed only a 30-day comment period, which would close one day before a long-scheduled meeting of the full SAB. According to the commenter, this rushed and largely secret process, antithetical to any standard of reasoned consideration or transparency, shows that EPA is not really concerned about transparency in public policy, much less in science.
Commenter (5022) states that the Proposal violates fundamental tenets of transparent rulemaking. The commenter suggests that the EPA apparently failed to consult with relevant stakeholders such as scientific, research, or health professional associations, did not consult with other federal agencies, and did not even make the proposed rule available to its own SAP for review. 
Similarly, commenter (6163) notes that the proposed rule was published without input from internal or external scientific experts, including the EPA SAB and the NASEM. According to the commenter, as a policy that would significantly impact the use of science at the agency and the clear solicitation of input about process and data management infrastructure included in the notice, experts in science and data management would have greatly informed the agency's actions.
Commenter (4831) states that they (The National Academies) have carried out numerous studies that advise EPA on the scientific bases of regulatory decisions related to human health and the environment. The commenter notes that examples of relevant reports that advise on dose-response analysis and models, as well as how to perform literature-based reviews, include:
 Application of Systematic Review Methods in an Overall Strategy for Evaluating Low-Dose Toxicity from Endocrine Active Chemicals (NASEM, 2017a; ),
 Review of EPA's IRIS Process (NRC, 2014; http://dels.nas.edu/Report/Review-Integrated-Risk/18764),
 Critical Aspects of EPA's IRIS Assessment of Inorganic Arsenic: Interim Report (NRC, 2013; http://dels.nas.edu/Report/Critical-Aspects-IRIS/18594),
 Review of the Environmental Protection Agency's Draft IRIS Assessment of Formaldehyde (NRC, 2011; http://dels.nas.edu/Report/Review-Environmental-Protection-Agency/13142),
 Finding What Works in Health Care: Standards for Systematic Reviews (IOM, 2011; http://www.nationalacademies.org/hmd/Reports/2011/Finding-What-Works-in-Health-Care-Standards-for-Systematic-Reviews.aspx), 
 Science and Decisions: Advancing Risk Assessment (NRC, 2009; http://dels.nas.edu/Report/Science-Decisions-Advancing-Risk-Assessment/12209), and
 Models in Environmental Regulatory Decision Making (NRC, 2007; http://dels.nas.edu/Report/Models-Environmental-Regulatory-Decision-Making/11972). 
According to the commenter (4831), these reports encourage EPA to consider all available science in the rule-making process and provide guidance about how the agency could be more transparent in describing how evidence is gathered and evaluated. Specifically, the commenter suggests that systematic-review methods be adopted to ensure objectivity, rigor, and transparency in performing literature-based reviews (IOM, 2011; NRC, 2011, 2014). The commenter states that NASEM, 2017a, includes four case examples of systematic reviews.
Commenters (2907, 4438, 4831, 4839, 5022, 5165, 5178, 6126, 6373, 8272) suggest EPA work with the National Academies. Commenter (4839) asserts that the design and content of the proposal contradicts credible and well-established scientific principles and methods used across academia, the business community and the public sector. The commenter requests that EPA withdraw the proposed rule and suggests that, if EPA wishes to re-engage in evaluating the topic of transparency in regulatory science, the EPA seek an independent review of key issues from the NAS/NRC and invite the input of other established, competent and credible professional bodies. Commenter (5022) asserts that heeding the advice provided by the National Academies to use the best and most current science to support and revise default assumptions, make implicit defaults explicit, and provide clear standards for the level of evidence needed to depart from default assumptions, taking into account data-sharing guidelines already developed by NIH. Commenter (4438) suggests that subsequent EPA transparency initiatives, if any, be based on consultation with the NAS and should not restrict EPA's ability to rely on the universe of best available science when promulgating regulations. 
Commenter (5011) states that crafting the rule without consulting with the scientific community is a fatal error for this proposal. According to the commenter, even the agency's own SAB has noted the need to consult with scientists in any further development of this proposal. Commenters (1012, 2427, 2472, 3135, 5178, 6144, 6447, 6897, 6915, 6916, 6943, 8272, 8281-PH10, 9225) suggest that the SAB should have been given the opportunity to review the rule and make recommendations. Comments include:
 From a process standpoint, the SAB should have been given the opportunity to consider the EPA's proposed rule before it was released for public input. The commenter notes that the SAB has advised the EPA about the quality of the scientific information utilized in policymaking and EPA research programs and plans for over 40 years. The commenter adds that this Board has an ethical responsibility to remain impartial, consider public input, and consider the best scientific evidence available when making decisions. The commenter notes that SAB's analysis would have provided unparalleled insight into the potential effects of the proposed rule. (6897)
 It seems to reason that when you propose a rule which directly affects how the EPA uses science, you'd want to consult with scientists. The commenter states that they hope that EPA will heed their advice and consult, not just with your SAB, but with the many scientific organizations and societies which have previously expressed concerns about the "Secret Science Act." In the meantime, the commenter strongly urges the EPA to withdraw the proposed rule. (1012)
 Commenter (3135) supports the SAB workgroup's May 12 memo recommending that the SAB review the agency's April proposed rulemaking, "Strengthening Transparency in Regulatory Science." The commenter notes that the workgroup points to a number of complex scientific issues for which the Agency should seek expert advice from the SAB. According to the commenter, the proposed rule suffers from lack of involvement of the scientific community, either within or outside of the EPA. The commenter urges the SAB to recommend that the Administrator:
 Not use the agency's regulatory authority to prescribe specific risk assessment processes;
 Not adopt any major changes to EPA's foundational policies on the use of science in rulemaking without thorough advice and consultation with the SAB, and other authoritative scientific bodies.
 The EPA has failed to consult its own SAB about this proposed rule despite the SAB's assessment that "this rule deals with a myriad of scientific issues for which the Agency should seek expert advice." The commenter contends that proposing a rule that limits the use of scientific data without even notifying, let alone consulting, the Agency's own expert scientific advisors is a text book example of an arbitrary and capricious failure to consider "relevant factors." Motor Vehicle Mfrs. Ass'n v. State Farm Mut. Auto. Ins. Co., 463 U.S. 29, 42-43 (1983). The commenter adds that the EPA offers no explanation for its inexplicable failure to consult with science experts, including the SAB, on this proposal, and it is beyond question that this highly consequential proposal demanded such consultation. The commenter argues that, not only is this statutorily required, see 42 U.S.C. §4365(c)(1), but as the SAB Work Group Memo states, "[t]he proposed rule deals with issues of scientific practice and proposes constraints that the [A]gency may apply to the use of scientific studies in particular contexts. As such, this rule deals with a myriad of scientific issues for which the Agency should seek expert advice from the [SAB]." SAB Work Group Memo at 2. The commenter provides that, underscoring the importance of the issue, the full 44-member SAB followed up on the Work Group's Memo with a unanimous vote to review the proposal and urged EPA to proceed no further until EPA does what it should have done in the first place: "request, receive and review scientific advice from the SAB." SAB June 28 Letter at 1. The commenter asserts that the EPA's effort to rush this proposed rule out the door without any input from the SAB or other scientists violates basic principles of good government and policymaking as well as EPA's legal duty. We urge EPA to withdraw this ill-conceived proposal and to consult with the SAB, the NAS, and the broader scientific community before determining if any rule is needed. (6915)
 The EPA failed even to confer with its chartered SAB, and seek its input, as required under the Environmental Research Development and Demonstration Authorization Act (ERDDAA), 42 U.S.C. § 4365(c)(1), even though EPA claims authority under the CAA, and admits in the Proposal that it is a "significant regulatory action" submitted to OMB for review. As EPA's SAB recently wrote to the Administrator, the commenter notes that a proposal like this, which "focus[es] on the EPA's foundational policies related to the use of science in rulemaking and policy development," is within the SAB's purview and should have been submitted to it for review." According to the commenter, EPA's failure even to submit the Proposal to SAB review, never mind take account of the SAB review results, is evidence of the arbitrary and capricious nature of this whole undertaking. (6916)
 Of particular concern, the EPA's own SAB was not given the opportunity to review the proposed rule before it was announced. The commenter provides that a June 28, 2018, letter from SAB Chair Dr. Michael Honeycutt explained that the SAB only found out about the proposed rule after the related press event and subsequent announcement in the Federal Register. "SAB members had no information regarding the timeline for finalizing the rule and the proposed rule was not identified as a major action in either of the Spring 2017 or Fall 2017 semi-annual Regulatory Agendas." The commenter adds that the letter goes on further to explain that, "the precise design of the proposed rule appears to have been developed without a public process for soliciting input specifically from the scientific community." The SAB consists of EPA-appointed members of the scientific community whose role is to review this type of rule prior to proposal. According to the SAB website, one of the roles of the SAB is to, "...advise the agency on broad scientific matters." The commenter contends that the proposed rule on "transparency" in science surely fits under the umbrella of a "broad scientific matter." (6919)
 The SAB should have been given the opportunity to thoroughly review the proposed rule and make recommendations that would serve to identify and address the specific problems inherent with the implementation of this rule. The commenter notes that this step would have allowed the acknowledged scientific leaders in their respective fields to consider elements in the proposed rule that may be either too far-reaching or otherwise inconsistent with current scientific protocol. (6943) 
 The EPA SAB has asked to review the rule under the authority vested in it by the Environmental Research, Development and Demonstration Authorization Act. The commenter provides that the SAB sent a letter to the EPA administrator, raising many of the same scientific issues of confidentiality, feasibility, and the need for a clearer definition of crucial concepts, such as replication and validation. The commenter urges the EPA to fully consult with the SAB before moving forward with this rule. After the SAB review is complete, the commenter suggests that EPA either withdraw the proposal, or provide an additional opportunity for public comment based on that SAB review. (8281-PH10)
 EPA's SAB, tasked with reviewing the Agency's regulatory agenda and recommending actions that merit independent review, only learned about the rulemaking after it was already proposed. As a result, the commenter provides that an SAB workgroup recommended that the advisory body review the merits of the rule because "it deals with a myriad of scientific issues for which the Agency should seek expert advice from the Science Advisory Board." After a nearly unanimous vote concurring with the memo, the commenter notes that the SAB wrote in a June 28 letter to former Administrator Scott Pruitt that "[t]he SAB urges the Agency to ... request, receive, and review scientific advice from the SAB before revising the proposed rule." (9225)
Response: Public input was an important part of shaping the final rulemaking. The EPA accepted public comment on the proposed rule from April 30, 2018 to August 16, 2018, and held a public hearing on July 17, 2018. In response to public comments, the EPA released a supplemental NPRM and took public comment from March 18, 2020 to May 18, 2020. In addition, the EPA's SAB provided advice and comments on mechanisms for secure access to PII and CBI under the proposed rule on September 30, 2020, and the proposed and supplemental rulemakings on April 24, 2020. All of these comments were considered in developing the final rule.
As discussed more fully in the preamble of the final rule, the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making.
Comment: Commenter (3971) states that, in an ideal world, a "middle ground" proposal would come from balanced committees of experienced professionals. Specifically, the commenter suggests that EPA return to its historical (pre-2000 era) position that these groups have representatives from nongovernmental organizations (NGOs), the private sector (corporations), academia, consultants, trade associations, as well as others who are genuine experts on the topic. The commenter adds that, in the future, it would be good to include at least one or two experts from countries outside the United States. Hopefully, the commenter contends, such committees would contain persons who have not been heavily influenced by forces that are stubbornly wedded to certain positions.
Commenter (3971) expresses that the demand that professionals cannot have ever worked in a corporation that has an interest (knowledge) in a topic, or that someone must be disqualified because he or she has performed research funded by the regulated community, or served as an expert in court for "one side or the other," is not appropriate criterion for exclusion. Often, the commenter notes, special interest groups claim that certain experts are unable to serve on various advisory boards because of claims of bias or conflict of interest. According to the commenter, the result is that SAB often convenes committees that lack persons who have spent a lifetime studying a particular issue.
Commenter (3971) contends that banning true experts because they have served as experts in litigation or conducted studies for the regulated community (often having QC and rigor far higher than studies in universities or government labs) is not an optimal or appropriate policy. Again, the commenter claims that, as this regulation attempts to address, "transparency" is important. The commenter states that if all SAB participants share their possible conflicts and bias with the ethics lawyers at EPA and the public, I believe the public deserves to hear their views.
Commenter (3971) expresses that, as noted in many published papers of the past 5 years, the trial lawyers have had a significant impact on the views of government employees, NGOs, and some experts. The commenter asserts that the SAB group within EPA needs to be more sensitive to possible or likely conflict of interest. According to the commenter, it is no surprise that trial lawyers write many letters complaining of conflicts of interest when consultants or corporate scientists (who often conducted the original studies) are nominated for appointment. The commenter provides that when university professors who receive government grants intended to highlight adverse effects (at any dose) of a chemical, no questions are raised. According to the commenter, it is well known that scientists who report an adverse effect, even if not a physiologically or environmentally dose (with a factor of 1,000), are more likely to get funding than those who do not. Currently, the commenter claims that there is no opportunity for an expert to rebut the claims of parties who have a vested interest in the outcome of the committee process (which should occur to insure proper "due process").
As has been noted by others, commenter (3971) notes that when EPA uses the new "conflict of interest" criterion, sometimes committees of persons who don't know the topic in an in-depth manner are convened, and they reach very poor scientific conclusions. The commenter contends that Committees of well-meaning volunteers who have not "labored in the field" for decades on a particular topic cannot recognize shortcomings in various studies that are brought before them because it does take 10-20 years of study of a particular topic to recognize such flaws. For example, the commenter notes that the presumption that all research that is conducted under a federal grant is valid, while that conducted by the regulated community is compromised, is not appropriate. Although the public and many NGOs believe that research conducted by universities surely must be objective and weighed more heavily than an industry sponsored study, the commenter states that is not a valid assumption. The commenter provides that few universities, if any, perform their work using GLP, or meet the expectations for analytical testing quality that the regulated community must meet. The commenter notes that studies by non-industry groups often have fewer animals, fewer doses, less quality control, improper statistical analyses, and other shortcomings. Future committees should reflect the expertise necessary to weigh the quality of the studies and evaluated the studies objectively.
Commenter (6104) provides contact information for any questions on their comments and requests that EPA periodically meet with states to share information the agency has learned, and to consider any intended and unintended impacts to state programs. The commenter notes that their members, the directors of state surface water quality programs, possess unique knowledge and insight into those clean water program areas that rely most heavily on data, scientific studies, and models. The commenter asserts that, as with all of their comment letters, they encourage the agency to also consider recommendations provided by individual states.
Response: Comments related to conflicts of interest reviews for members of the SAB or other advisory committees are outside the scope of this rulemaking. The EPA agrees that a thorough study quality evaluation should be conducted for all studies used in significant regulatory actions and influential scientific information. Studies are not considered more or less credible based on the affiliation of the authors. The rule requirements also direct the EPA to evaluate the availability of the underlying dose-response data for any subject matter expert, regardless of author's affiliation or funding status. 
2.1.7.2 Proposal Did Not Include Cost Benefit Analyses/Impacts Assessments
Comment: Commenters (2171, 2335, 2430, 2902, 5011, 5021, 5022, 5170, 5172, 5178, 5181, 6046, 6125, 6126, 6133, 6168, 6185, 6358, 6446, 6447, 6874, 6877, 6879, 6893, 6915, 6916, 6919, 6920, 6924, 8272, 8281-PH20, 8801, 9227) assert that the proposal fails to conduct appropriate costs and/or benefits analyses/impacts assessments. Comments include:
 The EPA provide more information about the potential impacts of this proposal in order for the public to fully weigh the proposed rule's implications. (2335)
 Requests that the EPA publicly define the potential data and policy impacts of the proposed rule. (2430)
 The Proposal is fatally flawed because it provides almost no analysis of the impacts of the proposed change in policy. (5011)
 Despite its professed fealty to cost effectiveness in rulemaking, the proposed rule provides no cost-effectiveness analysis whatsoever. According to the commenter, the proposal simply asserts blithely that "EPA believes the benefits of this proposed rule justify the costs." (5022)
 While the proposed rule states that EPA "believes the benefits of this proposed rule justify the costs," EPA offers no estimation or calculation of either the potential costs or benefits. The commenter states that, for such a significant change in policy, a statement of belief is not adequate. (5170, 6125)
 Although this proposal would do real harm to EPA's ability to protect public health and the environment, the proposed rule does not include any assessment of its impact on public health and the environment. (5172)
 Commenter (6879) states that the EPA is proposing to make significant changes to long-standing policies without adequate analysis of the costs and benefits of the proposal. Commenters (6877, 6879) note that EPA has failed to demonstrate that the purported benefits of this regulation would outweigh the costs, including costs in compliance as well as increased risks to human health and the environment. 
 Proposal fails to acknowledge the costs from delays in rulemaking proceedings while EPA performs the additional review called for above and beyond the extensive scientific peer review to which scientific studies have already been subjected. (6915)
 The proposed rule has not provided any supporting information to justify why the rule is beneficial. Moreover, the commenter adds that the EPA has not fully evaluated the costs and benefits of the rule. The commenter states that the costs of the rule could be significant, especially if EPA decides to apply the rule retrospectively, and use the rule as a tool to revise existing health-based air quality standards. The commenter urges EPA to include an evaluation the costs and benefits. The commenter also requests that EPA evaluate what data would be available for setting health-based air quality standards if the rule was applied retroactively and prospectively and consider how it would impact the EPA's ability to protect public health and the environment. (6920)
 The analysis of costs and benefits should include an examination of the studies that have previously informed regulation and the extent to which they will meet the proposed criteria; an analysis of regulations that would not have been promulgated, would have included less-protective limits, or that could be weakened in the future under the proposed criteria; an analysis of the time and costs required to bring all relevant studies into compliance with the criteria; and a calculation of the foregone public-health benefits and associated costs to the U.S. economy when regulations are delayed or are less protective due to this rule. Commenters assert that the analysis of benefits is also lacking, given that EPA failed to present convincing evidence that insufficient data access has caused a problem, or that addressing the purported problem would improve public health, including reproductive health and the health of children. (5178, 6893)
 EPA has completely failed to evaluate the costs and benefits of this action. The commenter asserts that it is a basic tenet and requirement of reasoned decision-making that an agency must provide a reasoned explanation of its action that considers both the advantages and disadvantages of issuing a rule [Michigan v. EPA, 135 S. Ct. 2699, 2707 (2015)]. The commenter states that there is no question that the proposed rule could have profound consequences on the regulatory process at EPA, and ultimately on human health and environmental quality. (6447)
 Commenters (6153, 6358, 6874) note that the EPA the proposal does not contain a thorough analysis of costs and benefits and EPA could have obtained information to inform a robust cost-benefit analysis by issuing an advanced notice of proposed rulemaking (ANPRM) and seeking SAB input, but it did not do so. Commenter (6358) suggests that the analysis of benefits is lacking, given that EPA failed to present convincing evidence that insufficient data access has caused a problem, or that addressing the purported problem would improve public health, including reproductive health and the health of children. Commenters (6153, 6874) contends that such an analysis should include an examination of the studies that have previously informed regulation and the extent to which they will meet the proposed criteria; an analysis of regulations that would not have been promulgated, would have included less-protective limits, or that could be weakened in the future under the proposed criteria; an analysis of the time and costs required to bring all relevant studies into compliance with the criteria; and a calculation of the foregone public-health benefits and associated costs to the U.S. economy when regulations are delayed or are less protective due to this rule. 
 In the background section of this proposed rule, the cost of compliance with "significant regulations" is provided as justification for the proposed rule. The commenter states that, subsequently, it is suggested that dose response modeling used by EPA scientists for "pivotal regulatory science" is overly protective of health and the environment and therefore places unnecessary regulatory and financial burdens on industry. The commenter states that this justification for the proposed rule does not account for the short and long-term costs to individuals and communities from environmental degradation and the resulting population health impacts. (6130)
 The proposed rule does not include any assessment of its impact on public health and the environment. The commenter asks: "Where is the EPAs risk assessment that shows that the proposed changes will have a positive impact on, and not unintended consequences of doing harm to, the groups it is purportedly intended to protect?" The commenter asserts that there are important questions that must be answered before EPA proceeds. The commenter notes that sound scientific information is crucial to creating regulations and policies that protect the health and well-being of us all and that they are is concerned that there is little evidence to suggest that the proposed changes would not adversely affect the quality of the resulting policies. (8801)
 The EPA does not present any analyses of benefit-cost, children's environmental health risks or environmental justice in support of the rule, which are required under Executive Orders 12291, 13045, and 12898. (6924, 8281-PH20) Commenter (6450) states there has been no environmental justice analysis conducted. 
Response: The EPA agrees that neither the 2018 proposed rule nor the 2020 SNPRM included a characterization of costs or benefits of the proposed action. As discussed in the preamble to the final rule, this is a rule of internal procedure promulgated under the EPA's housekeeping authority and a benefit-cost analysis is not required. However, the EPA has included a discussion of costs and benefits in the preamble of the final rule. 
Briefly, the additional procedures required by this rule only apply to pivotal science, which is a subset of the total number of studies that the EPA would evaluate. Given the costs of the current robust process for identifying and reviewing scientific studies and documentation that are existing Agency practice, as well as that the determination of dose-response data availability is limited to pivotal science underlying significant regulatory actions and influential scientific information, the EPA anticipates that the incremental costs of this rule will be small. The Agency may also incur other administrative costs to perform analyses and evaluations to support activities such as exemption decisions made by the Administrator, and documenting these or other decisions made pursuant to the requirements of the final rule. Again, the Agency anticipates that the incremental costs for these activities will be small relative to current administrative costs for developing significant regulatory actions or influential scientific information. Finally, this final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to pivotal science for which the underlying dose-response data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule. 
The main benefits of this rule spring from greater transparency in significant regulatory actions and influential scientific information. By placing greater emphasis on the availability of dose-response data underlying pivotal science, the rule will allow for greater scientific scrutiny as EPA decision makers are developing significant regulatory actions and influential scientific information and increases the likelihood that any errors will be identified and corrected. Greater transparency is also inherently valuable as a principle of good government and provides benefits to the public at large, including reducing the risk of errors in EPA analyses and in the science such analyses rely upon. The ability for independent subject matter experts to validate pivotal science will facilitate more effective comment and engagement with the public during development of future significant regulatory actions and influential scientific information. 
Comment: Commenter (6125) asserts that reasoned decision making requires an agency to identify and address the impact of its proposed actions on its ability to perform its Congressionally mandated functions, State Farm, 463 U.S. at 43, because the likely impacts of the proposal are a critical issue in deciding whether to proceed with the rule. Nevertheless, the commenter states that the proposal outright fails to consider any impacts at all  --  literally none at all! The commenter argues that the failure to consider potential environmental impacts in a proposal that fundamentally affects each of EPA's core statutory responsibilities is fatally arbitrary. See Sierra Club v. Costle, 657 F.2d 298, 326 (D.C. Cir. 1981) ("[W]e can think of no sensible interpretation of the statutory words `best technological system' which would not incorporate the amount of air pollution as a relevant factor.") See also N.L.R.B. v. Creative Food Design Ltd., 852 F.2d 1295, 1306 (D.C. Cir. 1988) (noting that agencies "may not ignore relevant factors" in making decisions); Small Refiner Lead Phase-Down Task Force v. U.S.E.P.A., 705 F.2d 506, 521 (DC. Cir. 1983) ("Arbitrary and capricious" review generally refers to the requirement that an agency "must consider all the relevant factors" and reach a "reasonable" conclusion; Sierra Club v. Costle,657 F.2d at 323."); Environmental Defense Fund, Inc. v. Costle, 657 F.2d 275, 283 (DC Cir. 1981) ("Thus, we must be assured that the agency action was "based on a consideration of the relevant factors," Citizens to Preserve Overton Park, Inc. v. Volpe, supra, 401 U.S. at 416, 91 S. Ct. at 823; and that "the agency has exercised a reasoned discretion, with reasons that do not deviate from or ignore the ascertainable legislative intent," Ethyl Corp. v. EPA, supra, 541 F.2d at 35-36, quoting Greater Boston Television Corp. v. F.C.C., 444 F.2d 841, 850 (1970), cert. denied, 403 U.S. 923, 91 S.Ct. 2229, 29 L.Ed.2d 701 (1971).)"
Moreover, commenter (6125) argues that as in CMA v. EPA, 217 F. 3d 861, 865-66 (D.C. Cir 2000), EPA's assertion of potential benefits without demonstrating their existence or providing any quantitative comparison of their value as compared to costs and disbenefits to public health is yet another clear indicator of arbitrariness and lack of support in the evidence before the agency. Id. (citing State Farm, 463 U.S. at 43).
Response: Based on a consideration of these and similar comments, the EPA has included a discussion of the effects of the final rule on the studies the EPA uses to support final significant regulatory actions and influential scientific information in the preamble of the final rule. Briefly, consistent with existing Agency practice, the EPA will use existing factors (including soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review) to evaluate study quality. When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence (consistent with Agency guidelines on hazard identification and dose-response assessment) to support a relationship between exposure and effect, the EPA will identify a subset of those studies for use in characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect. This will be based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both will be identified as pivotal science. Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. In compliance with the rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or propriety information, available through restricted access in a manner sufficient for independent validation. Including this review of data availability for pivotal science is critical to the EPA's progress toward increased transparency and providing increased opportunity for scientific reanalysis and scrutiny by independent third parties. This approach will result in final significant regulatory actions and influential scientific information that are based on the highest quality, most relevant studies and ensure adequate human health and environmental protections, while maximizing transparency and maintaining consistency with the statutes the EPA administers. When the dose-response data underlying pivotal science are not publicly available in a manner sufficient for independent validation, the EPA will give less consideration to the studies' evidence, findings, and conclusions unless the Administrator grants an exemption request. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science is unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. The EPA provides these factors in 40 CFR 30.5(d) and they are discussed more fully in the preamble of the final rule.
The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. As discussed more fully in the preamble of the final rule, the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making.
The EPA agrees that neither the 2018 proposed rule nor the 2020 SNPRM included a characterization of costs or benefits of the proposed action. As discussed in the preamble to the final rule, this is a rule of internal procedure promulgated under the EPA's housekeeping authority and a benefit-cost analysis is not required. However, the EPA has included a discussion of costs and benefits in the preamble of the final rule. 
Briefly, the additional procedures required by this rule only apply to pivotal science, which is a subset of the total number of studies that the EPA would evaluate. Given the costs of the current robust process for identifying and reviewing scientific studies and documentation that are existing Agency practice, as well as that the determination of dose-response data availability is limited to pivotal science underlying significant regulatory actions and influential scientific information, the EPA anticipates that the incremental costs of this rule will be small. The Agency may also incur other administrative costs to perform analyses and evaluations to support activities such as exemption decisions made by the Administrator, and documenting these or other decisions made pursuant to the requirements of the final rule. Again, the Agency anticipates that the incremental costs for these activities will be small relative to current administrative costs for developing significant regulatory actions or influential scientific information. Finally, this final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to pivotal science for which the underlying dose-response data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule. 
The main benefits of this rule spring from greater transparency in significant regulatory actions and influential scientific information. By placing greater emphasis on the availability of dose-response data underlying pivotal science, the rule will allow for greater scientific scrutiny as EPA decision makers are developing significant regulatory actions and influential scientific information and increases the likelihood that any errors will be identified and corrected. Greater transparency is also inherently valuable as a principle of good government and provides benefits to the public at large, including reducing the risk of errors in EPA analyses and in the science such analyses rely upon. The ability for independent subject matter experts to validate pivotal science will facilitate more effective comment and engagement with the public during development of future significant regulatory actions and influential scientific information.
Comment: Commenter (6133), without support, provides that the Proposal states that "EPA believes the benefits of this proposed rule justify the costs." 83 Fed. Reg. at 18,772. Commenters (6133, 6446, 6916) assert that the only further discussion in the proposal that the benefits of the proposed rule justify the costs is a reference to the Mercatus Paper. Commenters (6133, 6446) note that the proposal (83 FR 18772) includes the following quote from the Mercatus paper: 
      One recent analysis found that: "[i]mprovements in reproducibility can be thought of as increasing the net benefits of regulation because they would avoid situations in which costs or benefits are wrongly estimated to occur or in which regulatory costs are imposed without corresponding benefits. . . ." They concluded that "an increase in existing net benefits from greater reproducibility, which, if it occurred, would cover the costs of obtaining the data and making the data available."
Id. (quoting Randall Lutter and David Zorn, On the Benefits and Costs of Public Access to Data Used to Support Federal Policy Making, Mercatus Working Paper, September 2016). 
The commenter (6133) asserts that this quote is not close to a sufficient cost-benefit analysis. First, according to the commenter, the Working Paper's plausibility analysis is dubious. Among other problems, the commenter states that the analysis examines the time it takes for chemical manufactures, processors, and distributors to identify and provide studies in their possession related to a specific chemical and equates that to the time it would take EPA to obtain, review, process, redact, and publicly maintain data for any study it considers. Lutter and Zorn, at 21 - 22 (citing (40 C.F.R. pt. 716)). The commenter provides that the chemical study Health and Safety Data Reporting Rule and the cost estimate the Working Paper's analysis is based on does not require submission of underlying data unless requested by EPA. 40 C.F.R. § 716.10(a)(4). The analysis also does not include time or costs to the researchers outside of the agency. Lutter and Zorn, at 21 - 22. Further, the commenter notes that the Working Paper assumes that EPA would only receive the underlying data for 20% of the requested scientific studies EPA relies on. Id. at 25. Therefore, the commenter asserts that the Working Paper lowers the already questionable cost estimate by eliminating costs associated with collecting and preparing data for the other 80% of studies. Id. The commenter adds that the Working Paper does not explain what the authors expect EPA to do about 80% of studies EPA currently relies on for which it does not receive the underlying data, but the Proposal would require the agency to unlawfully ignore those studies in regulatory decision making.
Importantly, the commenter (6133) contends that even the partial quote the Proposal presents does not provide results of a cost-benefit analysis nor conclude the costs outweigh the benefits. Instead it says that if an increase in benefits occurred, the costs would be covered. The commenter provides that the same article states this point explicitly:
      Of course, our estimates of the benefits of public access to data supporting federal regulatory decisions fall short of proving that the benefits outweigh the associated costs. They do show, however, the plausibility of such a claim.
Lutter and Zorn, at 29. 
According to the commenter (6133), the Proposal does nothing to address this or try to determine how plausible such a claim is. The commenter argues that the EPA has not provided a defined Proposal, nor done any cost analysis of its Proposal, that could be analyzed. The commenter adds that the fact that this is the best support the EPA could provide for its baseless belief that the Proposal's benefits justify its costs further shows that EPA must withdraw the Proposal.
Similarly, commenter (6916) notes that the Agency fails to provide analysis accompanying the Proposal, as required by Executive Order 12866, "assess[ing] both costs and benefits" to assure "a reasoned determination that the benefits of the intended regulation justify its costs." Instead, the commenter states that the EPA makes unsupported statements about its "belie[f that] the benefits of the proposed rule justify the costs." The commenter adds that the only further discussion on this point is a reference to the industry-funded Mercatus Paper, which rejects a CBO projection that proposed legislation, with terms similar to the Proposed Rule here, would cost the country $250 million a year. The commenter provides that the Mercatus Paper, however does not find the new policy to be cost-free. Instead the commenter states that it reports that compliance with the proposed legislation would add $2,558 to the costs of each study  -  and EPA never explains how that compares with the $250 million price tag from the CBO. 
According to the commenter (6916), EPA baldly asserts that the benefits will outweigh the costs because the Proposal "will improve the data and scientific quality of the Agency's actions and facilitate expanded data sharing and exploration of key data sets," citing only a 2005 NRC report. But the commenter notes that the NRC report does not assess the cost of eliminating peer-reviewed quality science from consideration, it merely discusses the tension between providing more transparency and access to data while continuing to protect confidentiality. Indeed, the NRC explicitly states that its work does not "weigh[] the potential harm posed by disclosure against the benefits potentially foregone."
There is, moreover, according to the commenter (6916), no attempt made by the Agency to quantify or qualify lost benefits, were the new policy to take effect. The commenter asserts that there is no analysis of the number of studies that could be affected by the Proposal, the effects of the Proposal on research, the potential for weakening of environmental regulations, and the resulting environmental or public health costs, or any other costs for that matter associated with eliminating dose-response studies and methods from regulatory decision making because the underlying data cannot be released. In fact, the commenter notes that the EPA does not even seem to understand that there could be such costs associated with its Proposal  -  it asserts (without support) only that making underlying data available "will improve the data and scientific quality of the Agency's actions."
Commenter (6919) expresses that, on the whole, the proposed rule attempts to make a case for the need for open and transparent data and models, yet the notice solicits information and comments from the public without providing any analysis of the impact of the EPA's proposed action. The commenter asserts that this is the type of significant federal action that should have an accompanying regulatory impact analysis. The commenter states that a cost analysis for this proposed action is unavailable, but a comparison can be made with the projected costs from the CBO's review of the HONEST Act of 2017, on which this proposal is based. The commenter notes that the CBO analysis found that the EPA would, "...need to spend at least $100 million dollars per year to upgrade the format and availability of those studies' data to the level required by H.R. 1430." Under Executive Order 12866, the commenter provides that a significant regulatory action is one that may, "[h]ave an annual effect on the economy of $100 million or more or adversely affect in a material way the economy, a sector of the economy, productivity, competition, jobs, the environment, public health or safety, or state, local, or tribal governments or communities." 
Commenter (6919) expresses that the lack of sufficient analysis before publishing the proposed rule makes it extremely difficult, if not impossible, for commenters to provide meaningful and in-depth comments. According to the commenter, this fact, together with the extraordinary number and complexity of the questions that the EPA asks of its commenters, could make the ultimate rule vulnerable to challenges for insufficiency of notice and insufficient consideration of comments received.
Response: The EPA agrees that neither the 2018 proposed rule nor the 2020 SNPRM included a characterization of costs or benefits of the proposed action. As discussed in the preamble to the final rule, this is a rule of internal procedure promulgated under the EPA's housekeeping authority and a benefit-cost analysis is not required. However, the EPA has included a discussion of costs and benefits in the preamble of the final rule. The analysis by CBO noted by the commenters is not applicable to this rule because the analyses were not for this rule and the requirements analyzed are not the same as those in this rule.
Briefly, the additional procedures required by this rule only apply to pivotal science, which is a subset of the total number of studies that the EPA would evaluate. Given the costs of the current robust process for identifying and reviewing scientific studies and documentation that are existing Agency practice, as well as that the determination of dose-response data availability is limited to pivotal science underlying significant regulatory actions and influential scientific information, the EPA anticipates that the incremental costs of this rule will be small. The Agency may also incur other administrative costs to perform analyses and evaluations to support activities such as exemption decisions made by the Administrator, and documenting these or other decisions made pursuant to the requirements of the final rule. Again, the Agency anticipates that the incremental costs for these activities will be small relative to current administrative costs for developing significant regulatory actions or influential scientific information. Finally, this final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to pivotal science for which the underlying dose-response data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule. 
The main benefits of this rule spring from greater transparency in significant regulatory actions and influential scientific information. By placing greater emphasis on the availability of dose-response data underlying pivotal science, the rule will allow for greater scientific scrutiny as EPA decision makers are developing significant regulatory actions and influential scientific information and increases the likelihood that any errors will be identified and corrected. Greater transparency is also inherently valuable as a principle of good government and provides benefits to the public at large, including reducing the risk of errors in EPA analyses and in the science such analyses rely upon. The ability for independent subject matter experts to validate pivotal science will facilitate more effective comment and engagement with the public during development of future significant regulatory actions and influential scientific information.
Comment: Commenter (6125) asserts that the proposed rule is mute regarding its impacts on rules across each of EPA's core environmental statutes, both as to how this rule would apply and how it would affect existing regulations. The commenter notes that the proposal does not even identify the relevant statutory provisions, much less potentially affected regulations. According to the commenter, if the proposal would leave those rules in place, it would accomplish little. The commenter adds that, if the proposal would repeal some or all of them, either directly or by requiring them to be redone, EPA should say so and describe the scope of the proposal's impact. Instead, the commenter states that the proposal is completely silent about whether any existing rules would be affected, and does not even identify any such rules, much less describe how this rule would affect them. Thus, the commenter concludes that it provides no basis for informed public comment.
Response: Based on consideration of this comment and similar comments, the EPA clarified in the 2020 SNPRM the relationship between this rulemaking, the environmental statutes and their implementing regulations by adding language to proposed 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations would control in the event of any conflicts. With this final rule, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
2.1.7.3 No or Insufficient Basis for Rule
 2.1.7.3.1 EPA Fails to Provide Support for Proposal
Comment: Commenters (2171, 2561, 5011, 6046, 6125, 6133, 6137, 6164, 6879, 6916, 8771, 9227) suggest that the EPA has failed to provide the required reasoning/basis to support for why there is a need to change longstanding Agency practice/need for the rule or why other alternatives are not sufficient. Comments include:
 Based on the lack of meaningful information and articulated or demonstrated need for the proposed rule, EPA has not made the case for a new regulation at 40 CFR Part 30. (2171)
 EPA needs to provide the legal authority to impose the proposed rule (they find EPA's legal authority citations to be uncompelling and incomplete). (6109)
 The proposed rule is written with limited explanation, as stated by the Union of Concerned Scientists, limiting the ability for an effectively administered rule. (2561)
 The Proposal is fatally flawed because it provides almost no justification for the proposed change in policy. (5011)
 Neither the Proposal nor docket contains any factual, scientific, technical, logical, or legal support for the suggestion that science and data that are "publicly available in a manner sufficient for independent validation" are necessary elements for the "validity," "reliability," or "transparency" of scientific information. (6133)
 The EPA alludes to "transparency" multiple times, but the without defining what that concept means, or explaining why it believes existing statutes, regulations, guidelines and protocols fail to meet it. (6916)
 Commenter expresses concerns that the EPA is proposing to make significant changes to long-standing policies without adequate justification. The commenter adds that EPA has failed to show any compelling need or justification for the proposed regulation. (6879)
 There is insufficient justification for the proposed rule and portions of it are seriously flawed (e.g., no solutions for confidential data or costs and responsibility for such measures; one person determining exceptions). (6046)
 The proposed rule is predicated on irrational and unsupported conclusions. According to the commenter, a close examination of generally applicable data access policies and guidelines  -  including policies and recommendations of third party organizations and major scientific journals upon which EPA allegedly relies  -  reveals that EPA's Proposed Rule is not based on a reasoned explanation and has no rational connection to facts in the record, but rather is entirely baseless. The commenter adds that the EPA has not demonstrated that it is consistent with scientific principles to categorically exclude peer-reviewed scientific information from all consideration in a rulemaking, as EPA proposes to do. (6137)
 The EPA fails to adequately describe the justification behind this policy change, how it builds upon current policies regarding transparency, such as the IQA and EPA's Information Quality Guidelines, or what the potential impacts of restricting science will be on current and future decision-making. The commenter urges the agency to elaborate on these substantive details and provide the science community and the public the opportunity to provide feedback. (6164)
 EPA has completely failed to follow well-established principles for establishing the legal need for reasoned agency decision-making generally, and in particular for any change to a prior position. According to the commenter, to comply with these requirements, an agency must, among other things:
 Explain why the agency is rejecting policy judgments or factual determinations underlying the prior rule or position.
 Provide a "reasoned explanation" for changing course;
 Demonstrate that the new policy is itself consistent with the governing statute;
 Ensure that the new policy is itself supported by the record, "based on consideration of the relevant factors," and supported with "rational connection[s] between the facts found and the choice made"; and
 Consider relevant alternatives reflected in the prior rule's record, in particular, alternatives most consistent with statutory goals and purposes, and explain why the agency is not adopting them in the new rule  (6125) 
 There are "significant and viable and obvious alternatives" to this Proposal, which the Agency has failed to adequately consider, nor has the Agency explained why it believes such alternatives are not sufficient. The EPA has not supported why releasing confidential health and business information is necessary. Nor, according to the commenter, is the Agency's rationale completely clear in the Proposal. When commenters are left to guess, the commenter notes that is not lawfully sufficient notice of the subjects and issues involved. (6916)
 The failure to adequately justify a changed agency position is one of the hallmarks of arbitrary and capricious decision making The commenter adds that the Agency must "provide a more detailed justification than would suffice for a new policy...when, for example, its [revised] policy rests upon factual findings that contradict those which underlay its prior policy; or when its prior policy has engendered serious reliance interests that must be taken into account. It would be arbitrary and capricious to ignore such matters." The Proposal fails to engage with the reasoning or facts underlying the Agency's longstanding policy position, which respected the confidentiality of underlying data and accounted for all relevant science. (6916)
 Little use of the FOIA to obtain research data has been made over the years, as far as the Congressional Research Service could ascertain. The commenter states that this is an indication that the EPA proposal may impose burdens on researchers, making them produce products that will never be used, except in rare cases. Thus, if EPA proceeds with the rule, the commenter asserts that the EPA needs to justify the requirement that all studies on which it relies must provide public access, rather than requiring public access only in situations where EPA receives a request. (6955)
 The EPA must offer a detailed justification for the premise that there is a "reanalysis crisis" to solve, and a detailed analysis of how its refusing to look at studies that don't do the skeptics' work for them could provide incremental benefits -- otherwise, the proposal must be withdrawn for utter lack of purpose. The commenter states that the EPA has not yet even provided a single anecdotal example of a study whose meaningful reanalysis is hampered. (8771)
 EPA appears to leap to the conclusions (again without any supporting evidence) that the only way to strengthen the validity of the science is by enhancing transparency, that no other possible steps to enhancing integrity are worth considering, and that enhancing transparency means making underlying data and models publicly available. According to the commenter, this is all before EPA even gets to its obviously illogical conclusion that threatening exclusion of studies without publicly available data will "increase access to dose response data and models underlying pivotal regulatory science," rather than simply bar EPA from considering a vast universe of useful and rigorously vetted studies. The commenter adds that the evidence cited by EPA in support of the need to strengthen science through its proposed approach is so vague and perfunctory that it is largely impossible even to tell which conclusions various sources are supposed to support. The commenter provides that the EPA's rationale for its data availability requirements consists of a few conclusory statements by EPA itself, a reference to "the replication crisis," and citations to a handful of articles and guidance issued by EPA and OMB. The commenter contends that none of these provide a rational basis of support for the Proposal. (9227)
Response: In response to public comments, the EPA clarified many aspects of the proposed rule in the 2020 supplemental notice of proposed rulemaking and final rule, including the purpose, legal authority, and definitions of key terms. The purpose of the rule is detailed in the preamble of the final rule. In brief, the transparency provisions in this final rule are intended to build upon existing efforts and provide incremental progress toward the Agency's overarching goal of greater transparency. The EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. The scientific community has also embraced greater data transparency, as evidenced by data sharing and availability requirements for many high-impact journals and the emergence of several international organizations and initiatives focused on increased transparency and data sharing in science. The EPA supports these efforts and believes that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. The incremental progress made by this rule provides an important step towards prioritizing transparency in particularly impactful rules and assessments and will inform future statute-specific rulemakings. 
The EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. 
Further, in consideration of public comments, the EPA is also finalizing the alternative approach presented in the SNPRM, rather than providing for the exclusion of studies based on the availability of their underlying data. As described in the final rule, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. When characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect, the EPA will identify pivotal science and give greater consideration to pivotal science for which the underlying dose-response data are available in a manner sufficient for independent validation. Including this review of dose-response data availability for pivotal science is critical to the EPA's progress toward increased transparency and providing increased opportunity for scientific reanalysis and review by independent third parties. This approach will result in significant regulatory actions and influential scientific information that are based on the highest quality, most relevant studies and ensure adequate human health and environmental protections, while maximizing transparency and maintaining consistency with the statutes the EPA administers.
The EPA would also like to clarify that the rule does not impose requirements on parties external to the EPA, including researchers. The rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
Comment: Commenter (9227) asserts that the EPA has provided no explanation or justification for targeting dose response studies in particular or for not including other types of studies or scientific information. The commenter adds that the EPA has not suggested that these studies are inherently less reliable than other studies, that they more commonly fail to publicly disclose data and modeling information, that replication is more necessary for these studies than others, or any other conceivable reason. Absent any explanation from the agency, the commenter contends that it is impossible to comment on the factual predicates for EPA's proposed decision, or the reasonableness of EPA's justification, except to state that it appears completely arbitrary in the absence of any rationale. See, e.g., Transactive Corp., v. United States, 91 F.3d 232, 237 (D.C. Cir. 1996) ("A long line of precedent has established that an agency action is arbitrary when the agency offered insufficient reasons for treating similar situations differently.").
Response: As explained above, in response to public comments, the EPA clarified many aspects of the proposed rule in the 2020 supplemental notice of proposed rulemaking and final rule, including the purpose, legal authority, and definitions of key terms. As discussed more fully in the preamble of the final rule, the rule is designed to build upon data initiatives across the federal government and scientific community. The rule is not intended to fix a particular error, but rather to pursue an incremental approach to maximizing transparency in the science that EPA relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. The EPA is focusing on the underlying dose-response data for this rulemaking because of the influence these data have on particularly impactful decisions at the Agency. Risk assessments and regulations that target emissions and risk reduction of one or more pollutants, contaminants, or substances are integral to the Agency's mission and the underlying dose-response data that inform the quantitative value used to evaluate and mitigate potential risk are critical to understanding the assessment or regulatory action. In addition, the data underlying the dose-response assessment are more distinct than the broad range of data informing an entire risk assessment. Therefore, the EPA is concentrating its current efforts to increase transparency on a well-defined step in the quantitative assessment of risk supporting specific Agency actions. This final rule provides an important step in furthering the progress already being made toward maximizing transparency and will provide important insight for developing future statute-specific requirements.
Further, in consideration of public comments, the EPA is also finalizing the alternative approach presented in the SNPRM, rather than providing for the exclusion of studies based on the availability of their underlying data. As described in the final rule, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. When characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect, the EPA will identify pivotal science and give greater consideration to pivotal science for which the underlying dose-response data are available in a manner sufficient for independent validation. Including this review of dose-response data availability for pivotal science is critical to the EPA's progress toward increased transparency and providing increased opportunity for scientific reanalysis and review by independent third parties. This approach will result in significant regulatory actions and influential scientific information that are based on the highest quality, most relevant studies and ensure adequate human health and environmental protections, while maximizing transparency and maintaining consistency with the statutes the EPA administers.
2.1.7.3.2 Materials Cited/Administrative Record Do Not Support Proposal
Comment: Commenters (5022, 6133, 6447, 6450) express that materials cited/administrative record do not support proposal. Comments include: 
 The proposal lacks critical citations and documentation. According to the commenter, rather than furnishing the evidentiary support required for administrative action, the agency has merely adopted a legislative initiative that failed to pass despite support from the energy, chemical, manufacturing, and other key industries. (5022)
 Materials cited by EPA also do not provide any support for the Proposal, on scientific, technical, policy, logical, or legal grounds, and in fact, the materials actually undermine the Proposal. According to the commenter, the cited materials demonstrate that the Proposal is unsupported, arbitrary, capricious, and otherwise inconsistent with law. The commenter contends that the fact that EPA cites many of these materials because they contain, from different contexts, options EPA could enact as part of the proposed rule further demonstrates that the Proposal must be withdrawn as it fails to provide fair notice to the public of what is being proposed. (6133)
 The administrative record before the Agency offers no evidence of Congressional intent that the EPA should consider transparency of underlying data and models that inform the scientific research the Agency relies on when adopting regulations pursuant to the Acts. The commenter states that, without such Congressional intent, the EPA has no basis to adopt the Transparency Rule as proposed. The commenter states that to do so very likely would contradict the Congressional intent evident in the cited sections of the Acts. (6178)
 The Proposal is not supported by the administrative record. (6447)
 Many of the relevant documents underlying the proposal have been excluded from the record, meaning there is not sufficient specificity to allow for a sound response. (6450)
Response: As explained above, in response to public comments, the EPA clarified many aspects of the proposed rule in the 2020 supplemental notice of proposed rulemaking and final rule, including the purpose, legal authority, and definitions of key terms. As further discussed in the preamble of the final rule, the rule is designed to build upon open data initiatives across the federal government and scientific community developed over the last several years. The rule is not intended to fix a particular error, but rather to increase transparency in the pivotal science used in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
2.1.7.3.3 No Basis for Assumption that Science or Studies for Which Data are Publicly Available Yield More Valid Results
Comment: Commenters (6125, 6133, 6361) suggest that EPA has not provided support for their assertion that the proposal will "improve the data and scientific quality of the agency's actions." 
Commenter (6133) asserts that the EPA provides no basis for its assumption that science or studies for which data are publicly available yield more valid or reliable results than the best available, peer-reviewed, independent, credible science, for which the underlying data are not publicly available. The commenter adds that the proposal arbitrarily fails to address, much less explain, why prior EPA regulatory actions that relied upon studies, data, or other information did not reflect the "best available science" or why they were otherwise unreliable, despite failing to meet the Proposal's standards. 
Commenter (6133) states that the Proposal fails to explain how the term, "in a manner sufficient for independent validation," and the proposed §30.5 definition will increase transparency in science or why it is necessary to ensure that EPA will consider the best available science. To the contrary, the commenter contends that the Proposal's approach would preclude EPA from considering the best available science that is relevant to EPA's responsibilities. The commenter also states that the EPA also fails to explain why data underlying peer-reviewed studies must be publicly available "in a manner sufficient for independent validation" when independent researchers can verify science without making the underlying data, which is often confidential, publicly available. 
Commenter (6133) notes that the Proposal does not require that any information actually be independently validated before EPA may consider it or base regulatory decisions on such verification. Accordingly, there is an irrational disconnect between EPA's insistence that information be "publicly available for independent validation" and the Proposal's claim that this ensures EPA will consider and use the "best available science." See 83 Fed. Reg. at 18,769. The commenter adds that the EPA itself has not outlined a process by which "dose response data and models" would be validated, and the Proposal does not seriously consider the methodological complications of partial redaction of underlying study data.
Commenter (6361) suggests that the EPA also does not explain how the Proposed Rule approach will ensure the application of "best available science" which is the standard in all environmental laws. According to the commenter, it is possible that EPA's proposed regulation will result in a failure to rely on the "best available science" as EPA arbitrarily disqualifies important scientific studies if the underlying data cannot be disclosed to the public or cannot be independently validated.
Commenter (6125) states that the proposed rule directs EPA to make all reasonable efforts to explore methodologies, technologies, and institutional arrangements for making dose response models and data underlying pivotal regulatory science used in significant regulatory decisions available to the public in a manner sufficient for independent validation, consistent with law and protection of privacy, confidentiality, and national and homeland security. However, it does not compel the Agency to make that information available where it concludes after all such reasonable efforts that doing so in way that complies with the law and appropriate protections is not possible.
Commenter (6125) adds that, while it is not flatly untrue that the proposal "does not compel" EPA to make the information available, it is worse than disingenuous not to mention also that unless the information is publicly available it cannot be used as a basis for the rulemaking decision unless EPA grants an exemption. See at 18770 n. 7. The commenter contends that, even though limited by time and resources, their assessment shows the magnitude of the issue and the absolute need for EPA to address the potential damages to public health and the environment before any final action. To issue a valid rule, the commenter states that the EPA would have to analyze this issue on its own, including what studies the proposal will exclude, present the results for public comment, and explain why the benefits of the per se rule justified the public health sacrifice and regulatory confusion that it would cause.
Response: As detailed in the preamble to the final rule, the purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information and does not modify the Agency's interpretations of "best available science." The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. The EPA will then identify and consider pivotal science in accordance with the provisions of this rule, unless the implementation of the rule conflicts with statutory requirements and associated implementing regulations. The EPA also wishes to clarify that the rule does not require the EPA to collect, store, or publicly disseminate data and models underlying pivotal science.
2.1.7.3.4 No Support for Assertion that the EPA has Not Consistently Followed EPA Policies
Comment: Commenter (6133) adds that the Proposal provides no support for the idea that EPA has not consistently followed previous EPA policy that encouraged the use of non-proprietary data and models. To the extent EPA believes this is a problem, the commenter suggests that the EPA withdraw the Proposal and explain what policies it has not followed and how it has not followed those policies. The commenter notes that EPA should then present options to address those alleged shortcomings. 
Commenter (6133) argues that if EPA decides to move ahead with this reckless, unjustified, and unlawful effort to censor the science that EPA may consider, and must consider, to protect Americans' health and environment, the agency must first issue a supplemental proposal and actual administrative record to cover the multitude of issues, evidence, and specific regulatory text for which EPA fails to provide fair notice. According to the commenter, the Proposal fails to provide fair notice or justifications addressing numerous issues that our comments detail -- from an absence of any statutory authority, to failures to address statutory authorities that the Proposal squarely contravenes, to failures to provide reasoned explanations, including basic justifications for EPA's numerous departures from past practices. The commenter adds that the proposal fails to propose specific regulatory text addressing numerous implementation elements, as well as issues that are touched upon only in passing in the preamble (e.g., non-linearity and LNT). Apart from all of the significant substantive and procedural defects from which the Proposal suffers, the commenter contends that it still manages to be a shockingly shoddy effort missing actual regulatory text and supporting legal, factual, scientific, and technical information that would provide fair notice to the public.
Response: The purpose of the rule is detailed in the preamble of the final rule. In brief, the transparency provisions in this final rule are intended to build upon existing efforts and provide incremental progress toward the Agency's overarching goal of greater transparency. The EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. The scientific community has also embraced greater data transparency, as evidenced by data sharing and availability requirements for many high-impact journals and the emergence of several international organizations and initiatives focused on increased transparency and data sharing in science. The EPA supports these efforts and believes that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. The incremental progress made by this rule provides an important step towards prioritizing transparency in particularly impactful rules and assessments and will inform future statute-specific rulemakings. 
Further, in consideration of public comments, the EPA is also finalizing the alternative approach presented in the SNPRM, rather than providing for the exclusion of studies based on the availability of their underlying data. As described in the final rule, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. When characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect, the EPA will identify pivotal science and give greater consideration to pivotal science for which the underlying dose-response data are available in a manner sufficient for independent validation. Including this review of dose-response data availability for pivotal science is critical to the EPA's progress toward increased transparency and providing increased opportunity for scientific reanalysis and review by independent third parties. This approach will result in significant regulatory actions and influential scientific information that are based on the highest quality, most relevant studies and ensure adequate human health and environmental protections, while maximizing transparency and maintaining consistency with the statutes the EPA administers.
Finally, the EPA clarified many aspects of its rulemaking in the 2020 SNPRM and the final rule, including its authority. The EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule.
 2.1.7.3.5 Mismatch Between Problem Identified and Solution Adopted
Comment: Commenter (6933) suggests that, in general, the NPRM should be withdrawn because to the extent it can be understood what it requires -- through the haze of vague definitions and exceptions -- it both fails to address the issue which it purports to be concerned with, and is inconsistent with EPA's obligations to use best available science under both statutory requirements and caselaw. In essence, the commenter notes that the rule identifies the goal of improving the quality and transparency of EPA's science and uses it as an excuse to adopt policies that actually reduce the quality, and likely the transparency, of EPA's scientific decision-making. The commenter contends that there is a fundamental mismatch between the problem identified and the solution adopted. 
Commenter (6933) asserts that the contradiction between purpose and means that lies at the core of the NPRM is clearly exposed by the fact that on the one hand it purports to be a proposal "to provide a mechanism to increase access" to scientific information, 81 FR 18770, and other hand it purports to impose no requirements on outside parties at all. See 81 FR 18772 (noting that the rule is not covered by Executive Order 13771 because "it relates to `agency organization, management, or personnel"); id. (stating the action does not contain any paperwork collection activities); id. (Regulatory Flexibility Act (RFA) certification that the action "will not impose any requirements on small entities"). According to the commenter, the agency cannot maintain on the one hand that the proposal simply governs what science the agency can review, and is not intended to change the behavior of outside parties, and at the same time assert that imposing these requirements will lead to greater access to data by the public.
The commenter (6933) contends that it is apparent that any handwaving the NPRM makes in the direction of the idea that the proposal will ultimately encourage investigators to make data more available are merely window dressing for the rule's actual purpose and effect -- to reduce the scope of scientific information considered by the agency by making public access to data the sole criterion of the quality of scientific research.
Commenter (6933) states that, by failing to wrestle with significant dimensions of the problem and explanation on how its suggested solution will actually improve them, the agency fails to satisfy basic requirements of administrative law. In addition, the commenter notes that the rule is so vague that it fails to provide any basis on which the public can comment. The commenter adds that it is not mere happenstance that section II of the NPRM is labeled "background" and section III is "request for comment." The commenter argues that the NPRM spends some time discussing the alleged problem, almost no time describing what the rule actually does, and then has a long section identifying all of the issues which the NPRM does not address but on which comment is solicited, including such fundamental issues as to what data and models it should apply to, what rules it should apply to, how it applies to reviews of the NAAQS, whether or how to apply it retrospectively, whether and which exemptions are appropriate, etc. etc. In the absence of any information as to the agency's proposed course of action on these fundamental questions, the commenter asserts that the opportunity for public comment has been rendered meaningless just as much as if the agency had limited the scope of comment to a narrow subset of issues.
Commenter (6933) asserts that, even for things that are actually proposed, the NPRM is too vague to support a final rule. For example, proposed §30.2 ("Definitions") states that "all terms not defined herein shall have the meaning given them in the Act or in subpart A." But the commenter notes that given that there is no subpart A, how is anyone supposed to comment on the definitions? And the commenter adds that the definition of "pivotal regulatory science" is "the specific scientific studies or analyses that drive the requirements and/or quantitative analysis of EPA final significant regulatory decisions." The commenter states that often there is no single "specific" scientific study that "drive[s]" a regulatory decision. The commenter provides that ISAs for NAAQS may be thousands of pages long and cite many thousands of studies and that the scientific record for the Clean Water Rule was likewise a myriad of studies constituting thousands of pages. According to the commenter, it is reductivist and scientifically inaccurate to identify any single study as "the" specific study, so what are the criteria EPA will use to implement this definition? The commenter contends that without adequate information, the proposal could mean anything to everything and that is an inadequate basis for public comment. 
Commenter (6194) asserts that notices of proposed rulemakings must also include "the terms or substance of the proposed rule or a description of the subjects and issues involved." 5 U.S.C. § 553(b)(3). The commenter notes that the purpose of this requirement is to "provide sufficient factual detail and rationale for the rule to permit interested parties to comment meaningfully." Honeywell International, Inc. v. EPA, 372 F.3d 441, 445 (D.C. Cir. 2004). The commenter asserts that the Federal Register notice of the Proposal is too vague to allow for fully meaningful and targeted comments. Given the ambiguities of the rule, the commenter states that the public cannot adequately assess what EPA thinks the Proposal means or how it will implement it. Nevertheless, for purposes of these comments, the commenter assumes EPA intends a broad interpretation that would preclude a swath of scientific studies and information from use and consideration in regulatory proceedings.
Moreover, commenter (6194) notes that many of the sources cited in the Proposal either do not support the text of the proposed regulation, do not support the proposition for which they are cited, or are so broad that the public does not have fair notice of, and therefore cannot evaluate, EPA's rationale for the Proposal. The commenter adds that the overall lack of clarity is illustrated by EPA's own solicitation of feedback; it covers so wide a range of issues -- posing over two dozen questions -- that it makes clear EPA has not identified a proposed path forward on which the public can comment. The commenter asserts that the Proposal would be more appropriately posited as an Advanced Notice of Proposed Rulemaking; it does not meet the APA's standards for proposed rules. 
Response: In response to public comments, the EPA clarified many aspects of the proposed rule in the supplemental notice of proposed rulemaking and final rule. As further discussed in the preamble of the final rule, the rulemaking establishes internal Agency procedures and does not intend to change the behavior of external parties. Further, in response to public comments, the EPA did not finalize the 2018 proposal or the 2020 SNPRM proposed option that would have categorically required that the underlying data and models be available for independent validation in order for studies to be considered pivotal science. However, given that transparency is an important aspect of EPA's regulatory actions and assessments it should be an important consideration in how the Agency considers pivotal science. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating final significant regulatory actions and finalizing influential scientific information. As clarified in the preamble to the final rule, the EPA is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. The rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
2.1.7.4 Proposed Rule's Requirements Insufficient
Comment: Commenters (0057, 0556, 2876, 2907, 4831, 4838, 4881, 4884, 5021, 5165, 5170, 5178, 5287, 6046, 6104, 6109, 6110, 6125, 6126, 6133, 6142, 6146, 6147, 6152, 6163, 6164, 6185, 6188, 6194, 6233, 6351, 6447, 6450, 6875, 6909, 6915, 6916, 6919, 6920, 6924, 6933, 9224, 9225, 9227) oppose the proposed rule because the process it prescribes is too vague/lacking in detail, flawed, or illegal. Comments include: 
 The proposed regulation is too vague to be meaningfully evaluated and successfully implemented. For example, according to the commenter, it is unclear whether §30.7 requires EPA to conduct its own peer review of all pivotal regulatory science and, if so, whether EPA has the capacity or capability to perform those reviews. See 83 Fed. Reg. at 18774. Likewise, the commenter states that the exemption process does not provide sufficient standards to ensure that the Administrator makes consistent determinations. (4838)
 The consequences of the proposed rule, significant they may be, are indeterminate in light of the proposal's vagueness. (5021)
 The proposal fails to clearly articulate the specific proposed requirements, impairing informed public comment. (5021, 6185)
 EPA needs to first propose a rule for using studies confirmed by peer-led and noted duplicate study confirmation. According to the commenter, the regulation as it stands is highly incomplete and highly questionable. (5287)
 A detailed operational structure for implementation needs to be developed. (6142)
 The specific actions of the proposed rule are difficult to identify in the docket due to being intermingled with reasoning for why the proposed rule should be implemented. (6046)
 EPA has failed to provide the public with sufficient information on which to offer comment. In fact, the commenter contends that it is clear from the "Request for Public Comment" section that EPA does not have a full understanding of the very rule it is proposing. The commenter contends that the EPA clearly does not have a plan for how it will implement the rule or what its effects will be. The commenter states that the topics upon which EPA seeks comment should have been provided to the public as part of a fully thought-out rule proposal. The commenter adds that because EPA provides only questions and no proposed actions, EPA eliminates their ability to comment on many aspects of the rule. (6109)
 The Proposal suffers from what the commenter believes is unlawful vagueness. If EPA intends to finalize what they believe is an unlawful proposal, EPA first must withdraw the Proposal, then issue a supplemental proposal with the necessary definitions and explanations. Better yet, the commenter suggests that EPA abandon what they believe to be an illegal and harmful proposal altogether. (6133)
 Commenters are supportive of scientific transparency in regulatory development. Unfortunately, according to the commenters, the rule is vague in several areas and does not provide specific regulatory language for review and comment. Commenters (6104, 6878) request also that the EPA provide more details regarding the intent, scope, and implementation processes associated with this proposal. The commenter believes these discussions would improve the quality of the comments on the rule and will contribute to an enhanced and improved final rule, should this rulemaking go forward. Further, the commenter requests that EPA issue a supplemental notice of proposed rulemaking that includes actual regulatory language. As part of this supplemental notice, the commenter requests that EPA provide sufficient detail for an analysis of whether this new approach would achieve the results intended, while also continuing to support states efforts to implement environmental requirements in their states. (6104, 6878) . In the spirit of cooperative federalism, commenter (6104) suggests that before the rule is finalized, the EPA host coregulatory discussions.
 The proposed rule lacks clarity and it is difficult to ascertain what it seeks to achieve, whether EPA is interested in greater transparency across all of its programs and activities, and what the long-term implications could be of any changes made. The commenter welcomes additional dialogue with the Agency on what this rule aims to achieve, but even with an extended comment period, the commenter believes that the proposed rule is too ambiguous for the commenter to provide detailed comments. (6146)
 The rule is poorly written and in fact lacks transparency which it purports to support. The rule is vague, open-ended and without necessary safeguards to assure that scientific evidence is used without social, political, or economic modifying factors. (6147)
 The proposal is vague and unworkable and notes that it was developed without significant input from the scientific community, and therefore the definitions and processes in the proposed rule are not specific. The commenter contends that this leaves stakeholders unable to determine how the rule would be applied. (6152)
 The Proposal's vague language suggests unpredictable implementation. (6188)
 The current proposal is far too vague and missing key components that must be included in the final rule to ensure its understandability and appropriate implementation. For example, the commenter notes that currently the proposal lacks definitions for many key terms, omits critical protocols and methodologies necessary to put this rule into action, and does not fully explore the implications of implementing a rule of this nature. (6875)
 The EPA has not adequately considered the implications of this rule for the conduct of science as a whole, and those implications are very negative. According to the commenter, the proposed rule is so woefully incomplete and lacking in support as to deprive the public of a meaningful opportunity for comment. (6909)
 With respect to EPA's process, this proposal has been rushed, is vague, and creates more questions than it answers: it does not clearly state the actual parameters of the proposed rule, it is open-ended in terms of alternatives under consideration, and it fails to provide critical information such as projected costs. The commenter also notes that it is also completely unclear -- or worse, contradictory -- whether and how this proposed rule would apply to EPA's cost-benefit analyses. (6915)
 The Proposal's terms are so vague as to fail to afford an opportunity for meaningful comment. The commenter provides that significant issues are presented in the Proposal only in the broadest terms, despite an APA requirement that a proposed rule includes "the terms and substance of the proposed rule or a description of the subjects and issues involved," 5 U.S.C. § 553(b)(3), and with enough specificity "to permit interested parties to comment meaningfully." (6916)
 The process underlying the proposed rule is flawed due to its lack of adequate analysis and input from the start and the insufficiency of the notice and information provided to commenters. As such, the commenter suggests that the proposal be withdrawn until proper preliminary analysis has been completed. (6919)
 The language of the rule is confusing, contradictory, and poorly and incorrectly referenced with little science or policy foundation. According to the commenter, this suggests the rule authors lack understanding of the scientific process. (6924)
 The current proposal is far too vague and missing key components that must be included in the final rule to ensure its understandability and appropriate implementation. (9224)
 The ill-conceived, badly written, and unlawful proposal is flawed beyond repair and should be rescinded. Further, according to the commenter, this proposed rule runs counter to EPA's stated commitment to "robust and civil dialogue with the public." The commenter contends that any further time and money spent on this proposal would be a waste of valuable public resources and that EPA and OMB should focus their limited resources on protecting public health and the environment rather than continuing to consider such a flawed proposal. (9225)
Response: In response to public comments, the EPA clarified many aspects of the proposed rule in the supplemental notice of proposed rulemaking and final rule, including key definitions and an overview of the process that the EPA would follow. The final rule clarifies that the rule requirements set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. As further discussed in the preamble of the final rule, the EPA maintains that this rulemaking establishes internal Agency procedures and does not intend to change the behavior of external parties. Finally, in consideration of public comments, the EPA did not finalize the 2018 proposal or the 2020 SNPRM proposed option that would have categorically required that the underlying data and models be available for independent validation in order for studies to be considered pivotal science. However, given that transparency is an important aspect of EPA's regulatory actions and assessments, it should be an important consideration in how the Agency considers pivotal science. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations. The final rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
2.1.7.5 Insufficient Details Regarding the Handling of and Sensitive Personal Data 
Comment: Commenters (0068, 2429, 2787, 4594, 4831, 6125, 6133, 6362, 6448, 6893, 6911, 6916, 6955, 8272, 9227) suggest that there is insufficient detail regarding the handling of CBI and sensitive personal data. Comments include:
 Before EPA imposes a requirement that all primary data developed in future studies be publicly available, the Agency should perform a systematic review of the relevant literature on the strengths and limitations of strategies to anonymize. (0068)
 Proposed rule does not state the Agency's plans for disseminating publicly available data that are under the jurisdiction of other federal agencies, such as Medicaid and Medicare. The commenter points out that a recent study that might be excluded from consideration showed that long-term exposure to particulate matter and ozone at levels below the annual standards was associated with an increase in mortality in the Medicare population. (2429)
 Key issues including how data would be made available to the public and how private information would be protected are not addressed. According to the commenter, this is a serious deficiency in a rule meant to increase access to data for the public. (2787)
 EPA has solicited input on measures to "provide protected access to identifiable and sensitive data." The commenter asserts that this is a significant issue and one that EPA should fully understand prior to moving forward with any new rule. The commenter recommends that EPA initiate a formal request for public comment on this issue alone and use what is learned to help inform and guide any potential future rule on transparency. (4594)
 The proposal acknowledges that data may be "controlled by third parties," but its entire explanation of what this may mean is a vague promise that "EPA shall work with those parties to endeavor to make the data available in a manner that complies with this section." The commenter notes that this language does not begin to address the variety of complexities involved with data controlled by a wide variety of third parties each with different policies of access, making it nearly impossible for commenters to know whether third party studies can ever be used without some type of interactive process between the study authors and EPA. Again, the commenter states that even if EPA does not intend to apply this provision to categorically bar the use of third party studies, its failure to point this out, or otherwise explain this provision makes the rule too vague to implement and is still another instance of failure to provide adequate notice. (6125)
 Commenter suggests that EPA should provide greater clarity regarding what it intends to do in circumstances where raw data cannot be made publicly available. (6362)
 Concerned with the lack of details around appropriate access to confidential information. While noting that "[n]othing in the proposed rule compels the disclosure of any confidential or private information in a manner that violates applicable legal and ethical protections," the lack of specificity and applicability is found wanting and the Agency should expand at great length should the rule go forward. (6448)
 The proposed rule states that EPA aims to honor "legal and ethical obligations to reduce the risks of unauthorized disclosure and reidentification" and that concerns about access to confidential or private information can "be addressed through the application of solutions commonly in use," but it does not describe criteria or processes for determining which data sets would be exempted from data release requirements, what alternative requirements might be applied to data sets containing identifiable information, or which activities the Administrator will exempt. (6893)
 Commenter suggests that the Proposal be revised to further clarify how EPA will protect private, sensitive and confidential information, as well as provide an exemption process that allows for clearly and objectively assessing studies without excluding them from the rulemaking and guidance process. (6911)
 The Proposal provides no discussion of how underlying data of relevant studies can be released consistent with the various laws and regulations protecting privacy and prohibiting disclosure. According to the commenter, the failure to "consider an important aspect of the problem" renders the Proposal arbitrary and capricious. (6916)
 No protections have been built into the proposal to prevent or reduce potential abuse of researcher data. (6955)
 The proposal states that EPA aims to honor "legal and ethical obligations to reduce the risks of unauthorized disclosure and re-identification" and that concerns about access to confidential or private information can "be addressed through the application of solutions commonly in use," but it does not describe criteria or processes for determining which data sets would be exempted from data release requirements, what alternative requirements might be applied to data sets containing identifiable information, or which activities the Administrator will exempt. (8272)
 In the proposal, EPA blithely and irrationally ignores or assumes away the real and significant issues, suggesting that existing mechanisms and techniques can be used to protect privacy and confidentiality while making underlying research data publicly available. In fact, the commenter provides that evidence (including several of the sources that EPA cites) indicates that the potential mechanisms alluded to by EPA would only have the potential to address some of the barriers cited above, have serious limitations even for those, and are actually becoming less effective as it becomes easier to combine and manipulate public data sets. The commenter adds that the "solutions" EPA might have in mind do not address the issues raised by the Proposal because no other agency has tried to implement a requirement such as the one EPA proposes. The commenter provides that other agencies provide guidance and techniques to protect privacy during data collection and disclosure to allow more use of data collected by the government, not to mandate that data collected by academic or industry researchers be publicly available for purposes of replicating analyses. The commenter adds that the EPA cursorily mentions a range of options for facilitating secure access to confidential data, including: "[r]equiring applications for access; restricting access to data for the purposes of replication, validation, and sensitivity evaluation; establishing physical controls on data storage; online training for researchers; and nondisclosure agreements." The commenter states that the EPA does not indicate whether it would deem providing access with these types of controls in place sufficient to meet EPA's proposed requirement "publicly available in a manner sufficient for independent validation." Comments provided by this commenter that are specific to de-identification methodologies/issues, implementation costs, and privacy data requirements are presented in other relevant sections of this document. (9227)
Commenter (6133) provides that the EPA concedes that concerns about access to confidential or private information cannot always be addressed but says nothing about these instances or how it intends to evaluate them. The commenter notes that, for the times that EPA believes concerns about access to confidential or private information can be addressed, the Proposal does not explain how it plans to do so nor address the costs. The commenter adds that the Proposal merely directs readers to general and vague statements from different contexts and fails to provide fair notice or justification of what EPA would do to address issues with confidential or private information.
Commenter (6133) states that the Proposal merely says to "[s]ee examples from the U.S. Department of Health and Human Services, National Institute of Standards and Technology, U.S. Department of Education, and the U.S. Census Bureau," id. n.16, and points generally to Health and Human Services "Guidance Regarding Methods for De-identification of Protected Health Information in Accordance with the HIPAA Privacy Rule," id. n.17. The commenter notes that the Proposal does not say what actions from these examples EPA proposes to use.
Commenter (6133) also states that the Proposal states that the National Academies have noted that in the past, restricted data products were created by relatively simple data masking, coding, and de-identification techniques, and notes that "[n]othing in the past suggests that increasing access to research data without damage to privacy and confidentiality rights is beyond scientific reach." 83 Fed. Reg. at 18,771 (citing Expanding Access to Research Data Reconciling Risks and Opportunities, The National Academies Press, 2005, https://www.nap.edu/catalog/11434/expanding-access-toresearch-data-reconciling-risks-and-opportunities at 27, 36). First, according to the commenter, this is not fully supported as experience shows increasing access to data can damage privacy and confidentiality rights. Again, the commenter notes that the Proposal does not say which, if any of these techniques the EPA will use, or how the EPA will use them. And while the National Academies may believe that increasing access to data without damage to privacy and confidentiality is not beyond scientific reach, the Proposal does not explain how this belief translates to past, present, and future scientific studies EPA considers in regulatory decision making. The commenter provides that this document does not explain how EPA will address concerns about confidential or private information and does not support EPA's Proposal to preclude consideration of those studies that do not make public underlying data for those, or other reasons.
The commenter (6133) provides that the Proposal next cites to two National Academies documents and a document from the Bipartisan Commission on Evidence Based Policy. 83 Fed. Reg. at 18,771 & n.19. But, according to the commenter, the Proposal fails to explain how these documents support its proposed actions or explain how EPA intends to protect confidential information. The commenter notes that the Proposal merely states that they "have discussed the challenges and opportunities for facilitating to secure access to confidential data for nongovernment analysts." 83 Fed. Reg. at 18,771. The Proposal does not address those challenges or describe the opportunities it intends the EPA to use. Again, the commenter asserts that these documents do not support the vague Proposal.
Commenter (6133) states that the Proposal states that "the requirements for availability may differ," and "may range from deposition in public data repositories, consistent with requirements for many scientific journals, to, for certain types of information, controlled access in federal research data centers that facilitate secondary research use by the public." Id. (footnotes omitted). The commenter provides that the Proposal again cites to journal policies or recommendations generally and the policies for access to data from NIH and Census Bureau. Id. nn.20 & 21. Importantly, the commenter notes, the Proposal does not say how the requirements would differ, what studies would be required to deposit what data into what repositories, and what studies would be required to allow controlled access to what data in what federal research data centers. Moreover, the commenter provides that the Proposal does not address the costs that these actions would entail. Again, according to the commenter, if EPA intends to use these different ways to provide data that meet concerns about confidential and private information, the agency must withdraw the rule and issue a new proposed rule that explains the methods it proposes to use.
The commenter (6133) expresses that the proposal generally wraps up this section with:
      EPA should collaborate with other federal agencies to identify strategies to protect confidential and private information in any circumstance in which it is making information publicly available. These strategies should be cost-effective and may also include: Requiring applications for access; restricting access to data for the purposes of replication, validation, and sensitivity evaluation; establishing physical controls on data storage; online training for researchers; and nondisclosure agreements.
83 Fed. Reg. at 18,771 (footnote omitted). 
The commenter (6133) contends that the Proposal's many flaws are clear in these sentences. The commenter asserts that the EPA does not know what the Proposal entails. The commenter notes that the Proposal suggests that EPA should identify strategies in the future and that these strategies should be cost-effective. The commenter further notes that the Proposal does not say what cost-effective means, nor what EPA should do if it does not identify any cost-effective strategies, yet it still seeks to alter legal obligations and regulatory decision making in reliance on this unexplained suggestion. The commenter adds that the EPA also does not point to any authority for the proposition that the agency's consideration of peer reviewed scientific studies depends on the cost-effectiveness of some strategy the agency develops for publicizing and protecting the underlying data. The commenter argues that listing options EPA can use does not help. The commenter asserts that the Proposal fails to explain why EPA has not already identified the strategies or options and in what circumstances it would use them. The commenter adds that the Proposal suggests that it will exclude a large class of scientific studies from regulatory decision making but contains a vague assertion that it will look for "cost effective" ways in the future to exclude less them.
Commenter (6133) concludes that the EPA's belief that concerns about access to confidential or private information caused by the Proposal should be addressed in the future is problematic by itself. The commenter provides that the cited materials --  describing ways different organizations can address concerns in different contexts -- do not support this belief. The commenter adds that the Proposal does not propose or analyze any strategies it notes EPA should consider, even though it seeks to implement a binding legal change. While at least EPA recognized the merit to concerns about confidential or private information, the commenter states that in the four days since sending the version for review the agency clearly did not perform the analysis necessary to figure out how those concerns would be addressed. The commenter states that the impact and costs of the Proposal are dependent on such strategies and cannot be measured or analyzed without proposed regulatory text. The commenter contends that the EPA cannot publish a final rule without first proposing what it will do about confidential and private information and analyzing the option it proposes and that the EPA should withdraw the Proposal.
Response: Upon consideration of public comments, the EPA agrees that it is important not to categorically exclude any study because the data underlying a study at the stage required by this rulemaking may not be available for independent validation.. Therefore, the EPA is not finalizing the 2020 SNPRM proposed option that would have categorically required that for studies to be considered pivotal science the underlying data would need to be available for independent validation. However, given that transparency is an important aspect of EPA's regulatory actions and assessments, it should be an important consideration in how the Agency considers pivotal science. As described in 40 CFR 30.5 of the final rule, the EPA will rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies --  based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements or quantitative analyses of an EPA significant regulatory action or influential scientific information will be identified as pivotal science. Further, the EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The EPA would also like to emphasize that the EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule and the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the rule does not require the EPA to collect, store, or publicly disseminate data and models underlying pivotal science.
2.1.7.6 Proposal Should be Revised to Include a Clearly Defined Process for Checks and Balances in the Implementation of the Rule
Comment: Commenter (0562) suggests that the EPA release a revised proposed rule with explicit language as to who would be advising the Administrator on which data or models are valid for decision making, and what form of independent review is needed if they have been exempted under §30.9. According to the commenter, advisors should have the authority or recourse to call into question decisions by the Administrator that are counter to good scientific practice and that result in excluding valid data or models from consideration in a regulatory decision based solely on lack of transparency. The commenter states that the "scientific" criteria used by the Administrator or his or her advisors to make such decisions should be explicitly stated (or referenced) in the revised rule. In other words, the commenter asserts that there should be a clearly defined process for checks and balances in the implementation of the proposed rule to instill trust both within the Agency and for stakeholders, and that decisions on what data or models could or could not be used in a regulatory decision are not the result of the Administrator's personal bias. In the absence of such explicit language, the commenter states that one can only assume that these decisions "could" be made solely at the Administrator's discretion.
Commenter (2169) states that there is no mechanism in the regulation for assuring that the science drives the decision. Therefore, according to the commenter, there is no means for judging the term pivotal. The commenter contends that, without transparent consideration of all the information available, and public comment on whether all pertinent information was considered, it is pointless to promulgate rules on the availability of the information.
Response: In consideration of public comments, the Agency is retaining the Administrator's exemption provision because there are conditions under which compliance with the requirements in 40 CFR part 30 might be impracticable. For example, the underlying dose-response data for some studies, particularly older studies, may not be readily publicly available because of the technological barriers to data sharing (e.g., differences in data storage devices or data retention practices) that existed when they were developed. As a result, the EPA is finalizing the Administrator's exemption provision as proposed in the 2020 SNPRM, with additional conditions described here. 
The EPA does not agree with the comments regarding the role of the Administrator in determining whether to grant an exemption and finds that the Administrator is the appropriate decision maker in this context. To ensure that the Administrator's decision is appropriately transparent, in the final rule the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. This documentation would typically be provided as part of the proposed rulemaking given this will be part of the decision of what is the pivotal science for the rule. Regardless, the final rulemaking will provide clear documentation.
The EPA is also including criteria that may be used by the Administrator when he or she is determining whether greater consideration should be afforded to pivotal science for which the underlying dose-response data are not available in a manner sufficient for independent validation. As a result, the Administrator may determine that greater consideration is warranted when: 
 Technological or other barriers render sharing of the dose-response data infeasible;
 The development of the dose-response data was completed or updated before the effective date of this final rule;
 Making the dose-response data would conflict with laws and regulations governing privacy, confidentiality, CBI, or national security;
 Third-party independent validation of the study's underlying dose-response data through reanalysis has been conducted; or
 The factors in 40 CFR 30.5 used in determining the consideration to afford to the pivotal science indicate full consideration is justified.
To assist the Administrator in determining whether to grant an exemption, the EPA program or Region responsible for the significant regulatory action or influential scientific information and public commenters can provide input when the Administrator is considering an exemption. The EPA will document the rationale for the Administrator's exemption in the significant regulatory action or influential scientific information. The EPA is confident that the above criteria provide sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
Finally, in consideration of public comments, the EPA clarified the definition of pivotal science in the final rule. Rather than considering all studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact as "pivotal science," the EPA is balancing transparency and feasibility by focusing on those studies that inform the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect. Thus, "pivotal science" includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate points of departure (PODs)). These are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. In the context of this rule, studies informing the dose-response modeling (e.g., benchmark response selection), studies identifying data for toxicokinetic adjustments, or studies informing the selection of uncertainty factors do not drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. Future statute-specific rulemakings may interpret "pivotal science" more broadly.
2.1.7.7 Request for EPA to Clarify Why Additional Peer Review of Data is Necessary When it Has Already Been Reviewed or Adequately Describes the Process
Comment: Commenters (5165, 5419, 6125, 6373, 6915, 9227) suggest that EPA clarify why the proposed rule is requiring additional peer review of data and/or present an adequate description of what the peer review process would entail. 
Commenter (9227) notes that the EPA's proposed peer review regulations do not even mention EPA's Peer Review Handbook, let alone explain how the new proposed regulations would impact EPA's compliance with the Handbook. For example, the commenter states that the EPA's Handbook specifies "exemption criteria" in Section 3.3. The commenter requests that EPA clarify whether anything in the proposed peer review regulation would supplant instructions in the Peer Review Handbook, and if so, provide a reasoned explanation for the change. Likewise, the commenter suggests that EPA explain the role of the Peer Review Handbook going forward in administering peer review requirements. Additionally, the commenter notes, the preamble to the proposed rulemaking lacks any explanation whatsoever for why EPA is proposing this new peer review requirement or what its impact might be. The commenter adds that the EPA has additionally not provided any information to suggest that EPA is not already following OMB's Peer Review Bulletin. According to the commenter, EPA's lack of any supporting rationale or analysis frustrates the public's ability to provide meaningful comment on this provision, and is itself a sign that this requirement is fundamentally arbitrary.
Commenter (5165) notes that the proposed rule would require EPA to conduct "independent peer review" of scientific studies underlying its significant regulatory decisions, such as the establishment of health-based air quality standards and that EPA's in-house peer reviewers would also be tasked with articulating "the strengths and weaknesses of EPA's justification for the assumptions applied and the implications of those assumptions for the results." The commenter notes that it is difficult to provide meaningful comments on this aspect of the proposal because EPA has included no details about how the "independent peer review" requirement would be implemented. According to the commenter, the fact that EPA has requested comment on "which parts of the Agency should be responsible for carrying out these requirements" suggests that it has not worked out a plan for this fundamental provision. The commenter inquires: Would EPA have to hire new experts, and if so, how many and in what fields? How much would this cost? More fundamentally, the commenter asks why should scientific literature that has already undergone peer review and been vetted by EPA's science advisory panels be subjected to an additional layer of government peer review? The commenter asserts that these key questions should have been considered, and the answers made public, prior to the rule's proposal.
Commenter (5419) states that §30.7 of the proposed rule states that EPA shall conduct independent peer review of pivotal regulatory science but lacks any detail of this process, including selection criteria for peer reviewers, decision criteria, and how this compares to existing processes through the SAB and third‐party contractor‐managed peer reviews. The commenter requests clarification and additional detail related to this key provision of the proposed rule.
Commenter (6915) notes that the EPA apparently proposes to require that EPA itself conduct an additional "independent" review. See 83 Fed. Reg. at 18774. Yet the commenter notes that the proposal nowhere discusses how EPA would vet reviewers to identify persons who are purportedly more competent than those already used in past or current peer review processes, or the level of EPA staffing and associated costs that would be needed for additional review -- only stating that EPA will implement the proposed rule in a manner "that minimizes costs." Id. at 18774. But, according to the commenter, any requirement for EPA to conduct additional review would entail additional significant costs, contrary to the proposal's assertion. Id. at 18772. The commenter contends that the practical outcome of the proposal is that EPA may end up relying on a much smaller number of studies and/or on a less robust subset of relevant available studies, thus undermining the regulatory decision-making process. In sum, the commenter asserts that the EPA fails to acknowledge the rigor of existing processes in statutes, policies and federal procedures, or to explain how its proposal would provide any added value and minimize costs. EPA should abandon this unnecessary and counterproductive exercise.
Commenter (6125) states that the meaning of proposed §30.7 suggests that EPA will be required to re-peer review every study upon which it relies. The commenter notes that the regulatory text is accompanied by no explanation, leaving commenters to speculate about how it would apply, or what problems its implementation might present. The commenter contends that the proposal's complete failure to explain this far-from-self-explanatory provision is yet another instance of failure to give adequate notice of the substance and basis of the rule.
Commenter (6373) provides that the Rule does not describe how EPA intends on conducting the assessment of models and assumptions described in §30.6 or the independent peer-review process described in §30.7 or identify which individuals would be a part of these reviews. The commenter notes that it is unclear if the peer-reviewers will consist of EPA staff, a group of independent individuals, a single reviewer, or if the peer-reviewers will be required to have varying backgrounds free from personal and professional conflicts of interests to promote a complete and adequate review of the science. According to the commenter, the Rule provides no insight into if, or how, the "independent" analyses for validation would be incorporated into the federal decision-making process, or if such analyses would be required before the findings from peer-review studies could be considered.
Response: As discussed in the preamble to the final rule, the EPA would like to clarify that the rule requirements are not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review does not typically include reanalysis of the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information) and, thus, peer review is not considered a replacement for the data availability requirements of this rule. 
The EPA is, therefore, finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will conduct independent peer review, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review and the exemptions described therein. The EPA will evaluate whether or not to conduct additional peer review, of individual studies identified as pivotal science if the studies have already undergone journal peer review. If the Agency conducts peer review on pivotal science, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied in analyzing dose-response data and the implications of those assumptions for the results.
2.1.7.8 Request that EPA Provide Greater Detail on the Kinds of Research/Data that Could be Used/Excluded from the Rulemaking Process
Comment: Commenters (0556, 2335, 2876, 6357, 6873) request clarity on the kinds of research/data that could be used/excluded from the rulemaking process. Commenter (6357) recommends the EPA work with the scientific community to develop clear processes and inclusion and exclusion criteria regarding studies that can inform health impact, but for which ethical and practical considerations may preclude public release of data.
Commenter (6873) expresses concerns regarding what the proposed rule will actually do (albeit unintentionally) is create a new category of "EPA usable research." The commenter asks the following: 
 What does it mean to make research "usable" or "accessible"? 
 In what format? 
 At what stage of research? 
 Will data standardization be required (otherwise, usability can be quite limited)? 
 On what timetables, with what funding and on whose authority? 
 What protections will be applied for the use and reuse of data? 
 And what might be the potential broader effects on research in general if we don't know exactly which forthcoming studies will be potentially of interest to the EPA? 
 Will this cause all science to shift to an EPA-standard of openness? 
The commenter (6873) notes that the answers to these questions are not simple and will require a significant and lengthy undertaking.
Commenter (6357) encourages the EPA to clarify the process by which the Agency will make determinations about the inclusion of studies that cannot ethically or legally release certain types of data. The commenter notes that the proposed Rule states that "concerns about access to confidential or private information can, in many cases, be addressed through the application of solutions" already in use by the federal government. The commenter asserts that the document does not articulate how current EPA practices differ from other federal agencies' processes, what gaps the proposed EPA Rule would address, and what precisely would be updated or changed at EPA to address those gaps.
Commenter (6358) provides that the proposal states that EPA aims to honor "legal and ethical obligations to reduce the risks of unauthorized disclosure and re-identification" and that concerns about access to confidential or private information can "be addressed through the application of solutions commonly in use," but it does not describe criteria or processes for determining which data sets would be exempted from data release requirements, what alternative requirements might be applied to data sets containing identifiable information, or which activities the Administrator will exempt.
Commenter (0556) provides that EPA needs to address the following:
      First, they have to figure out what "using" a given research result even means. For example, proposed major rules are accompanied by a voluminous Technical Support Document. It may cite hundreds, or even thousands, of research journal articles. Does each of these have to meet the availability and replicability standard? Or is regulatory usage confined to just a few key studies? 
      Second, what does availability mean? For example, does the researcher have to document their data, or just provide it? What about the decisions made as the research progressed, which can be numerous? Does each of these have to be explained? How documented does the software have to be, etc. Here the danger is that the availability requirement might become too burdensome. 
The commenter adds that, assuming that these deep challenges finally get worked out, the EPA needs to consider what the regulation does. The commenter notes that the rule, in effect, creates a new category of "EPA usable research". According to the commenter, researchers (or their institutions) will want their research to be usable, even if EPA does not fund it, but that it will be a big step forward with regards to open science and a big job for the EPA and research community to work out the extension of Public Access.
Commenter (2335) states that there is a need for more details about the kinds of research that could be excluded from the rulemaking process and the kinds of decisions that could be affected.
Commenter (2876) requests that the proposed rule explicitly identify some number of publications with standards that would fail to meet the proposed transparency rules. Further, the commenter suggests that the proposed regulation clearly state how those publications could bring their practices into compliance with the new standards while still meeting legal and ethical requirements for protecting the privacy of research subjects.
Response: In response to public comments, the EPA clarified many aspects of the proposed rule in the 2020 supplemental notice of proposed rulemaking and the final rule, including definitions of key terms and the stage of data required under the rule. As discussed in the preamble to the final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. As noted by commenters, the step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
The EPA also clarified the definition of pivotal science in the final rule. Rather than considering all studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact as "pivotal science," the EPA is balancing transparency and feasibility by focusing on those studies that inform the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect. Thus, "pivotal science" includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). These are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. In the context of this rule, studies informing the dose-response modeling (e.g., benchmark response selection), studies identifying data for toxicokinetic adjustments, or studies informing the selection of uncertainty factors do not drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. Future statute-specific rulemakings may interpret "pivotal science" more broadly.
As also detailed in the preamble to the final rule, underlying data or models are considered available if they can be accessed through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. The EPA also modified the definition of "publicly available" in the final rule to add the following at the end of the definition: "The public must be able to access the information on the date of publication of the proposed rule (or, as appropriate, a supplemental notice of proposed rulemaking, or notice of availability) for the significant regulatory action or on the date of dissemination of the draft influential scientific information for public review and comment."
Finally, the EPA will not be categorically excluding studies based on the availability of their underlying data. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating final significant regulatory actions and finalizing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations. The EPA believes that the implementation of this final rule will increase the transparency of pivotal science for significant regulatory decisions and influential scientific information, while still ensuring that all quality, relevant literature is considered in the development process. The final rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
2.1.7.9 Rule Needs to be Clarified  -  Specific Requests
 2.1.7.9.1 How will the Rule be Applied to Past Regulatory Actions
Comment: Commenter (6447) states that the EPA has not explained whether or how this new policy will be applied to past regulatory actions. If EPA is planning to use this new policy to go back and revisit past regulatory decisions, it should say so explicitly. Commenter (6909) inquires whether the rule can and will be applied retroactively. The commenter states that the EPA has specifically asked for comment from the public as to how it should approach this issue. According to the commenter, this is hugely significant in terms of what the rule's effects will be. 
Commenter (6915) states that the proposal says the rule is intended to apply prospectively, but also states that EPA "should be guided by this policy to the maximum extent practicable during ongoing regulatory action." Id. at 18,771. Yet it never explains how or why ongoing EPA actions would be subject to the proposed rule and which existing scientific studies are implicated by the proposed rule. 
Response: This final rule applies prospectively to significant regulatory actions and influential scientific information and has no retrospective effect on existing significant regulatory actions or influential scientific information. As described in 40 CFR 30.3, the rule provisions apply to significant regulatory actions for which a proposed rule was published in the Federal Register after the effective date of this rule and influential scientific information submitted for peer review after the effective date of this rule. This final rule is effective upon publication in the Federal Register.
2.1.7.9.2 Specific Language of Proposed Rule that is Unclear
Comment: Commenter (6120) states that the language in the proposed rule is not clear. For example, the commenter notes that the proposed rule states that the "EPA should give appropriate consideration to high quality studies that explore: a broad class of parametric concentration-response models with a robust set of potential confounding variables; nonparametric models that incorporate fewer assumptions; various threshold models across the exposure range; and spatial heterogeneity." The commenter states that this type of statement needs to be described more explicitly. The commenter inquires: How does the EPA define "high quality studies"? The commenter requests that the language that describes the models should be written in language that is clear to the public and gives examples of what is truly being referred to here. The commenter contends that this type of language is found throughout the proposed rule. The commenter encourages the EPA to clarify the text throughout so that each proposed action is clear and easy to understand by everyone. 
Commenter (6194) asserts that key issues beyond definitions also raise issues in such a general manner that it fails to provide the public with adequate notice of what EPA intends. For instance:
 The Proposal requires that certain data and models be made "publicly available in a manner sufficient for independent validation," and indicates that information will satisfy this standard when "it includes the information necessary for the public to understand, assess, and replicate findings." 83 Fed. Reg. at 18,773-74. But it is not clear what degree of availability or explanation would satisfy the Proposal. For example, the "public" referenced in the Proposal could be scientists trained in the relevant field or members of the general public.
 The Proposal provides that, "where necessary, data would be made available subject to access and use restrictions." Id. What this process would look like is not addressed in the Proposal or discussed at all in the preamble. This leaves significant open questions, such as who will pay for and implement these restrictions and who will decide which people are allowed to access the data and under what standards.
 The Proposal appears to require EPA to conduct independent peer review of all "pivotal regulatory science," but provides no detail as to what this means or how it will be implemented. Id. For example, it is unclear who within the agency would perform this peer review, when it would occur, and whether (and when) prior peer review in the journal publication process would serve as a substitute for EPA peer review.
 The Proposal includes an exemption mechanism but does not provide information about the process or conditions under which exemptions shall or can be sought and granted.
Response: The preamble to the final rule clarifies that the requirements in this final rule set the overarching structure and principles for transparency in the EPA's final significant regulatory actions and influential scientific information. The EPA intends to develop internal implementing guidance and promulgate either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule, which will clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes that the EPA administers.
In the final rule, the EPA clarifies how the Agency will evaluate the quality of scientific studies when promulgating significant regulatory actions and developing influential scientific information. As described in the preamble to the final rule, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information, consistent with the EPA Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental Protection Agency. The EPA will generally continue to use the following, established factors when assessing study quality: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. 
The rule also clarifies other general aspects noted by the commenters. For example, criteria that the Administrator would consider in determining whether to grant an exemption to the provisions in this rule are provided in 40 CFR 30.7. As described in the preamble, the rule also does not direct or require any outside entity or the EPA to establish data sharing mechanisms, nor does it require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science. The EPA is only required to determine if such a mechanism was available, should a subject matter expert decide to use it. The EPA also modified the definition of independent validation to now mean the reanalysis of study dose-response data by subject matter experts who have not contributed to the development of the study to evaluate whether results similar to those reported in the study are produced. The preamble to the final rule also provides clarity regarding peer review of pivotal science.
2.1.7.9.3 Need for Greater Clarification on Data Refinement Issues
Comment: Commenter (6362) notes that an important aspect relevant to "public availability" is the level of data refinement EPA will require. The commenter notes that the NAS held a workshop in 2016 to discuss obstacles for sharing data and they defined several key terms to ensure clarity at the workshop. The commenter recommends that the EPA consider adopting a similar lexicon to increase the clarity of its regulation (and refers to Table 1 in Appendix B). Definitions in NAS Principles and obstacles for sharing data from environmental health research: Workshop summary) of their comment letter). In addition, the commenter provides that the NAS Report suggests a "cleaned dataset" would be acceptable to use for all routine analyses and verification (and refers to Table 2 in Appendix B (Data flow from NAS Report) of their comment letter). EPA should establish clear standards on the acceptability of "cleaned datasets." According to the commenter, this will help to standardize data reporting and formatting. It will also prevent over- and under-reporting.
Response: In this final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. As noted by commenters, the step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information. The EPA does not agree with establishing standards for the acceptability of data independent of the definition provided in the final rule because additional standards would be overly prescriptive and not informative.
2.1.7.9.4 Implementation Requirements Vague/Insufficient
Comment: In addition to comments on the overall general vagueness of the proposal, several commenters specifically request/point out the deficiencies in requirements set out in the Proposal that would allow them to properly comment on and implement the rule. Comments include:
 The proposed rule lacks clarity as to how the rule would apply and how EPA intends to implement the rule. (2876, 5165, 6351, 6920)
 For those researchers who do attempt to comply with the proposed rule's requirements, the extent and nature of the data that must be maintained and made publicly available is vague and unclear, making compliance virtually impossible. (6915)
 The proposal lacks details on how its provisions would be implemented, specifically when it comes to privacy issues, reproducibility, and possible exemptions. (6110)
 The rule as written is too vague with regards to implementation, which leaves far too much editorial discretion up to the politically appointed administrative roles on what constitutes "transparent" study data and model design. (6127) 
 The Proposal offers no process for public hearing or even consultation with the SAB over implementation. As written, the commenter notes that the Proposal would require review and assessment of volumes of existing research and revisions to internal processes yet to be determined and seems to give arbitrary decision-making authority to the Administrator to determine the fate of such research. (6351)
 The lack of clarity regarding how the EPA will implement the proposed Rule risks exclusion of scientifically valid and relevant studies during the Agency's decision-making process. The commenter asserts that the rulemaking process itself benefits from the same transparency, consistency, and objectivity. As such, they suggest that clear guidelines around how rules are implemented are as important as data transparency in ensuring public trust and an objective approach that prioritizes the EPA's mission of protecting human and environmental health. (6357)
 Commenter (6916) assumes (as they say they must, given no other information from the Agency), that the Proposal is aimed at precluding EPA's reliance on studies for which the data are not made public. The commenter notes that is the natural reading of the proposed regulatory language, which is the most choate information offered in the Proposal. But even that reading is not certain, and the commenter states given the very broad discretionary nature of the exemptions provided for in the rule text (see, e.g., 83 Fed. Reg. 18774, proposed 40 C.F.R. §30.9, giving the Administrator the ability to grant an exemption to the data publication requirement if that is "impracticable," which is undefined in the Proposal). As a result, the commenter states that it is unclear how this rule would apply to EPA decision making, and how it would be different than the Agency's current protocol for considering studies relevant to specific rulemaking proposals. (6916)
 While asserting an interest in "ensur[ing] that the data and models underlying science is [sic] publicly available in a manner sufficient for validation and analysis," the commenter notes that there is no more specific statement explaining how that would be implemented. (6916)
 The proposal also omits critical protocols and methodologies necessary to put this rule into action and does not fully explore the implications of implementing a rule of this nature. (9224)
Commenter (6046) contends that several of the requests for comment topics should have been evaluated and decided upon by EPA and then incorporated into the proposed rule. The commenter adds that just throwing out a rule without tools to implement the rule is inefficient and a waste of time. For example, the commenter notes that the EPA requested comment on the following: "EPA solicits comment on implementation of the proposed regulation, including which parts of the Agency should be responsible for carrying out these requirements." The commenter states that the EPA best knows which part or if multiple parts of the Agency should be responsible. Further, the commenter notes that a request for comments on implementing the regulation is analogous to saying that our agency's goal is to fly every person to the moon and then ask how the agency should do it. 
Commenter (6104) raises questions that states believe should be considered before the rule is finalized. These questions include:
      1. How would the $100 million economic threshold analysis be implemented? Would new rules be assessed differently than updates to current rules?
      2. Where data masking, coding, or de-identification is not technically feasible, will EPA still consider using high quality scientific research?
      3. While it appears this rule would only apply prospectively to regulations, would there be any impact to science historically used to inform regulations that have existed for years/decades? How will EPA ensure the science remains timely?
      4. If EPA were to phase in the requirements or prioritize certain specific actions, will states have any role in helping identify those priorities?
      5. What impact, if any, would this rule have on institutions of higher education, hospitals, and other nonprofit organizations that received EPA funding through grants and cooperative agreements?
Commenter (6109) states that the EPA should provide the public with the answers to the very questions it poses, including:
 When or how it will develop procedures for protecting private information such as patient health records of the sort currently kept confidential by researchers;
 How the proposed rule will be applied to its regulatory actions and under what circumstances;
 Whether other regulations might be necessary to implement the proposal, or whether statutory changes would be needed;
 Whether and how the agency will establish exceptions, including whether it will provide case-by-case exceptions;
 Whether the proposal will apply to a broader or narrower set of regulatory proceedings than it would as currently drafted, or whether certain categories will be exempt; and
 Whether the agency will add stronger data access requirements to the terms of research grants
Commenter (6133) notes that proposed §30.7 states that "EPA shall conduct independent peer review" without providing any coherent explanation or accompanying regulatory text about what that means: how will that peer review be conducted? By whom? Who will select the peer reviewers? How many will there be? Who will assure their independence and expertise? Will peer reviewers be subject to federal conflict of interest rules and policies? Will peer reviewers be anonymous? Where will the funds come from to conduct EPA peer reviews for "all pivotal regulatory science"? Has EPA estimated how many instances of "pivotal regulatory science" it anticipates conducting peer review for in one year? In prior years? Will the peer review be conducted openly and publicly? Will it be conducted in accordance with the Federal Advisory Committee Act? What will the duration of any peer review be? What purpose will that peer review serve? How will it affect future regulatory decisions? Or will it? Will there be an administrative docket? Will any product of the peer review be included in the administrative dockets for rulemaking? Will peer reviewer comments be part of the certified record for judicial review? Will the agency seek deference from future reviewing courts for the views expressed by peer reviewers? Does EPA not believe that peer review conducted by professional journals and societies is valid? Or sufficient? On what basis does EPA think professional peer review is invalid or insufficient, considering there is not one iota of evidence or support for that belief in the Proposal or the accompanying docket? What is the basic justification for proposed §30.7? The commenter asserts that the Proposal provides no answers to these questions.
Commenter (4831) suggests that the potential impacts of the proposed rule on the quality of regulatory science will depend on many aspects of the rule's implementation that are not described in detail in the Federal Register notice, including the following:
 Criteria and processes to make objective and transparent decisions about which studies will be included in scientific analyses used to inform federal regulations;
 Approaches for evaluating the data and models used to characterize the dose-response relationships underlying federal regulations; and
 Approaches for protecting the confidentiality of certain kinds of data while balancing the need to make data publicly available. 
Response: As discussed in the preamble of the final rule, the EPA intends to develop internal implementing guidance and promulgate either statute-specific science transparency regulations or programmatic guidance implementing this procedural rule, which will clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes EPA administers. The final rule also provides more detail regarding selection of pivotal science, consideration of restricted access to underlying data to protect privacy, criteria for the Administrator's exemption, and peer review. 
In brief, the EPA will continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies --  based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements or quantitative analyses of an EPA significant regulatory action or influential scientific information will be identified as pivotal science. In compliance with the rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or propriety information, available through restricted access in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science is unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. The EPA provides these factors in 40 CFR 30.5(d) and they are discussed more fully in the preamble of the final rule. The EPA will comply with OMB guidance on peer review for pivotal science and will evaluate whether or not to conduct additional peer review of individual studies identified as pivotal science if the studies have already undergone peer review. Criteria for the Administrator's exemption are provided in 40 CFR 30.7.
The rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
Comment: Commenter (5181) states that they find it difficult to predict the effect of the proposed transparency rule given the broad and general nature of the described intent and applicability, and lack of information regarding how it would be implemented in practice. According to the commenter, it is impossible based on the information provided in the Federal Register notice to fully evaluate the potential demand on time and agency resources, and to ascertain the benefits or impediments that might result from the proposed regulation. The commenter expresses concern that the notice fails to provide sufficient detail for an analysis of whether the new approach will achieve the stated purpose without creating unintended consequences that could make it difficult for states to implement clean water programs.
Response: The EPA agrees that neither the 2018 proposed rule nor the 2020 SNPRM included a characterization of costs to the Agency. The EPA emphasizes that this is a rule of internal procedure promulgated under the EPA's housekeeping authority. However, the EPA has identified some incremental costs that the Agency may incur as a result of this final rule. 
As detailed in the preamble to the final rule, the EPA will continue its current practice of conducting extensive review of scientific studies during the development of significant regulatory actions and influential scientific information. The additional procedures required by this rule only apply to pivotal science, which is a subset of the total number of studies that the EPA would evaluate. Given the costs of the current robust process for identifying and reviewing scientific studies and documentation that are existing Agency practice, as well as that the determination of dose-response data availability is limited to pivotal science underlying significant regulatory actions and influential scientific information, the EPA anticipates that the incremental costs of this rule will be small. The Agency may also incur other administrative costs to perform analyses and evaluations to support activities such as exemption decisions made by the Administrator, and documenting these or other decisions made pursuant to the requirements of the final rule. Again, the Agency anticipates that the incremental costs for these activities will be small relative to current administrative costs for developing significant regulatory actions or influential scientific information. Finally, this final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to pivotal science for which the underlying dose-response data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
The EPA also agrees that the benefits of the rule were not fully characterized in the 2018 proposed rule or the 2020 SNPRM. The EPA emphasizes, however, that this is a rule of internal procedure promulgated under the EPA's housekeeping authority. As discussed in Section III.A.1, the main benefits of this rule spring from greater transparency in significant regulatory actions and influential scientific information. By placing greater emphasis on the availability of dose-response data underlying pivotal science, the rule will allow for greater scientific scrutiny as EPA decision makers are developing significant regulatory actions and influential scientific information and increases the likelihood that any errors will be identified and corrected. Greater transparency is also inherently valuable as a principle of good government and provides benefits to the public at large, including reducing the risk of errors in EPA analyses and in the science such analyses rely upon. The ability for independent subject matter experts to validate pivotal science will facilitate more effective comment and engagement with the public during development of future significant regulatory actions and influential scientific information.
2.1.7.9.5 Definitions
Comment: Commenters (2426, 3971, 5170, 6125, 6126, 6133, 6137, 6194, 6351, 6360, 6361, 6362, 6373, 6447, 6449, 6875, 6910, 6915, 6924, 6955, 8281-PH20, 9224) express that definitions are not defined or ill-defined. Comments include:
 Commenter (6373) asserts that the impacts of the proposed changes will be difficult to predict unless EPA better defines what data the Rule applied to, and terms such as "independent validation." Commenter (6126) suggests that after EPA clarifies definitions, a new round of public comment will likely be needed to address the basic questions raised by EPA in the proposed rule on other matters.
 The EPA solicits comment on the definition of "pivotal regulatory science" and the "dose-response data and models" and how to implement such definitions" (page 13 of 27; second para). The commenter responds to this solicitation by stating that this question is not trivial, but it can be resolved by assembling small groups of dedicated scientists who will invest the time to share information with the Agency. The commenter notes that EPA did this on many occasions in the 1980s and 1990s when they were working on the cancer risk assessment and exposure assessment guidelines. According to the commenter, they participated on several highly productive SABs that saved the Agency tens of thousands of wasted hours. (3971)
 There are multiple terms that will need to be defined in the final rule. In particular, they suggest that "transparency"; "data", singular versus set; and "reasonable effort/endeavor", in relation to how much work the agency must put in before justifying the use of data that cannot be made available to the public. The commenter suggests that if the agency wants a rule focused on the raw data then EPA must better define what "data" would be included within this rule. The commenter inquires: Is it just data produced by EPA? If not, is it the methods and protocols or the actual raw data? (6875)
 The terms "pivotal regulatory science," "replication," "reproducible," and "research data," are not defined or are problematic. (6924, 8281-PH20)
 Neither the proposal, nor the legislation it is based on, define "reproducible," or "replicable" as used in the rule and are in fact not clear on how broadly this goal might be applied. §30.5 (information must be sufficient to "replicate" findings). The commenter notes that under standard usage, a study can be replicated without access to the underlying data. (6125)
 The proposed §30.2 refers to "pivotal regulatory science as the studies or analyses that drive the requirements and/or quantitative analysis of EPA final significant regulatory decisions." The commenter adds that this definition is distinguished from "regulatory science," defined as "scientific information, including assessments, models, criteria documents, and regulatory impact analyses, that provide the basis for EPA final significant regulatory decisions." According to the commenter, these two definitions can be interpreted as simultaneously referencing something identical as well as one being a subset of the other. Therefore, the commenter contends that the definitions are vague and need clarification. (6362)
 Commenter suggests that the Agency clearly communicate its expectations regarding "replicability", "validation" and availability of "data" to scientists designing and publishing their studies. (6449)
 The Proposal argues that research must be "replicable" without defining what that means. The commenter provides that many studies cannot be specifically repeated, especially those that examine the impacts of historic events, such as the exposure of a half-million Americans to no longer-existing levels of air pollution, or the health effects stemming from a massive oil spill. The commenter notes that subsequent, similar studies from around the world have echoed their findings on health impacts. The commenter asks "[w]hich concept would EPA consider as replication?" (6351)
 Commenter expresses that the definition of the goal of study validation, is missing and no indication is given as to how public access is actually expected to produce validation. The commenter states that, based on the top-down nature of the proposal and the views of the former EPA Administrator who promoted it, the commenter notes that it is possible to guess at the unspoken definition: a study is validated if no stakeholder or true believer in alternate models can find anything to criticize about it after a (post-hoc) analysis of a study's data, which the commenter states is an impossible standard to meet. (6955)
 Although the proposed rule includes a definition for "pivotal regulatory science," the definition is useless. As defined, the commenter states that "pivotal regulatory science" could be as narrow as the study or studies from which a specific point of departure was selected, or as broad as all information considered by EPA in a risk assessment or rulemaking. The commenter notes that the public has no way of knowing how narrow or broad EPA intends this proposal to be. (6447)
 The proposed rule would require the Agency to make certain data and models are "publicly available in a manner sufficient for independent validation." Proposed §30.5. The commenter provides that, as the SAB notes, the concept of "validation" is not clearly defined in the rule. Generally speaking, the commenter contends that due in large part to the vague language used, the proposed rule gives EPA broad discretion to suppress science. The commenter expresses that this appears to be the whole point of the proposal. (6447)
 Where possible, the EPA should be explicit about important definitions, including "publicly available," "pivotal regulatory science," "dose response data and models," and "validation." (2426)
 Definitions that would be helpful in framing the proposed rule include "pivotal regulatory science," "dose response data and models," and "validation." According to the commenter, although "pivotal regulatory science" is likely intended to mean just the key study or studies underlying a regulatory analysis, it could be argued that supporting studies (i.e., those that help to build the weight of evidence or describe the mode of action) are also pivotal to the analysis. In addition, the commenter notes that it is unclear what the term "dose response data and models" includes. The commenter suggests that this term could mean anything from summary data to specific calculations and code, which matters for the potential sensitivity of the information. Finally, the commenter recommends that the EPA define what it means to "validate" study results. The commenter contends that this could be interpreted to mean that the study must be repeated (i.e., a new group of human subjects are exposed in a controlled experiment or are assessed in an observational setting) or simply that the statistics and models are recalculated from the existing data. Further, the commenter suggests that the EPA determine what level of validation is necessary for pivotal regulatory science so that it can be consistently applied. (2426)
 The term "independent validation" is insufficiently defined in the Rule. The commenter provides that validation typically refers to the ability to replicate a study's results, based on the published methods and using the original data. The commenter notes that, to be independent, validation must be performed by groups without conflicts of interest related to the exposures and outcomes of the original research and its findings. According to the commenter, independence is not clearly required by the Rule, and therefore "independent" could be interpreted to require only that validators be external to the original research study team. The commenter contends that, by not clearly defining independence, the Rule creates the opportunity for groups with strong economic conflicts of interest related to environmental regulations to try to undermine scientific processes related to validation. The commenter asserts that, unless the Rule clarifies the process for the "independent validation" and who would conduct it, such "validation" may lead to an analysis of poorer scientific quality than the original study. (6373)
 The proposal does not indicate how "validation" will occur. The commenter notes that scientists familiar with these types of studies are also familiar with the protocols under which these studies are carried out and the studies are peer-reviewed, providing an independent, assessment of the studies. EPA has not identified any deficiencies with the current, time-tested approach, and thus, EPA's desire for "independent validation" is unclear how EPA believes it would be best suited to provide this validation and how this independent review is completed. (6361)
 The commenter provides that outside the context of this proposed rule, "publicly available" generally means that information is available without restriction to anyone who wants it. According to the commenter, while this level of accessibility is acceptable for most information, it becomes problematic for sensitive personal information, personal health information, or CBI. The commenter recommends that the EPA strongly consider implementing a formal request and approval process for qualified researchers to obtain access to data that is protected by confidentiality laws, rather than aiming for all data to have unrestricted access. According to the commenter, this process would still allow identifiable or fairly refined de-identified data to be re-evaluated by experts external to the EPA or the original study authors, while protecting the privacy of the study participants or businesses who created the model or dataset. (2426)
 The proposed rule is lacking in defining the boundaries of the terms used in section I. General Information, including, " strengthen ," " transparency ," " publicly available ," and " inform ." The commenter encourages the EPA to revisit these terms and discuss how the public will benefit from this new proposed rule. The commenter raises the following questions regarding the ambiguous nature of the listed terms:
 How would this new transparency initiative change things? Is the proposed rule referring to research, general science, or regulation outreach?
 What specific things are happening now that are not considered to be transparent in nature?
 How will resources become more publicly available? Will there be more online outreach regarding policy & regulations? Will there be additional supporting documents and resources for pivotal regulatory science research? How will this be more publicly accessible? (6910)
 The proposed rule lacks definitions as well as key implementation and operational information, including critical information about what it would cover and how it would be implemented. The commenter states the following terms need to be explained and operationalized:
 replication
 to the extent practicable
 minimizes costs
 not feasible. (5170)
 The proposal lacks definitions for many key terms. The commenter suggests, in particular, "transparency"; "data", "singular" versus "set"; and "reasonable effort/endeavor", be defined in relation to how much work the agency must put in before justifying the use of data that cannot be made available to the public. The commenter adds that EPA should also clarify what it considers to be "publicly available". (9224)
 The purported need and objective of the Proposal is couched in references to "transparency" and "validation" but that these terms are not defined in the Proposal, nor do they have a single definition amongst professionals in the fields impacted by the Proposal. The commenter adds that the Proposal also improperly uses distinct concepts, such as "reproducibility" and "replicability," as though they are interchangeable. (6194)
 The proposal defines "pivotal regulatory science" as "the specific scientific studies or analyses that drive the requirements and/or quantitative analysis of EPA final significant regulatory decisions." 83 Fed. Reg. 18773 (April 30, 2018). The commenter notes that the proposal does not specify to what extent studies must support regulatory decisions to be considered a "driver," who will determine what qualifies as pivotal regulatory science, or at what stage of the rulemaking process such determinations will be made. The commenter adds that the proposal is also unduly vague in its use of undefined terms that are subject to interpretation, such as the use of the term "uncertainty" in §30.6 of the proposed rule. 83 Fed. Reg. 18774. (6915)
 Commenter (6137) notes that the Proposed Rule defines "regulatory science" as "scientific information . . . that provide the basis for EPA final significant regulatory decisions." 83 Fed. Reg. at 18,773. The commenter states that this definition lacks any discernible meaning. According to the commenter, there is no standard or definition for determining when scientific information does or does not "provide the basis" for a regulatory decision, nor are there any standards or definitions for what subset of "regulatory decisions" should be deemed "significant." Likewise, the commenter notes that the Proposed Rule defines "pivotal regulatory science" as studies that "drive the requirements and/or quantitative analysis" of EPA's action, id., but the commenter notes that this too lacks any understandable meaning. The commenter states that there is no standard governing what science does or does not "drive" the regulatory action. The commenter contends that the lack of regulatory standards here means that EPA staff will impermissibly determine applicability of the rule "based upon their own unwritten personal standards." White v. Roughton, 530 F.2d 750, 754 (7th Cir. 1976).
 Commenter states that, while proposed §30.2 specifies that "all terms not defined herein shall have the meaning given them in the Act or in subpart A," the proposal nowhere says to what "Act" it is referring. The commenter states the proposal purports to implement multiple Acts administered by EPA, with different terms and definitions and court interpretations that may contradict one another. The commenter states that nowhere does the proposal square this factual and legal reality with structure of its unlawful approach, and the language in proposed §30.2. The commenter states that, in the Proposal at issue here, it is incoherent and internally inconsistent across the different statutes that EPA administers. The commenter states it also is not clear to what "subpart A" EPA is referring, because there is no citation to the Code of Federal Regulations. If this is intended to reference 40 C.F.R. Part 30, Subpart A, that Subpart was removed from the C.F.R. in 2014. See 79 Fed. Reg. 75,871; see also 80 Fed. Reg. 61,087. (6133) Similarly, one commenter states that §30.2, addressing definitions, unhelpfully provides that all undefined terms "shall have the meaning given them in the Act or in subpart A," without identifying either the Act or subpart A. (The commenter notes that these are yet other examples of failure to notify the public of the substance and basis of the rule). (6125)
Response: Based on a consideration of this and other similar comments, the EPA clarified and modified key definitions in the supplemental notice of proposed rulemaking and final rule. The EPA considered comments received on the 2018 NPRM and 2020 SNPRM when finalizing the definitions included in the final rule. 
Comment: Commenter (6360) expresses that the EPA could fundamentally improve the rulemaking through more precise definitions in both the preamble and regulatory text states that the rule's definitions should start with grounding EPA's rulemaking in fundamental terms. The commenter provides suggestion for definitions for what they believe are the most foundationally important terms here:
 Science. The study of nature as it is. Science can also refer to the collection of scientific facts and relationships between these facts. For example, biology is a branch of the study of nature that is concerned with living organisms.
 Scientific fact. A clam about nature that is objective and reproducible. A scientific fact must be transparent to be objective  -  no matter who the viewer or experimenter is, the same result of nature is observed. Objectivity is a necessary foundation for reproducibility. If a claim is not transparent enough to be objective or reproducible, it is not a scientific fact.
 Applied Science. The use of scientific facts to create a tool. Fields like engineering, toxicology, economics use scientific facts to construct models, cell phones, bridges, dose-response relationships, markets, and many other things that give us longer, happier, and safer lives. 
According to the commenter (6360), the EPA's rulemaking's scope is applied science. The commenter notes that the EPA must use tools developed by the applied sciences to achieve its statutory responsibilities. Traditionally, according to the commenter, terms important to EPA's purview such as "best available science" encompass both scientific facts and applied science constructions. The commenter asserts that choosing among different models, dose-response relationships, or mousetraps involves judgement -- e.g., what is "available" and what is "best?" The commenter states that this judgement is independent from the question of whether a claim is a scientific fact. The commenter asserts that error is the choice to apply it where it is not useful and that the practical question is EPA's judgement. The commenter states that using the word "judgement" for EPA's application of science carries with it a minimum expectation of transparency in our society. We expect almost all our government decisions of judgement to be open and to be able to be observed by all citizens. The commenter asserts that trials, notice-and-comment rulemakings, permits, records of decision, and many government judgements contain opportunities for public comment and levels of transparency.
Response: Based on a consideration of this and other similar comments, the EPA clarified and modified key definitions in the supplemental notice of proposed rulemaking and final rule. The EPA considered comments received on the 2018 NPRM and 2020 SNPRM when finalizing the definitions included in the final rule. 
The EPA does not find it necessary to include definitions for the terms identified by the commenter but agrees with the commenter's general observation of the need for transparency in government decisions. The EPA believes the final rule will result in final significant regulatory actions and influential scientific information based on the highest quality studies that maximize transparency.
2.1.7.10 Proposed Rule Requirements are Based on a Fundamentally Invalid Assumption
Comment: Commenter (4589) notes that the proposed rule makes a fundamentally invalid assumption that the creation of public policy typically has been and should be based on specific studies targeted to policy application. According to the commenter, this is an unwarranted assumption because it suggests that individual studies must be policy-relevant and therefore subjected to a standard of evaluation laid down by a risk manager that is different from the study's authors or other scientists. The commenter contends that the proposed rule appears to ignore the practical problems of conducting human epidemiologic studies, which are often of value only in retrospect, especially toxicological studies dealing with exposure-response relationships and low-dose extrapolation.
Response: As described in the preamble to the final rule, the EPA is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. The EPA is focusing on the underlying dose-response data for this rulemaking because of the influence these data have on particularly impactful decisions at the Agency. Risk assessments and regulations that target emissions and risk reduction of one or more pollutants, contaminants, or substances are integral to the Agency's mission and the underlying dose-response data that inform the quantitative value used to evaluate and mitigate potential risk are critical to understanding the assessment or regulatory action. In addition, the data underlying the dose-response assessment are more distinct than the broad range of data informing an entire risk assessment. Therefore, the EPA is concentrating its current efforts to increase transparency on a well-defined step in the quantitative assessment of risk supporting specific Agency actions. This final rule provides an important step in furthering the progress already being made toward maximizing transparency and will provide important insight for developing future statute-specific requirements.
2.1.7.11 The Process EPA Used to Develop the Notice of Proposed Regulation Lacked Transparency, and Thoroughness
Comment: Commenters (0055, 0057, 1742, 2907, 3644, 3971, 4594, 4831, 4894, 5022, 5165, 6115, 6119, 6125, 6126, 6144, 6194, 6373, 6879, 6894, 6897, 6904, 6924, 8272, 8281-PH39, 9227) express concern that the process EPA used to develop the NPRM lacked transparency and thoroughness. 
Commenter (6125) asserts that the substantive concerns about inadequate justification can only be magnified by the process EPA followed in developing the proposed rule, which was anything but transparent and indicates that the proposal was not the result of reasoned consideration. The commenter notes that, even their imperfect knowledge of the speed with which this proposal was put together, and the specific steps taken to develop it, indicates a complete departure from approaches that have been used by all past EPA Administrations for developing rules, policies, and procedures that involve the assessment and use of scientific information.
Commenter (6126) provides that the Commission on Evidence-Based Policymaking, established through bipartisan legislation in 2016, recently completed a lengthy fact-finding and deliberative process, followed by the release of unanimous recommendations relevant to EPA's proposed rule. The commenter states that the commission's fact-finding process included seven public meetings, three public hearings, a Federal Register request for public comment, a survey of federal offices, and meetings with representatives of more than 40 organizations. According to the commenter, this thorough effort to understand the implications of using data along with the necessary protections for confidential data -- which included participation from researchers, former program administrators, privacy experts, and dozens of federal agencies -- demonstrates the importance of giving this regulation due consideration with extensive stakeholder feedback. The commenter contends that the commission's research process itself could serve as a valuable model for EPA in taking additional steps to develop this proposal.
Commenters (0046, 1689, 5178, 6155, 6918, 8272) suggest EPA treat the proposed rule as an ANPRM. Commenters (6155, 6918) recommend that EPA review the comments it receives on the current proposal and then issue a new or supplemental proposal with revised proposed regulatory language that reflects the full range of issues on which the Agency has sought comment. 
Commenter (1689) joins others in asking EPA to withdraw this proposal and re-propose the rule after considering the many comments received, rather than proceeding to a final rule. The commenter states that EPA's request for comment "on whether the disclosure requirements applicable to dose response data and models in the proposed rule should be expanded to cover other types of data and information, such as for example economic and environmental impact data and models that are designed to predict the costs, benefits, market impacts and/or environmental effects of specific regulatory interventions on complex economic or environmental systems "is more like an ANPRM than a proposed rule. The commenter suggests that EPA treat the Proposed Rule as an ANPRM, and that EPA review the comments it receives on the current proposal, then issue a new or supplemental proposal with revised proposed regulatory language that reflects the full range of issues on which the Agency has sought comment.
Commenters (5178, 8272) urge EPA to undertake a more thorough process to identify if problems with data transparency exist that impede health protections, including by issuing an ANPRM and requesting a study by the National Academies. The commenters state that, if a problem is identified, EPA should then issue a proposed rule that includes a full analysis of costs and benefits. 
Commenter (6918) suggests that, because the Proposed Rule identifies a number of open questions, many of which address core issues, EPA should evaluate and reach decisions on before issuing a new or supplemental proposed rule. The commenter states that the ANPRM approach would allow the Agency to address some lack of clarity in the language of the proposal.
Commenter (6918) expresses that, given the breadth of the request for comments in the preamble as compared to the proposed regulatory language, they suggest EPA consider treating the Proposed Rule as an ANPRM. The commenter recommends that EPA review the comments it receives on the current proposal and then issue a new or supplemental proposal with revised proposed regulatory language that reflects the full range of issues on which the Agency has sought comment. The commenter notes that the Proposed Rule identifies a number of open questions, many of which address core issues that EPA should evaluate and reach decisions on before issuing a new or supplemental proposed rule.
Commenter (6918) asserts that the ANPRM approach would allow the Agency to address some lack of clarity in the language of the Proposed Rule. For example, the commenter states that the heading of proposed 40 C.F.R. §30.7, "What role does independent peer review in this section?" seems to be missing a verb. See 83 Fed. Reg. at 18774. In addition, the commenter provides that some of the citations in the authority section of the proposed rule text, id. at 18773, appear incorrect. The commenter expresses that Section 115 of CERCLA is codified at 42 U.S.C. § 9615, not at 42 U.S.C. § 9616, as indicated in the proposed regulatory language; Section 7009 of the RCRA (or the Solid Waste Disposal Act), 42 U.S.C. § 6979, concerns labor standards; Section 8001 of that act, 42 U.S.C. § 6981, which concerns "Research, demonstration, training and other activities," would appear to be a more appropriate citation as providing authority for the Proposed Rule.
Response: Based on a consideration of these and other similar comments, the EPA issued a SNPRM in 2020, which included clarifications, modifications, and additions to certain provisions proposed in the 2018 NPRM and allowing for additional public comment. Public comments received on both the 2018 NPRM and the 2020 SNPRM were considered in the development of the final rule. In addition, as detailed in the preamble to the final rule, the EPA is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. The EPA is focusing on the underlying dose-response data for this rulemaking because of the influence these data have on particularly impactful decisions at the Agency. Risk assessments and regulations that target emissions and risk reduction of one or more pollutants, contaminants, or substances are integral to the Agency's mission and the underlying dose-response data that inform the quantitative value used to evaluate and mitigate potential risk are critical to understanding the assessment or regulatory action. In addition, the data underlying the dose-response assessment are more distinct than the broad range of data informing an entire risk assessment. Therefore, the EPA is concentrating its current efforts to increase transparency on a well-defined step in the quantitative assessment of risk supporting specific Agency actions. This final rule provides an important step in furthering the progress already being made toward maximizing transparency and will provide important insight for developing future statute-specific requirements. In addition, the EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule.
Commenter (9227) provides that Executive Order 12866 requires "an assessment of the potential costs and benefits of the regulatory action" in addition to other analyses. As indicated above, the commenter states that the Proposal gives scant consideration to the costs of the proposed action. 
Commenter (6126) notes that reports that EPA did not follow the agency's own internal guidance for development of regulations through the Action Development Process (ADP) are alarming. As the ADP states, the commenter provides that the guidance helps to "ensure that Agency actions are of consistently high quality, involve senior managers early in the development process, are supported with strong analysis, and are developed via an open process." According to the commenter, failure to engage in thoughtful policy discussion with EPA's career senior leaders likely resulted in a less well-developed and lower-quality proposed regulation than the agency could have otherwise put forward for public debate. Altogether, the commenter states that there is an irony in EPA's rationale that the proposed rule is intended to promote transparency while the very process used to develop the regulation lacked transparency and engagement.
Commenter (8272) asserts that the EPA's process for promulgating this rule represents a gross deviation from the agency's typical action development process. The commenter expresses that the EPA failed to perform an appropriate regulatory impact analysis, despite indications that implementation costs could be substantial. The commenter adds that the EPA does not appear to have met with outside groups during the development process, or with its SAB, which states that the rule "deals with a myriad of scientific issues for which the Agency should seek expert advice from the Science Advisory Board."Taken together, the commenter states that these deviations from typical process leave EPA unable to consider its proposal accurately, and constitute an arbitrary and capricious approach to rulemaking.
Response: Public input was an important part of shaping the final rulemaking. The EPA accepted public comment on the proposed rule from April 30, 2018 to August 16, 2018, and held a public hearing on July 17, 2018. In response to public comments, the EPA released a supplemental notice of proposed rulemaking and took public comment from March 18, 2020 to May 18, 2020. All of these comments were considered in developing the final rule.
The EPA agrees that neither the 2018 proposed rule nor the 2020 SNPRM included a characterization of costs or benefits of the proposed action. As discussed in the preamble to the final rule, this is a rule of internal procedure promulgated under the EPA's housekeeping authority and a benefit-cost analysis is not required. However, the EPA has included a discussion of costs and benefits in the preamble of the final rule. 
Briefly, the EPA will continue its current practice of conducting extensive review of scientific studies during the development of significant regulatory actions and influential scientific information. The additional procedures required by this rule only apply to pivotal science, which is a subset of the total number of studies that the EPA would evaluate. Given the costs of the current robust process for identifying and reviewing scientific studies and documentation that are existing Agency practice, as well as that the determination of dose-response data availability is limited to pivotal science underlying significant regulatory actions and influential scientific information, the EPA anticipates that the incremental costs of this rule will be small. The Agency may also incur other administrative costs to perform analyses and evaluations to support activities such as exemption decisions made by the Administrator, and documenting these or other decisions made pursuant to the requirements of the final rule. Again, the Agency anticipates that the incremental costs for these activities will be small relative to current administrative costs for developing significant regulatory actions or influential scientific information. Finally, this final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
The EPA also agrees that the benefits of the rule were not fully characterized in the 2018 proposed rule or the 2020 SNPRM. The EPA emphasizes, however, that this is a rule of internal procedure promulgated under the EPA's housekeeping authority. As discussed in Section III.A.1, the main benefits of this rule spring from greater transparency in significant regulatory actions and influential scientific information. By placing greater emphasis on the availability of dose-response data underlying pivotal science, the rule will allow for greater scientific scrutiny as EPA decision makers are developing significant regulatory actions and influential scientific information and increases the likelihood that any errors will be identified and corrected. Greater transparency is also inherently valuable as a principle of good government and provides benefits to the public at large, including reducing the risk of errors in EPA analyses and in the science such analyses rely upon. The ability for independent subject matter experts to validate pivotal science will facilitate more effective comment and engagement with the public during development of future significant regulatory actions and influential scientific information.
This action did not trigger the PRA; because it does not contain any information collection activities, it does not impose an information collection burden. Comments related to the deliberative interagency review process are outside the scope of this rulemaking.
Comment: Commenter (6879) cites Pruitt's Utterly Opaque `Transparency' Proposal," by Professor Daniel Farber (April 26, 2018):
      This proposal raises a host of legal and policy issues ... I wanted to kick off the conversation with a basic legal point: the document fails to meet the fundamental legal requirements for a valid rulemaking proposal. It is vague as to the actual parameters of the proposal, open-ended in terms of the alternatives under consideration, and utterly lacks the data transparency it purports to advocate. Any rule issued on the basis of this notice will be invalid, quite apart from its merits. ... 
      Really, the agency might just as well have said: `We want to do something about data transparency. Just spit balling, folks, but here's one idea. We're also open to doing something completely different. Let's just bat this around and see what we come up with.' That might be fine for a brainstorming session over a few beers. But it's not what the courts expect from a notice of proposed rulemaking.
Response: Based on a consideration of these and other similar comments, the EPA issued a SNPRM in 2020, which included clarifications, modifications, and additions to certain provisions proposed in the 2018 NPRM and allowing for additional public comment. Public comments received on both the 2018 NPRM and the 2020 SNPRM were considered in the development of the final rule. As detailed in the preamble to the final rule, the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
2.1.7.12 EPA's Proposed Data Transparency Regulation Can Be Substantially Improved to Promote the Rule's Stated Goal, But Substantial Modifications Will Be Needed
Comment: Commenter (6126) states that the preamble to EPA's proposed data transparency regulation indicates that the purpose is to "strengthen the transparency of EPA regulatory science." The commenter notes that their assessment of the proposed regulatory text is that this goal cannot currently be achieved based on the information provided in the Notice of Proposed Rulemaking in conjunction with the current EPA organizational infrastructure. The commenter contends that the proposed rule is vague in ways that make some details of its likely implementation difficult to understand in the absence of further elaboration or subsequent implementation guidance. Based on the issues EPA solicits comment on, the commenter (BPC) offers 10 areas for consideration about how to improve the proposed rule consistent with the principles of evidence-based policymaking, while also setting an expectation about responsible and reasonable implementation of an effort that encourages science transparency while protecting the confidentiality of data subjects. The commenter states that their comments on the proposal identify select areas for improvement or concern and are not intended to be comprehensive of all aspects of the rulemaking that may need modification.
Response: Based on a consideration of these and other similar comments, the EPA issued a SNPRM in 2020, which included clarifications, modifications, and additions to certain provisions proposed in the 2018 NPRM and allowing for additional public comment. Public comments received on both the 2018 NPRM and the 2020 SNPRM were considered in the development of the final rule. As detailed in the preamble to the final rule, the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings. The EPA also intends to develop internal implementing guidance and promulgate either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule, which will clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes EPA administers. 
2.1.8 Other Effects/Impacts
 2.1.8.1 Openness and Transparence Improvements Will Take Time and Needs to Be Developed to Benefit and Not Harm Research
Comment: Commenter (6873) states that everyone in science acknowledges there is room for improvement when it comes to openness and transparency, which is why the systems for handling transparency and fact-checking communication (peer review, journal publishing, conferences, replicability, statistical analysis, etc.) are slowly changing for the better. The commenter notes that perhaps where the conflation comes in about secrecy is that, at the study level, there is a tolerance of and even need for certain research practices that are not entirely open (including but not limited to the privacy concerns mentioned above. The commenter asserts that this will continue to be a reality even as science becomes more open -- there will be a continued need to preserve the research and development (R&D) incentives of drug companies by keeping certain data private, for instance, and to recognize that researchers want to avoid being scooped (to preserve their job security and future funding) by releasing their data too soon. 
Commenter (6873) adds that, overall, the global research community is working hard to make the transformation to more openness and transparency but it's a long way from getting there and they are only at the starting edge of discussing how some of these changes will be able to happen (especially with regard to open data). The commenter notes that animating concern in this global push for transparency and openness is to benefit science and society. Along the way, though, the commenter states that there is a need to be careful to ensure that the openness and transparency improvements we make end up benefitting research and not harming it.
Response: The EPA agrees that transparency and openness are benefits to science and society. The EPA also agrees that this rulemaking augments other important transparency initiatives occurring both inside and outside of the federal government.
2.1.8.2 Limits on the Use of Scientific Evidence by EPA
Comment: Commenters (0040, 0043, 0055, 0057, 0061, 0064, 0566, 0569, 0670, 1011, 1012, 1170, 1358, 1433, 1796, 1822, 1943, 1994, 1997, 2170, 2171, 2259, 2336, 2353, 2361, 2428, 2429, 2431, 2479, 2495, 2548, 2907, 2908, 3131, 3132, 4410, 4542, 4588, 4590, 4591, 4595, 4596, 4597, 4840, 4842, 4844, 4877, 4878, 5011, 5129, 5165, 5170, 5172, 5173, 5176, 5178, 5184, 5977, 6046, 6103, 6108, 6110, 6111, 6114, 6119, 6121, 6123, 6125, 6127, 6130, 6131, 6133, 6137, 6142, 6144, 6145, 6152, 6153, 6156, 6157, 6158, 6159, 6166, 6168, 6170, 6171, 6173, 6188, 6194, 6351, 6355, 6357, 6373, 6378, 6426, 6447, 6450, 6464, 6872, 6874, 6877, 6879, 6880, 6882, 6884, 6893, 6894, 6896, 6897, 6904, 6909, 6915, 6919, 6920, 6924, 6932, 6941, 6955, 8272, 8275, 8317, 9227) oppose the limits on the use of scientific information/best available science. Commenter (6125) asserts that, without the information provided in excluded research studies it would no longer be possible to assess the full weight of the available scientific evidence  -  a key guiding principle for judging the scientific integrity of the decision-making process. Comments include:
 The proposal could result in unprecedented limits on the use of scientific evidence by EPA. The commenter notes that these limits in turn will shape the development of evidence-based public health policies, including air pollution standards, drinking water regulations, pesticide tolerances, worker protections, and more. (0566)
 The proposed rulemaking jeopardizes the integrity of the science that the EPA relies upon. (1012)
 The promulgation of this proposed rule would set a dangerous and potentially life-threatening precedent regarding the use of health-based data, modeling, and research in regulatory decision-making. (4410)
 Studies will have difficulty being approved by the IRB if raw patient data must be made public. (4844)
 If EPA's rule takes effect, it could introduce selection bias that may slow studies and alter results and thereby affect regulatory decisions. (4878)
 Commenter asserts that they depend on the science the EPA uses to protect their health and the health of their family and community. The commenter notes that the proposed rule, which would place unprecedented limits on the use of science at the EPA, would prevent the agency from using the most strongly scrutinized peer-reviewed science. (5129)
 Commenter states that multiple aspects of the proposed rule will be costly, including its demand to make data publicly available and its metrics of high-quality studies that cover a broad range of models and doses. The commenter states that costs such as these not only eat up scarce resources, but they also lead to selection biases regarding which studies EPA will include in risk analysis. The commenter states that smaller organizations will likely lack the financial resources both to support such comprehensive studies and to make their data publicly available, so their studies may be excluded. The commenter states that, alternatively, if EPA is covering part of the cost of making data public, it may choose fewer, more general studies to include in its analyses. (6168)
 Denying the EPA the ability to use studies without making all the data public would seem to cripple the standard-setting process with no actual gain. (5170)
 The proposal would dramatically limit the human health studies the EPA can use to support development of health-protective regulations, eliminating critically important research from the policy-making process. (5172)
 By disallowing EPA researchers and analysts the use of study data which are not published, the non-published study data, regardless of their quality and applicability, will not be used or discussed by EPA, thus hiding their existence from the public. Thus, according to the commenter, the public is less informed about the spectrum of data and a thorough critical weight of evidence evaluation cannot be conducted. (6046)
 The Proposal may have the unintended consequence of excluding sound, peer-reviewed science in the rulemaking process. (6108)
 Although the proposed rule may be striving to improve regulatory science, they fear that it could be a case in which the "best is the enemy of the good. The commenter contends that the EPA cannot let their pursuit of "perfect science' lead them to disregard good scientific studies. The commenter suggests that, just as scientists cannot pick and choose the data they use for analysis, it is imperative that EPA use all available scientific studies to formulate its decisions. The EPA should not ignore existing data. (6110)
 State and tribal environmental agencies rely on the EPA to provide accurate scientific information. The commenters contend that the proposed rule would slow the EPA's work to a near standstill or leave it open to criticism in terms of credibility, leaving states with data gaps that they cannot fill. (6114, 6131, 6361)
 The implementation of the proposed rule would undermine the ability of the Agency to use the best available science to protect public health and the environment. The commenter adds that the proposal will not improve the use of science at EPA, but instead would restrict the types of science the Agency may use in regulatory decision-making. (6166)
 Many academic scientists whose research is relevant to EPA regulations may not conduct and report their studies in a way that satisfies the requirements of the proposed rule. The commenter adds that the proposed rule's provisions would require significant additional resources and could impose unreasonable and impractical requirements beyond those included in current protocols. The commenter notes that academic researchers, who often study sensitive and relevant health effects that are not evaluated in industry-sponsored GLP studies, typically focus on publishing their studies in peer-reviewed journals and obtaining research funding; they may not be concerned about or even consider whether their studies would qualify for use in establishing EPA regulations, and/or may not have the resources to reshape their approach to maintaining data. Additionally, according to the commenter, many researchers, particularly in other nations but also in the United States, may not even be aware of EPA requirements for a study's use in regulations. (6915)
Commenters (0043, 5176, 6880) object to the proposed rule because it would weaken science by rejecting a century of scientific and medical knowledge and progress. Comments include: 
 The proposed rule will weaken science by drawing a line in the sand that effectively disregards scientific discoveries prior to the first part of the 21st century when the importance of open data started to be increasingly recognized. The commenter states that the EPA proposal would throw out the mass of accumulated scientific evidence from the past, when open practices were not customary  -  and indeed often not practical before computers for big data were available. The commenter adds that most studies that would be relevant to EPA take years to do, and even longer to filter through to affect policy. The commenter states that recognition, for instance, of the dangers of asbestos, took years because the impacts on health were not immediate. The commenter states that work demonstrating the connection occurred many years ago and the data are not openly available, yet the EPA's proposed rule would imply that it could be disregarded. The commenter states that, similarly, open data demonstrating the impact of lead in paint or exhaust fumes, or of pesticides such as DDT are not available: does this mean that manufacturers would be free to reintroduce these? (0043)
 The proposed rule, if applied retroactively, would prohibit the EPA from considering studies completed prior to the effective date of the rule for which the data are not publicly available. The commenter states that this entails rejecting the contributions of thousands of individuals who -- for the benefit of future generations and to advance the public good -- agreed to take part in research. The commenter states that many important public health and environmental studies have taken place after one-time events and natural disasters and replicating these studies would be ethically impossible, as doing so would require intentionally exposing people to harmful contaminants or recreating one-time public health disasters. The commenter states that disqualifying such studies, as the rule proposes to do, means losing valuable public health knowledge, for instance, on the effects of environmental pollutants on human health. (5176)
 Commenter (6880) suggests the rule would apply its restrictions to any scientific study conducted before or after the rule is promulgated. The commenter states that, since most science conducted prior to the Internet age would likely fall short of the rule's transparency standard, the proposed rule would essentially throw out a century (and more) of scientific and medical knowledge and progress. The commenter states that this certainly contradicts the stated purpose of the proposal in the preamble, which is that the proposal will "help ensure that EPA is pursuing its mission of protecting human health and the environment."
Response: The EPA considered these and other similar comments when developing the final rule. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions and finalizing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider the availability of underlying dose-response data when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available for independent validation. By placing greater emphasis on the availability of dose-response data underlying pivotal science, the rule will allow for greater scientific scrutiny as EPA decision makers are developing significant regulatory actions and influential scientific information and increases the likelihood that any errors will be identified and corrected. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations. This approach will result in significant regulatory actions and influential scientific information that are based on the highest quality, most relevant studies and ensure adequate human health and environmental protections, while maximizing transparency and maintaining consistency with the statutes the EPA administers. The EPA does not believe that this final approach will lead to bias towards any party.
The EPA intends for the final rule to be implemented in a transparent manner. The final rule requires the EPA to document critical assumptions and methods used in its dose-response assessment, studies considered pivotal science, the rationale for the level of consideration for studies in the significant regulatory action or influential scientific information, and any Administrator exemption in the significant regulatory action or influential scientific information. Additionally, the EPA expects to identify pivotal science in proposed significant regulatory actions or external review drafts of influential scientific information, which will allow the public, if they so choose, to independently validate the pivotal science and provide comment to the EPA.
2.1.8.3 Research Will Not be Pursued or Will Be of Less Value
Comment: Commenters (1012, 2336, 2431, 4594, 4840, 4844, 4878, 5170, 5418, 6114, 6121, 6131, 6132, 6145, 6152, 6188, 6361, 6373, 6877, 6894, 6909, 6924, 6937, 8274, 9227) express concern that the Proposal will result in research not being pursued and/or it would be of less value. Comments include:
 It is likely that fewer scientists or research institutions will want to participate in EPA funded research if they are concerned that the personal health information, they collect may have to be publicly disclosed. The commenter adds that scientists and researchers may also find it difficult to recruit participants for long-term, large-scale studies that are needed to understand air and water quality issues if the participants are concerned that their medical records may be made publicly available. (1012)
 To satisfy a blanket disclosure requirement, a blanket redaction will become necessary. The commenter states that the resulting increase in costs will substantially reduce the number of future studies that can be conducted under a given level of spending. (2336)
 Individuals may be dissuaded from participating in new studies if privacy information commitments are not assured, and the quality of future science would suffer. (2431)
 The proposed rule could force scientists to disclose the health records of the families and children who suffer from asthma or are sickened by polluted water or to not conduct scientific studies at all. (2909)
 A failure to protect privacy data will hinder future scientific investigations as people will refuse to participate in research studies if they are not confident that their most personal information is protected. (4594)
 The proposal will impose costs and impediments that will discourage scientists from undertaking studies of great importance. (4840)
 In terms of future studies, researchers may have difficulty recruiting participants if they fear that their confidential information will be made public. (4844)
 Persons willing to participate in studies may be different as a result of the proposal in ways we cannot currently predict or describe, which could introduce confounding variables and bias that may question the study's results. (4878)
 If the proposed rule were to be implemented as currently written, it is highly likely that fewer scientists or research institutions will participate in EPA funded research because of concerns that the personal health information collected as part of research may have to be publicly disclosed. Consequently, the commenter suggests that researchers will find it difficult to recruit participants for long-term, large-scale studies that are needed to understand the health impacts of environmental quality issues as few participants are willing to have their medical records made public. (5170)
 Without the ability to protect and respect patient/human subject privacy and confidentiality, signatories and other researchers would not be able to conduct the studies that are pivotal to their work and to EPA's ability to fulfill its statutory duty to protect public health. (5418)
 While the language of the proposed rule would allow personal data to be redacted, it would also allow that data to be unredacted by the EPA Administrator. The commenters state that no information is given about how the Administrator would reach this kind of decision. Therefore, the commenters believe that participants from academia, industry, and members of the public would likely be hesitant to participate in studies, not knowing if or when confidential information could be released. (6114, 6131, 6361)
 The proposal will damage the ability of researchers to conduct future studies. The commenter suggests that researchers' ability to recruit potential research participants will be severely undermined if they fear that their personal information may be made available for public scrutiny. The commenter notes that this is particularly true for vulnerable communities that are both disproportionately exposed to hazardous chemicals and have historical reasons to distrust researchers. (6121)
 The proposal could result in lost publications (loss of intellectual property) and academic career derailment. The commenter adds that, when people no longer wish to enroll for fear their medical data will be released, new scientific studies could be inhibited, thereby damaging future scientific research. (6132)
 Commenter suggests that the release of data underlying scientific studies will limit the level of participation in future studies. (6145)
 Concerned that the proposal would undermine the quality of science that EPA uses to promulgate regulations. (6152)
 The proposed rule would impede the development and utilization of new science. (6894)
 Proposed rule has the potential to have significant negative impacts on the conduct of human health research in the United States. The commenter states that privacy concerns may begin to influence what science gets done and what does not. The commenter states that lines of scientific inquiry that would have been pursued may not be. The commenter states that the quality of data that is obtained may be poorer than it otherwise would have been. (6909)
 The rule's emphasis on publicly available data would have the perverse effect of discouraging individual participation in scientific studies due to personal privacy concerns. The commenter notes that potential study subjects, uncertain about EPA's willingness or ability to protect their identities and sensitive health information, would be more likely to be hesitant about participating in any study that could inform an EPA regulation. As a result, the commenter states that researchers could face increased difficulty in attracting subjects for crucial studies, reducing the availability of scientific information upon which to base EPA's decisions. The commenter contends that studies upon which EPA would be forced to rely could be less rigorous and trustworthy; the fact that a study's underlying data is publicly available has no bearing on the quality of the data or the study, and ground-breaking research on the health and environmental effects of pollution often relies upon the very kinds of highly sensitive data that would be most likely to be excluded from EPA review if the Proposed Rule were to be implemented. (6188)
 If EPA requires the distribution of identifiable data (i.e., individual data elements such as date of admission and residential zip code, a combination of which can identify a person), participants would have to sign agreements allowing the release of their private information and, as a result, they may decline to participate. According to the commenter, resulting impacts on participation rates could greatly increase the cost of studies and in the worst case, render research inconclusive because of a lack of representation. The commenter notes that researchers may have to redesign studies in a way that would reduce invasion of privacy, which could affect the accuracy or nuance achievable in a study, or account for the time and expense necessary to redact private information. The commenter contends that both of these consequences could damper studies' ability to answer critical research questions. (6373)
 Public health research, specifically observational cohort studies, can't be conducted without the cooperation and trust of study participants. Knowledge that their private data may be released to the public may severely impact the information study participants are willing to share. This in turn will compromise the quality of the research I and other public health researchers can conduct. There could be spillover effects from this proposed rule into other research studies that won't necessarily be considered for regulatory purposes, also compromising the quality of those studies. (6378)
 New studies may be discouraged for privacy and/or cost considerations. The commenter explains that the need to provide data will make it harder to obtain consent from subjects worried about personal identification and that it is possible that even very expensive measures such as use of the Federal Research Data Centers may not be adequate to maintain adequate privacy. The commenter adds that, with the stringent data requirements, and the increased concern about privacy, it is likely that the costs of compliant research would go up and that this may discourage researchers from undertaking future studies. (6877)
 Commenter is concerned that researchers would choose not to pursue research with human subjects, stifling scientific discovery; or that they will forgo compliance with the EPA proposed rule to obtain IRB approval and their research will be ignored by EPA because: (1) the researcher would have to make participant data publicly available and would not be able to ensure privacy/confidentiality; and (2) the researcher would have no control over how data they make publicly available would be used -- it could very well be used in a way that adversely impacts research subjects through privacy invasions or weakening of protections that safeguard communities. (6924)
 Under the proposed regulation, there will be pressure to submit protected data to third party partners, federally sanctioned privately managed data centers, and/or federal repositories. The commenter notes that this will be a Catch‐22 for researchers. The commenter provides that the researcher's choice will be to submit data to unproven or untrusted repositories, risking catastrophic release or misuse  -  or to not participate at all in research and grants that touch on policy‐relevant issues. (6937)
 Communities with lower socioeconomic status have been shown to be underrepresented in studies for various reasons. The commenter adds that restriction of evidence based upon data that is reliant on public access will certainly further confound this matter in these communities. The commenter notes that when community members are fearful that identification could lead to job loss or other negative personal impacts they are much less likely to participate in studies that examine personal health and environmental impacts linked to an employer., The commenter suggests that trust of confidentiality of and restriction to data accessibility increases study representation by reducing the exposure risk factors based upon individual identity. The commenter contends that restricting representation of communities that would result from these data accessibility issues leads to bias in a study's evidence by misrepresenting the community group that is being studied. (8274)
 Commenter (2478) expresses that the extent to which efforts to "strengthen transparency in regulatory science" would have led to a reduction in sponsored research into climate science, the EPA should have the burden of proving that this proposed rule was not designed to prejudice any particular area of scientific inquiry germane to the charge of the EPA. In other words, the commenter states that the EPA should have to demonstrate that this proposed rulemaking is not a pretext to de-prioritize research into climate science.
 Commenter (9227) states that the proposed rule does not disclose the cost -- highlighted on the very first page of a NAS report on data access -- that "perceived risks to privacy and confidentiality reduce survey participation," a cost that the NAS explains is "borne out by research." The commenter notes that NAS explains that this "threatens the research enterprise itself, because concerns about privacy and confidentiality are among the reasons often given by potential respondents for refusing to participate in surveys, and those concerns have been shown to affect behavior as well. The commenter adds that the NAS panel emphasized: "Any confidentiality breach that became known would be likely to heighten such concerns and, correspondingly, reduce survey response rates. Efforts to increase researchers' access to data must, therefore, take into account the need to avoid increasing the actual and perceived risks of confidentiality breaches."
      The commenter (9227) provides that this confidentiality risk has a further cost: it affects the quality of the data collected. As the NAS explained:
            The reason for confidentiality pledges and for stringent procedures to prevent disclosure is that they improve the quality of data collected from individuals, households, and firms. It is essential that respondents believe they can provide accurate, complete information without any fear that the information will be disclosed inappropriately. Indeed, if the information was disclosed, harm might come to an individual respondent.
      The commenter (9227) asserts that the proposal's only acknowledgment of this complex problem and cost is its statement that "EPA believes that concerns about access to confidential or private information can, in many cases, be addressed through the application of solutions commonly in use across some parts of the Federal government." Remarkably, the commenter adds that the EPA does not cite a single example of these common solutions, citing only vaguely to "examples from the U.S. Department of Health and Human Services, National Institute of Standards and Technology, U.S. Department of Education, and the U.S. Census Bureau" and some hyperlinks not in the Proposal added to the docket almost a month into the comment period. Accordingly, the commenter states that, not only does the Proposal include no analysis of these alleged solutions and their costs and benefits, it does not even explain what the solutions are that EPA believes address this concern.
Response: The EPA considered these comments when finalizing this rulemaking, and the EPA does not believe the final rule will result in research not being pursued and/or research having less value. The provisions of this rulemaking are not restricted to any particular scientific discipline, but apply to all pivotal science used in significant regulatory actions and influential scientific information. As discussed in the final rule, underlying data or models are considered available if they can be accessed through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. The EPA would like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedure and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the rule does not require the EPA to collect, store, or publicly disseminate data and models underlying pivotal science.
2.1.8.4 Competitive Nature of Scientific Funding
Comment: Commenter (1170) expresses that the proposed rule is that it also ignores the intrinsically competitive nature of scientific funding, which means that even when there is no legal barrier to making data public, groups are strongly incentivized to keep some data non-public at least in the short term, so that they may carry out further analysis. The commenter notes that this is particularly true of data that takes months or years to obtain. 
Response: The EPA considered this comment and similar comments when finalizing this rulemaking. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7. The EPA would also like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedure and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the rule does not require the EPA to collect, store, or publicly disseminate data and models underlying pivotal science.
2.1.8.5 Proposal Would Result in Fundamental Changes in the Use of Scientific Data and Research
Comment: Commenter (5170) states that the proposed rule would result in fundamental changes in the use of scientific data and research by the EPA, which they note is one of the most significant users of scientific data in the federal government. The commenter adds that there is a great deal at stake, in human health, lives and economic benefits and costs in such a significant change to the use of science as is being proposed in this rule and should, therefore, have a compelling, demonstrated need.
Response: The EPA considered this and other similar comments when developing the final rule and believes the approach used in the final rule will result in final significant regulatory actions and influential scientific information based on the highest quality studies that maximize transparency. The final rule also clarifies how the EPA will consider studies when developing final significant regulatory actions and influential scientific information. As more fully described in the preamble to the final rule, the EPA will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider the availability of underlying dose-response data when evaluating pivotal science and give greater consideration to pivotal science where the underlying dose-response data are available for independent validation. The rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings. 
2.1.8.6 Benefits of Environmental Protection Standards
Comment: Commenters (0017, 1331, 1424, 5120) suggest the economic benefits of environmental rules are greater than the implementation costs. Commenter (1331) asserts the public health costs of its implementation severely outweigh its benefits.
Commenter (0017) states the proposal will have a tremendous negative impact on the U.S. economy. The commenter states that, according to a 2011 EPA study, the estimated annual economic benefits of air quality improvements are estimated to be nearly $2.0 trillion in 2020 if they were to be fully implemented as laid out under the CAA. The commenter states that, in comparison, the cost of this benefit due to regulatory compliance actions would be approximately $65 billion. The commenter states that the EPA report estimates that the economic benefits of clean air to human health and environmental improvement exceed the costs by more than 30 to one. 
Commenter (1424) states that all cost-effective studies show that protective environmental regulations save money, as they save billions in health care costs caused by toxic chemicals that cause cancer (PCBs), neurological deficits (lead), lung disease (PM2.5) and others. Commenter (5120) states that air quality legislation, culminating in amendments to the CAA in 1990, has produced an estimated $59-140 billion in health savings. 
Response: This rule sets the overarching structure and principles for transparency of pivotal science in significant regulatory actions and influential scientific information. The final rule provides that if implementing the rule results in any conflict between this rule and the environmental statutes that the EPA administers, and their implementing regulations, this rule will yield and the statutes and regulations will be controlling. This is a rule of internal procedure promulgated under the EPA's housekeeping authority. This rule exclusively pertains to the internal practices of the EPA and does not regulate the conduct or determine the rights or obligations of any entity outside the federal government. Additionally, this rule does not require the EPA or other parties to disclose or host data. Hence, this rule does not impose costs on the EPA or any other party to make such data available. The EPA recognizes there will be some incremental review and administrative costs to the Agency but anticipates these will be small. As discussed in the preamble to the final rule, the main benefits of this rule spring from greater transparency in significant regulatory actions and influential scientific information. By placing greater emphasis on the availability of dose-response data underlying pivotal science, the rule will allow for greater scientific scrutiny as EPA decision makers are developing significant regulatory actions and influential scientific information and increases the likelihood that any errors will be identified and corrected. Greater transparency is also inherently valuable as a principle of good government and provides benefits to the public at large, including reducing the risk of errors in EPA analyses and in the science such analyses rely upon. The ability for independent subject matter experts to validate pivotal science will facilitate more effective comment and engagement with the public during development of future significant regulatory actions and influential scientific information.
2.1.8.7 Proposed Rule Could Further Delay Development and Update of Toxicity Values/Reference Values/Guidelines/Policies/Rules
Comment: Commenters (0476, 2171, 5977, 6119, 6127, 6130, 6142, 6168, 6446, 6872, 6881, 6905, 6935, 6941, 8272) object to the proposal because it could delay needed federal action, development of protective standards, and development/update of toxicity values or reference values. Comments include the following:
 Properly implementing this rule will require EPA to first address many legal aspects surrounding public and nonpublic data, significantly expand database tools and peer review capacity, and develop guidelines for the use of "independent validation" -- all of which require considerable time and financial resources. The commenter states that, as a result, the proposed rule will delay the adoption of human health-based pollution standards for the foreseeable future. (2171)
 The proposal could further delay development and update of toxicity values or reference values published in the CERCLA, TSCA, CWA, SDWA and other federal acts authorized by EPA. The commenter provides that states, EPA and the public are already struggling to respond to emerging contaminants for which toxicity values and risk-based standards have yet to be developed. There are currently more than 129 million organic and inorganic substances identified with a chemical abstracts service number in the open scientific literature. Yet, the commenter notes that only a handful have been studied and even fewer with "dose response data and models" (reference values (reference dose and/ or reference concentrations)) used for "pivotal regulatory science" as noted in the proposed rule. The commenter contends that unnecessarily restricting the data currently in practice or potentially used for regulatory development would do little to rectify the situation in a timely or "transparent" manner. (6119)
 The added administrative barriers resulting from the proposed rule are likely to significantly delay the development of EPA guidelines and policies. The commenter adds that such delays are expected due to decreased access to industry claimed CBI, exclusion of relevant peer reviewed scientific studies, and the need for researchers to prepare publicly disclosable datasets. (6130)
 The proposed rule, if adopted, would inevitably delay setting protective standards through prolonged evaluation of the sufficiency of the public availability of research data and methods, rather than evaluation of the actual quality and import of scientific research. The commenter adds that the fact that the public would be able to de-anonymize much epidemiological data, after de-identification processes have been applied, would almost certainly have a chilling effect on voluntary public participation in important research, and would also prevent EPA from relying on the "best available science" in carrying out its work. (6446)
 Overly prescriptive and poorly defined list of modeling approaches will not either decrease or illuminate uncertainty and will not provide the basis for the specific decisions that confront the Agency; they are merely a recipe for obfuscation and delay. (6881)
 Commenter suggests EPA consider the impact of this proposed process on the pace of regulatory action and ensure the proper balance is struck between the potential value of independent validation -- which could take substantial time and, if so, be at odds with statutory requirements -- and the certainty and efficiency of regulatory decision making. The commenter states that, for example, if new transparency requirements cause delays in implementation of statutory provisions that cause statutory deadlines to be missed, then consequences can include deadline suits. (6935)
 Commenter is concerned that excluding data that cannot be made publicly available would slow the development and implementation of test methods and strategies to replace animal testing. The commenter states that such alternative methods and strategies can provide information of equivalent or better scientific quality and relevance for assessing risks of injury to health or the environment, and their increased acceptance is one of TSCA's goals. (6941)
Response: The EPA disagrees that the requirements of this rule will result in any meaningful delay in promulgating regulations. While this final rule requires the Agency to evaluate the availability of dose-response data for pivotal science, the incremental burden to the Agency to carry out these requirements is expected to be small given (1) the extensive scientific review the EPA already conducts regularly and (2) that the requirement is limited to pivotal science (i.e., typically a small, though highly important, subset of the studies the EPA would review). Additionally, this rule does not require the EPA to disclose or host data, but to determine if the dose-response data underlying pivotal science are available and to give greater consideration to pivotal science for which the underlying dose-response data are available in a manner sufficient for independent validation.
2.1.8.8 Proposal Will Not Increase the Public's Understanding
Comment: Commenter (6046, 6133, 6168) contend that the proposed rule would not result in more transparency to the public. Commenter (6046) suggests that, instead of the proposed rule, the EPA could greatly improve its transparency by making its documents and memos (particularly those cited by pivotal science EPA documents) more easily accessible to the public. 
Commenter (6133) suggests that the Proposal utterly fails to demonstrate or even support the claim that its approach ensures the information relied upon by EPA would be more understandable to the public. According to the commenter, the EPA fails to establish or even support the premise of its wrongheaded belief: that the best available, peer-reviewed, independent, credible science is not understandable already to the public, or the informed, knowledgeable members of the public versed in the scientific, technical, legal, economic or policy matters relevant to EPA's regulations, actions and mission. Additionally, the commenter notes that the proposal's claim about enhanced public understanding suffers from a fundamental internal contradiction and logical failing inherent to its approach: nothing in the Proposal requires that (1) publicly available data be actually considered, addressed, verified or replicated by EPA prior to the agency being allowed to consider the study based on that data; (2) publicly available data be actually considered, addressed, verified or replicated by any other person or party prior to EPA being allowed to consider the study based on that data; and (3) publicly available data be actually considered, addressed, verified or replicated by EPA, any person, or any party ever, before or after EPA is allowed to consider the study based on that data. Accordingly, the commenter contends that it is false and unsupported to suggest that the proposal ensures greater public "understanding" than the longstanding regulatory landscape where the Proposal's prescriptions and proscriptions do not exist.
Commenter (6168) notes that, if the EPA intends transparency in how research informs regulations, it must devote substantial resources to improving explanations of research utilized and their various caveats and not require dumps of "raw" data and algorithms without context. The commenter contends that "raw" data and algorithms are largely unintelligible, so this rule gives the appearance of transparency without actually enhancing science or participation. According to the commenter, as any scientist knows, data is never just data -- it is never really "raw" because it is always shaped by techniques of sampling, collection, entry, storage, and so forth. Because the proposed rule fails to acknowledge this basic fact about scientific data, the commenter states that the rule fundamentally misrepresents the practice and applications of research. The commenter notes that this is one sense in which the proposed rule impedes meaningful public understanding and engagement.
Response: The EPA disagrees with these comments. The final rule will result in final significant regulatory actions and influential scientific information based on the highest quality studies that maximize transparency. Additionally, the EPA intends for the final rule to be implemented in a transparent manner and expects to identify pivotal science in proposed significant regulatory actions or external review drafts of influential scientific information, which will allow the public, if they so choose, to independently validate the pivotal science and provide comment to the EPA.
2.1.8.9 Proposal Would Constrain EPA's Consideration of a Regulation's Public Health Benefits
Comment: Commenter (6922) expresses concern that the proposal, if finalized, would constrain EPA's consideration of a regulation's public health benefits. The commenter notes that environmental statutes often call on EPA to evaluate the benefits and costs of a rulemaking when determining the most appropriate standard, control technology, and other regulatory requirements. In addition, the commenter adds that Executive Order 12866 requires agencies to conduct a cost-benefit analysis of major regulations. The commenter provides that the proposal, if finalized, would have the effect of precluding EPA's consideration of CBI and epidemiological studies for which underlying data is protected from public disclosure in EPA's development of the regulations and in assessing the costs and benefits of those regulations. Historically, the commenter notes that the EPA has been able to monetize only the public health benefits of reducing pollutants for which large-scale epidemiological studies are available and epidemiological studies, which predict mortality risks and other health effects, often rely on data that are not public, for various legitimate reasons. 
Commenter (6922) contends that prohibiting reliance on epidemiological studies and/or CBI in analyzing benefits or costs would reduce the likelihood that EPA could consider the full suite of costs and benefits of regulation. The commenter is concerned that this would have the effect of elevating some considerations, such as cost of compliance, over other considerations, to the detriment of reasoned decision-making. The commenter concludes that both benefits and costs must be based on the best data in order to have reasoned decision-making.
Commenter (6915) states that restricting the use of studies that underlie emission standards for criteria pollutants could significantly impact the cost-benefit analyses for various other health-related rules by failing to account for all the benefits, making it far more likely that the costs will be predicted to exceed the benefits and that the regulatory standards will, accordingly, be lowered. The commenter states that, for example, in calculating the costs and benefits of rules to reduce air emissions, in some cases the majority of the benefit estimates are attributable to reductions in one or more criteria pollutants that are not the primary objective of the rule. The commenter states that these reductions are referred to as co-benefits, and the health impacts and monetized benefits are based on studies used in the air quality standards-setting process for criteria pollutants. The commenter states that, for example, in promulgating the Mercury and Air Toxics Standard rule governing air emission standards for hazardous air pollutants (including mercury) from power plants, EPA states "[i]t is important to note that the monetized benefits include many but not all health effects associated with PM2.5 exposure." 77 Fed. Reg. 9304, 9431 (Feb. 16, 2012); see also id. at 9305. 
Response: The EPA considered this and other similar comments when developing the final rule and believes the approach used in the final rule will result in final significant regulatory actions and influential scientific information based on the highest quality studies that maximize transparency. The final rule also clarifies how the EPA will consider studies when developing final significant regulatory actions and influential scientific information. As more fully described in the preamble to the final rule, the EPA will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider the availability of underlying dose-response data when evaluating pivotal science and give greater consideration to pivotal science where the underlying dose-response data are available for independent validation. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating final significant regulatory actions and finalizing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations. The rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
Based on consideration of this comment and similar comments, the EPA clarified in the 2020 SNPRM the relationship between this rulemaking, the environmental statutes and their implementing regulations by adding language to proposed 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations would control in the event of any conflicts. With this final rule, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
2.1.8.10 Proposal will Lead to Increased Litigation
Comment: Commenter (6937) suggests that the most likely outcome of the proposed regulation is not an improvement in the regulatory process or outcomes, but that every major regulation will be litigated on the topic of whether regulatory standards of transparency and replicability were met. The commenter notes that trying to write a regulatory one size fits all "bar" for transparency and replicability is an impossible task, given the incredible heterogeneity of scientific data and "the array of workflows across scientific fields" as well as "the case for data sharing at different levels of stringency" (quotations from source in footnote 10 of the Federal Register Notice). For example, the commenter provides that they can easily imagine lawsuits where EPA is challenged with the accusation that it did not work hard enough to meet the following proposed requirement: "where data is controlled by third parties, EPA shall work with those parties to endeavor to make the data available..." (§30.5, page 18774). Or that EPA could be EPA challenged over the degree of "access" and "use restrictions" (§30.5, page 18774), since these are left very vague in the proposal. 
Response: The EPA disagrees that implementation of this rule will necessarily lead to increased litigation. The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. The rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings. The EPA would also like to emphasize that the final rule does not require the EPA to collect, store, or publicly disseminate data and models underlying pivotal science.
2.1.8.11 Credibility/Public Trust of the Agency
Comment: Commenter (8913) suggests that an unintended consequence of the proposal is the inevitable negative impact the Transparency Rule would have upon commerce and American industry. The commenter contends that the disregard of sound science and the drastic constriction of evidence considered by the EPA, will greatly reduce the credibility of the agency. In short order, the commenter adds that this will result in reduced confidence in businesses and industries which manufacture or use toxic agents. The commenter notes that it will also damage the credibility of industries involved in containment activity and fabrication of products and structures designed to control or mitigate the release of chemicals, radioactivity, and other harmful substances. Thus, according to the commenter, the products and services even of companies and industries which perform careful robust safety studies will be suspect. Lastly, the commenter states that the EPA imprimatur will also cease to have strong force globally, resulting in less competitive export industries. 
Commenter (5165) asserts that transparency concerns must not override EPA's obligation to consider the full range of peer-reviewed, sound scientific research that is available and relevant to its regulatory decisions. In the commenter's view, the proposal would likely hinder, rather than promote, EPA's use of best-available science and it would tend to diminish public confidence in the integrity of EPA's scientific decision making.
Conversely, commenters (1971, 5185, 5891, 6150, 6376) suggest that the proposed rule could be a step that increases public trust in and support for EPA itself. Commenter (5185) states that, while the agency's proposed reforms are clearly controversial, they are grounded in an accepted democratic principle: citizens have a right to the data and information that are used in the development of public policy. Commenter (5891) states that refusing to disclose the studies, surveys, etc., upon which any rule is based, is disheartening, dishonest and in-American. 
Commenter (6150) states that new causal analytical techniques can be used to test validity of existing data thereby increasing public confidence in results. The commenter states that, by relying on data, models and processes subject to rigorous disclosure, the Agency will better inform the public and enhance the public's ability to understand and meaningfully participate in the regulatory process. Commenter (6176) states that the public should not be expected to just trust the EPA (or any agency) to promulgate any rule it wants and draw its own conclusions without the public knowing how those conclusions were reached. Commenter (6376) states that creating and codifying a standard of scientific transparency should result in better regulations and strengthen public trust in the regulatory agenda. 
Response: The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty of the assessment. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. The EPA's continued progress toward maximizing transparency is vital to building and maintaining trust with the public and credibility in the Agency's decisions. The requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings. 
Finally, because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
2.1.8.12 Unable to Fully Assess Likely Impacts on EPA and State Programs
Comment: Commenter (6126) suggests that given the vague nature of certain definitions, the commenter states that they are unable to fully assess the likely impacts on EPA programs. 
Commenter (5181) states that it is unknown how the proposed rule would impact development of Water Quality Standards for wetlands. The commenter notes that many states include pollutant and toxic discharge standards among those that are applied to wetlands. The commenter states that the proposal should clearly address and answer this question, and also clarify the impact on grants to states.
Commenter (6915) states that, based on EPA's complete failure to consider or discuss the effects of its action on state programs, the proposal should be withdrawn so that EPA can adequately consult with state officials to analyze these important impacts. See Exec. Order 13132 § 6(a) (instructing agencies to "ensure meaningful and timely input by State and local officials in the development of regulatory policies that have federalism implications"); Chem. Mfrs. Ass'n v. U.S. Envtl. Prot. Agency, 870 F.2d 177, 203 (5th Cir. 1989). 
Response: The EPA clarified in the final rule that the requirements in 40 CFR part 30 apply to significant regulatory actions and influential scientific information. With this final rule, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
2.2 Purpose/Basis and Effects (SNPRM Comments)
 2.2.1 Purpose/Basis for Rule
 2.2.1.1 Alternative Purpose for Rule Besides Transparency
Comment: Commenters (10974, 10975, 11339, 11361, 11391, 11392, 11401, 11920, 13788) contend that the rulemaking is not an intention to improve transparency in EPA decision-making, but instead an attempt by EPA to undermine the scientific basis of the process. Specific concerns include:
 Commenter (10974) states that EPA's revisions do not clarify how this proposed rule enhances the public's ability to understand or meaningfully participate in the regulatory process and questions who represents "the public" in the context of independent validation. The commenter suggests that the limitation of reanalysis to individuals with expertise contradicts EPA's broader argument that the rule would benefit "the public's" understanding of science and EPA's rulemaking process. Further, the commenter contends that reanalysis by "subject matter experts" is redundant of current practices and that EPA is overstepping its authority to determine how individuals and organizations share their data, in turn confusing rather than clarifying the public's understanding and participation in the rulemaking process.
 Commenter (10975) states that the purpose of the proposed rule is to allow EPA to bypass its statutory duties and congressional mandates to enact restrictive rules in the first place, even when the evidence of potential public harm is clear. According to the commenter, this proposed rule does not come from rational public administrative concerns, nor from science, but from what political scientists call "Agency capture." The commenter also queries how this proposed rule can avoid the legal and moral taint of a highly partisan de-regulatory agenda for cutting back on public protections.
 Commenter (11339) states that the proposal is the cornerstone in a large-scale attack on health science at EPA, specifically, the scientific process of CAA rulemaking. The commenter provides examples of changes to the NAAQs rulemaking process including: disbanding advisory panels, lessen scientific review without adequate expertise, barring EPA funded scientists from serving on advisory panels, while creating no equivalent limits on the appointment of industry funded scientists.
 Commenter (11361) asserts that the "clarifications" offered in the SNPRM demonstrate the intention to undermine science through the façade of increasing transparency and that the framing of the rule around "recent innovations and policies surrounding information access" is actually disingenuous to the spirit of that spirit. The commenter contends that the data disclosure requirements in the 2020 SNPRM "allow stakeholders to reanalyze the data and models and explore the sensitivity of the conclusions to alternative assumptions" would make it easier for outside parties to cast doubt on peer-reviewed science. The commenter states that similarly, EPA's statement that the Proposed Rule does not "require that EPA, a member of the public or other entity must independently validate a study before it can be considered" indicates that the Proposed Rule would not improve the quality of the scientific information relied upon by EPA by ensuring it is independently verified, but would instead advance tactics used by industry to avoid regulation.
 Commenter (11392) states that this proposed rulemaking undermines the ability of EPA to protect human health and the environment and contravenes the Agency's statutory responsibility to use the best available science in promulgating health-protective regulations. The commenter contends that this rulemaking is not only dangerous, but is fundamentally flawed by failing to justify a legitimate purpose beyond arbitrary, political manipulation of the Agency's science-based work.
 Commenter (13788) contends that the suggestion that the rule could apply to studies retroactively if the Administrator does not grant an exemption further illustrates the intention to undermine longstanding reliance on the human health studies that have done the most to advance air quality and public health. The commenter states that the EPA has provided no reasoning to explain how retroactive application to past studies -- including the Harvard Six Cities and American Cancer Society particulate matter health studies and subsequent work that links exposures to air pollution with declines and damages to human health -- is justified. The commenter states that it certainly has not provided any information to suggest that this rule would encourage the release of more data from those studies. The commenter states that this proposal's focus is clearly on arbitrarily excluding or diminishing the value of quality scientific research rather than any real commitment to transparency.
Response: Because of the potential impact of the EPA's significant regulatory actions and influential scientific information on American lives and livelihoods, the American people deserve environmental decisions and policies that are based on the best scientific information. Only through continuous improvement to its procedures, especially those focused on transparency, can the EPA fully demonstrate that commitment. Data transparency is vital for individuals who have not contributed to the study to be able to verify the quality and strength of published studies and the opportunity to independently validate the pivotal science that the EPA relies upon is important in furthering scientific understanding and the Agency's mission. Increasing the availability of the underlying dose-response data upon which the EPA makes its decisions is integral to demonstrating accountability and this dedication to public health and the environment. Therefore, the EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalyses, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information. The final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
As discussed in the preamble of the final rule, the rule is designed to build upon open data initiatives in the federal government and scientific community. The rule is not targeted to a particular set of rulemakings or scientific studies, but rather intends to increase transparency in the pivotal science used in significant regulatory actions and influential scientific information. The EPA believes that the implementation of this final rule will increase the transparency of pivotal science for significant regulatory decisions and influential scientific information, while still ensuring that the highest quality, most relevant studies are considered in the development process.
Comment: Commenters (11190, 12720) contend that the rule is designed as an avenue to allow polluters to provide legal challenge to regulations under the new data availability requirement.
Commenter (11920) contends that through restricting the use of relevant and high-quality scientific information, the EPA is seeking to protect polluters and regulated industries, rather than carry out its statutory mandate to protect the environment and American citizens.
Response: The EPA disagrees that this rule is intended to benefit industry. The EPA is committed to its mission of protecting public health and the environment through sound policy decisions that are informed by robust scientific and technical research. Because of the potential impact of the EPA's significant regulatory actions and influential scientific information on American lives and livelihoods, the American people deserve environmental decisions and policies that are based on the best scientific information. Only through continuous improvement to its procedures, especially those focused on transparency, can the EPA fully demonstrate that commitment. Data transparency is vital for individuals who have not contributed to the study to be able to verify the quality and strength of published studies and the opportunity to independently validate the pivotal science that the EPA relies upon is important in furthering scientific understanding and the Agency's mission. Increasing the availability of the underlying dose-response data upon which the EPA makes its decisions is integral to demonstrating accountability and this dedication to public health and the environment. Therefore, the EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access.
2.2.1.2 Rule Does Not Have Adequate Basis or Justification
Comment: Commenters (11332, 11392, 11490 11894, 12430, 12455, 12456, 12727, 13788) assert that the Proposal and subsequent SNPRM do not have adequate basis or justification for the need to alter EPA's rulemaking process. Specifically:
Commenter (11332) states their concern considering EPA's lack of justification for this sweeping reach of the supplemental proposal, which is all-encompassing compared to even the original proposed rule. The commenter states that EPA's rule would apply to any data and models used by the Agency to craft its regulations, and that the change is significant. The commenter notes that it is an expansion of the net cast in the original proposal, which was limited to dose response data and models, and that this change could weaken a wider range of current pollution controls all across this country. The commenter contends that this major change in scope was never justified in the proposal.
Commenter (11392) notes that they agree with the members of EPA's handpicked SAB that this rule "risks serious and perverse outcomes" and that the Agency has not justified "why existing procedures and norms utilized in the U.S. scientific community, including the federal government, are inadequate, and how the Proposed Rule will improve transparency and the scientific integrity of the regulatory outcomes."
Commenter (11894) states that since EPA has failed to justify any per se, a priori restrictions based on data availability, any proposals for relief from those restrictions are proposals for relief from something that should not exist in the first place. The commenter asserts that as such, they are both unnecessary, and unable to increase in any way the legal defensibility of those restrictions, since they still allow the Agency, to the exact extent that the relief provisions do not apply, to unjustifiably refuse to consider certain studies. Furthermore, the commenter queries what possible sense does it make to exclude, arbitrarily and without justification, categories of studies that can contribute to understanding of particular problems that EPA is directed by statute to manage. The commenter agrees that EPA should be sure that regulatory decisions with binding legal effect rest on information whose quality has been tested, but the studies themselves have no binding effect; they are building blocks and evidence that the Agency uses to reach a conclusion. The commenter contends that process is debased when science is excluded for reasons that have nothing to do with its quality and probity, and to do so violates well-developed understandings of how science works. The commenter notes that the cure is not arbitrary exclusion, but, as John Milton said in a different context, to "Let [Truth] and Falsehood grapple; who ever knew Truth put to the worse, in a free and open encounter?" The commenter questions what EPA is so afraid of that it proposes to exclude these studies from the decision process as EPA manages major, consequential public health issues?
Commenter (12430) contends that as part of its notice-and-comment rulemaking requirements, the APA requires an Agency to provide notice of a proposed rulemaking, and to "disclose in detail the thinking that has animated the form of a proposed rule and the data upon which that rule is based." The commenter provides that the D.C. Circuit Court of Appeals has noted, "a prerequisite to the ability to make meaningful comment is to know the basis upon which the rule is proposed.", yet EPA has not provided a justification, let alone a detailed disclosure of its reasoning and data, for either the Proposal or the dramatic expansion proposed in the SNPRM. According to the commenter, the proposals merely assume and assert that: transparency is the determinant of scientific quality; transparency is presently lacking; public data availability is the only guarantor or indicator of transparency; and there exist no less disruptive means of increasing transparency that EPA could consider. The commenter states they have discussed EPA's failure to explain the basis of its NPRM in their 2018 comments and that the SNPRM fails to rectify this issue. The commenter notes that as in the NPRM, the SNPRM provides no references to U.S. EPA regulatory science or influential scientific information that subsequent review suggests were falsified or otherwise problematic, or instances in which public unavailability of data or models has resulted in irrational or arbitrary regulations or other harms. The commenter also contends that instead of providing a justification for profoundly expanding the scope of the Proposal in the SNPRM, EPA references only unspecified public comments and uncomplicated requests for clarification of the Proposal. The commenter provides that for example, the Proposal's preamble clearly explained the proposal's applicability only to dose-response data and models, but one provision of the proposed regulatory text was ambiguous on this point. The commenter asserts that EPA now cites this single ambiguity as its basis for the SNPRM's dramatic expansion of the proposal to all types of data and models. Additionally, the commenter contends that the citation of the OMB Memorandum does not provide basis for the proposed rule. Ultimately, the commenter states that EPA provides no reasoned basis for the SNPRM's drastic expansion of the Proposed Rule and that instead, the Agency summarily announces its intent to expand the provisions of the rule, which does not comprise the detailed disclosure of basis and data that the APA requires.
Commenter (12727) states that EPA does not explain or acknowledge its departure from existing approaches for assessing the weight of scientific evidence. The commenter notes that EPA's Risk Assessment Forum recommends weighting evidence based on relevance, strength, and reliability. Even within reliability, the commenter provides that the Forum observes that "scoring the most important component properties" is useful. According to the commenter, transparency is only one such component, and it is merely "presumed to increase reliability by reducing the likelihood of hidden faults." The commenter contends that EPA offers no reason to give overwhelming importance to one component of reliability in this framework, and, in turn, to reliability over the strength and relevance of a study. In fact, the commenter states EPA does not discuss existing approaches to weighting evidence at all and has therefore failed to acknowledge that it is changing policies or to provide a good reason for doing so, rendering a final rule that is similarly deficient and unlawful. 
Additionally, commenters (12455, 12456) request that EPA provide specific examples to justify the 2018 proposed rule and the SNPRM.
Commenter (13788) asserts that EPA has provided no reasoning to explain how retroactive application to past studies -- including the Harvard Six Cities and American Cancer Society particulate matter health studies and subsequent work that links exposures to air pollution with declines and damages to human health -- is justified. The commenter notes EPA has not provided any information to suggest that this rule would encourage the release of more data from those studies
Response: As discussed in the preamble of the final rule, the rule is designed to build upon open data initiatives in the federal government and scientific community and advances the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Having a better understanding of the underlying data will enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information. 
As described in the preamble to the final rule, there will be no categorical exclusion of studies based on the availability of their underlying dose-response data and the applicability of the rule is focused on dose-response data to provide incremental progress toward the Agency's goal of greater transparency. Briefly, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information, consistent with existing Agency practice. The EPA will continue to use the following, established factors to assess the quality of studies used to develop significant regulatory actions and influential scientific information: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence (consistent with Agency guidelines on hazard identification and dose-response assessment) to support a relationship between exposure and effect, the EPA will identify a subset of those studies for use in characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect. This will be based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both will be identified as pivotal science. Once the EPA has identified pivotal science -- in either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA believes that the implementation of this final rule will increase the transparency of pivotal science underlying significant regulatory actions and influential scientific information, while still ensuring that the highest quality, most relevant studies are considered in the development process.
The EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information. 
Finally, the EPA is maintaining that the rule provisions apply equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that are unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator, as detailed under 40 CFR 30.7. One of the criteria the Administrator may use in his/her determination to grant a waiver is that the development of the dose-response data was completed or updated before the effective date of the rule. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
2.2.1.2.1 Provided Citations Do Not Provide Solid Backing
Comment: Commenters (11894, 12645, 12727) provide that the SNPRM's citations are unable to be used to provide firm support for the rulemaking. Specific concerns include:
Commenter (11894) states that the memorandum cited in the supplemental proposal says nothing about precluding use of, or downgrading consideration given to, scientific information for which raw data are not publicly accessible. The commenter notes that the memorandum in fact continues to recognize (at p. 5) that agencies must "ensure that privacy and confidentiality are fully protected and that data are properly secure so that open data do not disclose personally identifiable information." The commenter asserts that it does discuss circumstances under which tiered access to data is appropriate (p. 9), a discussion that does nothing to support the sweeping scope of this supplemental proposal. Further, the commenter provides that the supplemental proposal also cites a NAS report entitled "Principles and obstacles for sharing data from environmental health research." The commenter contends that that assertion rests on a generic reference to an unspecified portion of a 148-page document. According to the commenter, the Agency is apparently unaware (or, worse, failed to disclose) that the NAS document in question is merely a workshop report even though its own citation clearly states just that. The commenter states that workshop reports are assembled to represent all divergent points of view, and have no other purpose; these are not NAS consensus recommendations and contain a very strong disclaimer to this effect.
Commenter (12645) contends that the most authoritative citation is to a Workshop on Principles and Obstacles for Sharing Data from Environmental Health Research. According to the commenter, this Workshop is cited 9 times, all within the Federal Register section on Definitions, which addresses the central issues raised by the proposal. The commenter notes that none of the other 9 references within this section is cited more than once. The commenter states that the U.S. NAS is perhaps the most respected source of unbiased scientific opinion in the world, so citation to the NAS would seem to provide authoritative backing for EPA's proposal, but not if it is to a workshop. The commenter provides that speakers are chosen by the NAS to accurately and fairly represent divergent points of view rather than to present a recommendation obtained through the consensus of an expert scientific committee. The commenter asserts that in this case, industry representatives were involved in the planning committee for this Workshop and three were speakers, including the industry consultant who controversially heads EPA's CASAC. Additionally, the commenter provides that the NAS report has a very strong disclaimer that EPA chose to ignore:
      The statements, recommendations, and opinions expressed are those of individual presenters and participants, and are not necessarily endorsed or verified by the [NAS] and they should not be construed as reflecting any group consensus.
According to the commenter, individual opinions summarized by rapporteurs in the Workshop report, as well as specific quotations, are citable, but only to the individual who made the comments.
Commenter (12727) notes that the original proposal's cited support proved to be either inapplicable, irrelevant, or contrary to EPA's proposed action. The commenter states that further, EPA's failure to explain its proposed departure from current Agency policies and its reversal of prior Agency conclusions on data availability are a hallmark of arbitrary decision-making. The commenter provides that the supplemental proposal drastically expands the scope of the original proposal, yet it rectifies neither its earlier citation of inapplicable, non-supportive authorities nor its failure to acknowledge or explain contradictions with current policies and prior conclusions. The commenter also states that the supplemental proposal does not cite any new authorities that support its approach aside from OMB memorandum M-19-15, which the commenter asserts does not provide support for the proposal. According to the commenter, EPA's failure to provide adequate support for its proposed course of action is the antithesis of reasoned decision making.
Response: As discussed in the preamble of the final rule, the rule is designed to build upon open data initiatives in the federal government and scientific community. Additional discussion and citations for the rationale are provided. In addition, underlying dose-response data are considered available if they can be accessed through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Further, the EPA would like to clarify that the rule does not prevent the Agency from using research with confidential data and does not rely on an Administrator's exemption to be able to consider such research. As explained in the preamble for the final rule, the EPA did not finalize the 2018 proposal or the proposed option in the 2020 SNPRM that would have categorically required that the underlying dose-response data be available for independent validation in order for studies to be considered pivotal science. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Also as discussed in the preamble to the final rule, the EPA is retaining the Administrator's exemption provision with some added considerations in response to public comments. To ensure that the Administrator's decision is appropriately transparent, in the final rule the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. This documentation would typically be provided as part of the proposed rulemaking given this will be part of the decision of what is the pivotal science for the rule. Regardless, the final rulemaking will provide clear documentation. The EPA is confident that the criteria provide sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
The EPA has also clarified references to the 2016 NAS workshop report entitled Principles and obstacles for sharing data from environmental health research: Workshop summary (Ref. 26) in the preamble text of the final rule. All references to this report in the final preamble clearly state that this was a workshop report.
2.2.1.3 The EPA Has Not Sufficiently Articulated the Problem the Rule Is Intended to Fix
Comment: Several commenters (10742, 10974, 11332, 11339, 11390, 11391, 11395, 11396, 11404, 11408, 11473, 11479, 11484, 11497, 11576, 11579, 11892, 11894, 11920, 12403, 12412, 12427, 12464, 12546, 12617, 12645, 12688, 12692, 12707, 12716, 12720, 12727, 13549) express that EPA has not sufficiently articulated the problem that the EPA is intending to fix by its proposal/supplemental proposal. These commenters generally also note that EPA has not sufficiently justified their purported effort to address "transparency," and that sufficient peer review and other mechanisms already exist to validate studies and ensure transparency. Examples of some of the specific comments include:
Commenter (11390) contends that the proposed rule disregards settled scientific standards and drums up a transparency problem that does not exist, focusing on data availability instead of study reliability. According to the commenter, it adds nothing to -- and in fact completely ignores -- the complex discussions within the scientific community about how best to achieve additional transparency and enhance replicability while maintaining commitments to patient confidentiality and production of high-quality science. 
Commenter (11473) states that the rulemaking is needless regulation seeking to solve a purported problem that does not exist. According to the commenter, sufficient safeguards already exist in the present peer review system, and the approach put forward by this proposed regulation will not achieve the Agency's claimed goals of "transparency." 
Commenter (12720) contends that it is problematic that, two years since the original proposed rule, EPA has not identified any reasons to introduce such drastic changes to the Agency's science activities through the supplemental proposal. 
Commenter (11497) states that multiple steps of independent scientific review already ensure the integrity and objectivity of science. The commenter notes that soundness of scientific methodology is thoroughly reviewed at many points in the process of designing, conducting and publishing research. The commenter adds that the competitive grant process; IRB requirements; peer-review prior to publication; the expertise and judgment of career EPA scientists when considering the strength and relevance of studies included in EPA decisions; and finally, the review of those decisions and the science they are based on by EPA's SAB all provide more than sufficient opportunities to assess the soundness of scientific studies. 
Commenter (11894) expresses that the Agency suggests that it might consider studies for which the underlying data were unavailable due to concerns about privacy, confidentiality, CBI, or national security. The commenter notes that you will search the proposal in vain for any reference to considering studies because they had repeatedly been confirmed by other studies, or because their quality had been assured by other means, or because analysis could adjust for any likely flaws, or because they were the only data available to address a pressing regulatory issue. 
Commenter (10742) notes that after a careful review of the supplemental notice of proposed rulemaking, the commenter found that the Agency has failed to fully identify the problem that finalizing this rule will solve. The commenter asserts that it is, therefore, difficult to analyze the individual components of the rule and if they will solve any problems. 
Commenter (11920) contends that, quite simply, this rulemaking addresses a problem that does not exist, and would dramatically undermine the ability of the EPA to make use of best available scientific information to protect the environment and public safety. 
Commenter (10974) states that there is no information-based demonstration of either the need for additional procedural transparency, or for furthered public understanding of EPA's critical rulemakings. The commenter contends that this proposed rule, if enacted, would provide no such clarity  -  it is a solution looking for a problem. The commenter also notes the SNPRM's reference to OMB M-19-15, and notes that the memo addresses, for all Federal executive agencies, what EPA is trying to do with this internally focused rulemaking. The commenter provides that the EPA says that this SNPRM is partly intended to "ensure consistency" with M-19-15, but, at the expense of duplicating work, and doing so to the detriment of "[e]nsuring and [m]aximizing the [q]uality, [o]bjectivity, [u]tility, and [i]ntegrity of [i]nformation", EPA has not established the need to take further action beyond following M-19-15 and the underlying guidance document. According to the commenter, the argument could be made that publishing this SNPRM in order to align a previous attempt at rulemaking to a new memo further confuses the issues at hand, rather than providing clarity. 
Commenter (11332) provides that EPA has not in the original proposed rule or in the SNPRM adequately identified any particular problem to be addressed by this unprecedented Agency action. According to the commenter, the concern was identified earlier by EPA's own SAB. The commenter notes that this was months ago in regards to the original proposal, but the Agency did not address that concern in the supplemental proposal. The commenter contends that EPA has not meaningfully engaged with the SAB in assembling its supplemental proposal, and in the supplemental proposal, EPA has not responded in any meaningful way to the major questions and concerns identified by the SAB about this rushed effort. Additionally, the commenter states that the recently finalized ISA for Fine PM, Soot Air Pollution demonstrates that the existing scientific review processes are fully functioning to capture and characterize the best available science as mandated by law and that the health and economic benefits to the American public using the current approach to parsing the best available science over the past fifty years provide no reasonable legal or scientific rationale for the SNPRM. In summary, the commenter asserts that the Agency has not adequately shown the need for this proposed regulation. To the contrary, the commenter provides that the SNPRM would ignite cascading waves of unnecessary, unworkable, and hugely expensive implementation issues and also directly enable selective interference in the science. 
Commenter (11350) states that EPA science assessments generally include an exhaustive and critical review of relevant studies and a full explanation of how they are being interpreted and adds that extensive information about each study is typically part of the public record even if all underlying data may not be included. The commenter notes that EPA assessments are normally subject to public comment and independent peer review and members of the regulatory community are free at any time to replicate studies they deem flawed, or to independently seek access to underlying data and reanalyze them. The commenter suggests that the so-called problem that the proposal seeks to fix is largely imaginary and asserts that the stakes for EPA science and the protection of public health are simply too high to finalize this deeply problematic and unnecessary proposal.
Commenters (12412, 13549) state that proposed 40 C.F.R. § 30.5 remains a poorly formed scheme in search of a problem that would ultimately force EPA to arbitrarily make public health and environmental decisions based on incomplete or improperly weighted information. 
Commenter (11395) contends that the SNPRM does not address the need for the regulation and does not provide evidence that implementation of the proposed rule would improve regulatory outcomes. The commenter provides that appropriate data sharing mechanisms already exist and major funding sources, including the NIH and EPA, require scientists to establish a data sharing plan as part of the scientific granting process. Additionally, the commenter notes that the NIH currently operates multiple data repositories and that major journals, including Lancet, the Journal of the American Medical Association, and the New England Journal of Medicine, require researchers to specify a mechanism for sharing data with other scientists consistent with NIH policies and the FAIR (findable, accessible, interoperable, reusable) principles as part of their manuscript submission to facilitate replication of findings, or pooling of data from multiple studies. The commenter asserts that EPA should focus on implementing existing initiatives and guidelines for improving data sharing and transparency at federal agencies rather than promulgating a regulation that requires additional data sharing as a condition for consideration of a study. 
Commenter (11404) provides that there is already a robust scientific information vetting process outlined in the executive orders and guidance documents referenced in the background document for this rulemaking. According to the commenter, these include Executive Order 12866: Regulatory Planning and Review; Executive Order 13563: Improving Regulation and Regulatory Review; EPA's Scientific Integrity Guidance; OMB's Guidelines Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information. 
Commenter (11576) contends that the request for comment on how to improve information availability over time and how to incentivize researchers is a request for solutions to a problem that does not exist. The commenter states that EPA has not shown that its regulatory decisions and scientific analyses have been at all compromised due to the public not having access to the models and data underlying scientific studies. 
Commenter (12688) provides that this proposal is both unnecessary, and inferior to current practices. The commenter notes that FOIA, including the Shelby Amendment, already serves the purpose of transparency, and Agency reliance on the best available science and considering all information available in rulemaking records is both a superior practice, and required by law. 
Commenter (12546) contends that the proposal continues to have fundamental flaws and this supplement will not improve the use of science for decision-making; EPA has failed to provide a reasonable science or policy rationale supporting these actions. The commenter provides that EPA and numerous other agencies (such as the FDA) routinely make regulatory decisions without having access to "all data and models." According to the commenter, lacking "access to part or all of the data and Models" or "not hav[ing] the authority to provide access to part or all of the data and models," does not prevent EPA from determining the validity of scientific methods and conclusions and using science to inform decisions, as EPA repeatedly states in its own plan to increase access to results of EPA-funded scientific research. The commenter states that overall, the proposed rule and supplemental proposal is not consistent with the principles of open science,, and continues to lack a scientific basis or policy rationale. 
Commenter (12727) states that EPA has failed to articulate a defensible rationale for this proposal or to describe what effects the proposal might produce. The commenter notes that the SAB recently observed that "[c]osts of processing and documenting data will be difficult to assess in advance until EPA has developed a system for dealing with the requirements of the rule." The commenter asserts that basic ambiguities in the proposal might further obscure potential costs, as the SAB noted with respect to the definition of "data." The commenter contends that a reasoned decision-making process would have generated more information upon which to analyze costs and benefits, rather than the near-total absence of information that we now face. According to the commenter, this lack of information is not carte blanche to forego an analysis, but rather an indication that this rulemaking is arbitrary and unsupported. 
Similarly, another commenter (11579) states that EPA has yet to offer any evidence or substantial arguments in their defense of why the proposed restrictions on science are necessary. The commenter provides that the Agency's own SAB recently noted in its comments on the proposal, "[t]here is minimal justification provided in the Proposed Rule for why existing procedures and norms utilized across the U.S. scientific community, including the federal government, are inadequate, and how the Proposed Rule will improve transparency and the scientific integrity of the regulatory outcomes in an effective and efficient manner." According to the commenter, the EPA's continued failure to provide such "justification" has significant substantive and procedural implications. Substantively, the commenter contends that the Agency's unwillingness to address the implications of its proposal has disguised the severe problems the regulation would cause. The commenter notes that, in the words of the SAB (11579): 
      It is plausible that in some situations, the Proposed Rule will decrease efficiency and reduce scientific integrity, [and] determining if in fact that will be the case requires a thorough and thoughtful examination that is currently absent in the Proposed Rule. Moving forward with altered transparency requirements beyond those already in use, in the absence of such a robust analysis, risks serious and perverse outcomes. 
Commenter (12716) states that as the Agency points out, there are numerous laws and policies that already govern the requirements that must be met for the Agency to use science in its rulemaking, including OMB's Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and Integrity of Information Disseminated by Federal Agencies and Memorandum 13-13: Open Data Policy  -  Managing Information as an Asset, EPA's 2016 Plan to Increase Access to Results of EPA-Funded Scientific Research to Significant Regulatory Decisions Plan; EPA Open Government Plan 4.0; (3) Open Data Implementation Plan; and EPA's Scientific Integrity Policy; Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental Protection Agency.
The commenter (12716) contends that an examination of the EPA's Information Quality Guidelines demonstrates that EPA is already required to have transparency in its reliance on scientific studies used in the establishment of regulations. For example, the commenter notes that in regard to ensuring and maximizing the quality of "influential" information (which includes information in support of Agency actions including rules), EPA states:
      EPA recognizes that influential scientific, financial, or statistical information should be subject to a higher degree of quality (for example, transparency about data and methods) than information that may not have a clear and substantial impact on important public policies or private sector decisions. A higher degree of transparency about data and methods will facilitate the reproducibility of such information by qualified third parties, to an acceptable degree of imprecision. For disseminated influential original and supporting data, EPA intends to ensure reproducibility according to commonly accepted scientific, financial, or statistical standards. It is important that analytic results for influential information have a higher degree of transparency regarding (1) the source of the data used, (2) the various assumptions employed, (3) the analytic methods applied, and (4) the statistical procedures employed. It is also important that the degree of rigor with which each of these factors is presented and discussed be scaled as appropriate, and that all factors be presented and discussed. In addition, if access to data and methods cannot occur due to compelling interests such as privacy, trade secrets, intellectual property, and other confidentiality protections, EPA should, to the extent practicable, apply especially rigorous robustness checks to analytic results and carefully document all checks that were undertaken. Original and supporting data may not be subject to the high and specific degree of transparency provided for analytic results; however, EPA should apply, to the extent practicable, relevant Agency policies and procedures to achieve reproducibility, given ethical, feasibility, and confidentiality constraints.
In addition to all of the already existing laws and guidance under which EPA is directed to review scientific information in its promulgation of regulations, the commenter (12716) notes that EPA also has "several mechanisms for vetting science through several expert panels, including the EPA SAB, the EPA CASAC and the EPA [Federal Insecticide, Fungicide and Rodenticide] FIFRA Scientific Advisory Panel . . .." Altogether, the commenter asserts that these safeguards are sufficient to ensure that the science relied upon by EPA is the best available science and that it is subject to peer review and available to the public.
Commenter (12427) states that EPA is trying to "solve" a "problem" that doesn't exist. According to the commenter, there is not a problem with "transparency" in current scientific practice; reputable journals encourage or require complete disclosure of data and models, with the reasonable exception in cases where disclosure would infringe on patient right to privacy or would disclose confidential information. Thus, the commenter contends that EPA is attempting to solve a problem that doesn't exist. The commenter provides that PLOS One requires full disclosure but also instructs that "All efforts should be made to protect patient privacy and anonymity." Additionally, the commenter states that EPA should address the following questions:
 What is the current practice among scientific journals regarding "transparency"? (answer: journals already promote this)
 What is the actual problem? What actual harm has been done because of lack of transparency? In what ways have existing peer review mechanisms failed to address studies that lack "transparency" but that are otherwise rigorous?
  If there is a problem, is a regulation the best approach? Why not do some test cases? Why not a white paper? Why not develop an internal guidance document?
Commenter (12727) contends that there is a plethora of evidence that EPA's existing approaches to evaluation of scientific studies, models, and other information underlying influential scientific information are not only functioning well, but are exemplary. The commenter states that EPA's requirement of peer review of all influential scientific information assures that influential scientific information and underlying information are rigorously examined both within and without the Agency before dissemination or other utilization. Furthermore, the commenter asserts that EPA's Scientific Integrity Policy successfully promotes use of best science by not imposing a priori constraints on which information is to be used, but rather by considering data on its individual merits, unconstrained by political or other interference. The commenter notes that although the policy recognizes the importance of the ability to independently validate scientific information and methods, as well as the importance of access to data and non-proprietary models used to support Agency action, these values are not absolute rules barring or downgrading consideration of legitimate data and information. The commenter contends that EPA's traditional weight-of-evidence approach to data evaluation has express judicial imprimatur, as does its decision to consider studies for which not all underlying data are publicly available for legitimate reasons.
Response: The final rule provides more detail explaining the EPA's rationale for this rule. In brief, the EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. This rule is designed to build upon OMB M-19-15, which highlights the need to characterize the sensitivity of an agency's conclusions to analytic assumptions, as well as other federal guidance documents that require greater data transparency. The EPA's attention to data transparency is also responsive to the broader interest in greater data and model transparency observed in the numerous transparency initiatives in the scientific community and federal government, as well as the criticism the EPA has received from members of the public, scientific community, and Congress on the transparency of the scientific basis for EPA's decisions in previous influential scientific information assessments and regulatory actions. The EPA's continued progress toward maximizing transparency is vital to building and maintaining trust with the public and credibility in the Agency's decisions. Further, although the scientific community and government agencies are making great strides in data transparency, improvements can still be made over existing policies and mechanisms. Therefore, the rule requirements for the EPA's independent evaluation of the availability of data are necessary and critical to prioritizing data transparency in the pivotal science underlying its significant regulatory actions and influential scientific information. Although current methods and processes do allow for public comment and, where appropriate, peer review in the development of significant regulatory actions and influential scientific information, subject matter experts and peer reviewers (including the SAB and other science advisory panels) generally have not had access to the underlying data necessary to fully evaluate scientific conclusions. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information. The promulgation of these internal Agency procedures in this rulemaking highlights the Administration's commitment to ensuring transparency in the EPA's most impactful regulatory actions and influential scientific information.
In addition, as explained in the preamble for the final rule, the EPA did not finalize the 2018 proposal or the proposed option in the 2020 SNPRM that would have categorically required that the underlying dose-response data be available for independent validation in order for studies to be considered pivotal science. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
The EPA agrees that data sharing mechanisms already exist. The preamble to the rule clarifies that restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. The rule does not prescribe a particular mechanism but does require the EPA to independently verify that such a mechanism exists for dose-response data underlying pivotal science.
The EPA received comments from the SAB on the 2018 NPRM and 2020 SNPRM on April 24, 2020. The EPA considered these comments in development of the final rule. In brief, the SAB consensus was that it supported the precept of enhancing public access to scientific data and analytical methods to ensure scientific integrity, consistency, and robust analysis and their consensus recommendations were that the EPA develop additional policy and/or guidance documents; further describe how pivotal science would be selected, made available, and peer reviewed; consider establishing an office on data sharing with the support of a peer review panel; provide greater clarity in the definitions of key terms, including data, independent validation, and publicly available; develop specific criteria for the Administrator's exemptions under the rule; consider applying rule requirements only to information developed after the effective date of the rule; seeking input from experts to identify best practices to ensure efficiency, utility, and protection of data that are made available; and consider funding reanalysis work. 
As noted in the preamble for the final rule, this rule sets the overarching structure and principles for transparency of pivotal science in significant regulatory actions and influential scientific information and technical considerations, including those noted by the SAB, could be addressed in subsequent implementation guidelines and/or statute-specific rulemakings. Further, in response to comments from SAB and the public, the EPA clarified and modified several key definitions, including data, independent validation, and publicly available, in the final rule. The EPA also provided criteria for the Administrator's exemption in 40 CFR 30.7 of the final rule. The EPA would like to clarify that this rule does not direct or require any outside entity or the EPA to establish data sharing mechanisms and does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science. Comments related to making data available and future funding opportunities are not within the scope of this rulemaking. Finally, the EPA is maintaining that the rule provisions apply equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. However, in consideration of instances where data cannot be made available, the EPA amended the rule to provide technical considerations (40 CFR 30.5(d)) that can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability in the absence of maximum transparency and included an exemption criterion -- that the development of the dose-response data was completed or updated before the effective date of the rule -- to 40 CFR 30.7 to account for the concerns raised by the SAB and other public commenters.
Comment: Commenter (12427) asserts that the need to establish the quality of published information described in the proposed rule is accomplished through peer view, as discussed in the following sentence from the 2004 OMB Information Quality Bulletin for Peer Review: "Peer review is one of the important procedures used to ensure that the quality of published information meets the standards of the scientific and technical community." The commenter asserts that science should be evaluated by the scientific and technical community, not by political appointees of EPA. The commenter contends that the transparency rule and SNPRM vest decisions about what studies to exclude from these proposals solely with the politically-appointed Administrator and do not include any science-based criteria for making such decision, nor are such decisions vested in a scientific advisory committee. Furthermore, the commenter states that there are existing scientific Federal advisory committees that serve EPA that already review, evaluate, and assess scientific knowledge pertinent to regulatory and other decision making. The commenter notes that yet, EPA is silent regarding the important role of peer review and fails to explain why existing peer review mechanisms are inadequate. 
The commenter asserts that peer review related to scientific rigor and not the narrow definition of "transparency" that EPA is attempting to codify into a regulation. Additionally, the commenter asserts that the lack of consideration for CBI and intellectual property in the proposed rule is inconsistent with current practice in scientific journals, and OMB's Information Quality Bulletin for Peer Review. The commenter also expressed concern for patient privacy.
Response: The EPA agrees that existing mechanisms within the EPA, the federal government, and in the scientific community are increasingly pushing towards greater data sharing and increasing scientific rigor. The transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency. Although current methods and processes do allow for public comment and, where appropriate, peer review in the development of significant regulatory actions and influential scientific information, subject matter experts and peer reviewers (including the SAB and other science advisory panels) generally have not had access to the underlying data necessary to fully evaluate scientific conclusions. The final rule also provides that the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA believes that the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
Also as discussed in the preamble to the final rule, the EPA is retaining the Administrator's exemption provision with some added considerations in response to public comments. To ensure that the Administrator's decision is appropriately transparent, in the final rule the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. This documentation would typically be provided as part of the proposed rulemaking given this will be part of the decision of what is the pivotal science for the rule. Regardless, the final rulemaking will provide clear documentation. The EPA is confident that the criteria provide sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
Commenter (12427) contends that EPA has failed to justify that its existing peer review mechanisms fail to address scientific rigor in assessing science relevant to regulatory decision making. The commenter provides that peer review mechanisms already available to EPA include but are not limited to scientific Federal advisory committees (e.g., EPA SAB, EPA's CASAC, EPA FIFRA Scientific Advisory Panel), as well as ad hoc reviews (organized as panels or based on reviews from multiple individuals). Furthermore, the commenter states that for changes that may broadly affect the scientific enterprise, EPA has routinely sought advice from the NASEM, including but are not limited to, Risk Assessment in the Federal Government: Managing the Process; Review of EPA's Integrated Risk Information Systems; Science and Judgment; Science and Decisions; and many others. According to the commenter, EPA has failed to engage CASAC, the SAB, or any other of its advisory committees on the transparency rule or its supplement, has not engaged external experts to review any scientific work products for this proposed rule and supplement, and has not engaged NASEM.
Response: The EPA agrees that existing mechanisms within the EPA, the federal government, and in the scientific community are increasingly pushing towards greater data sharing and increasing scientific rigor. The transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency. Although current methods and processes do allow for public comment and, where appropriate, peer review in the development of significant regulatory actions and influential scientific information, subject matter experts and peer reviewers (including the SAB and other science advisory panels) generally have not had access to the underlying data necessary to fully evaluate scientific conclusions. This rule provides that the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA believes that the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
The EPA's SAB provided advice and comments on mechanisms for secure access to PII and CBI under the proposed rule on September 30, 20 and the proposed and supplemental rulemakings on April 24, 2020. All of these comments were considered in developing the final rule. In brief, the SAB consensus was that it supported the precept of enhancing public access to scientific data and analytical methods to ensure scientific integrity, consistency, and robust analysis and their consensus recommendations were that the EPA develop additional policy and/or guidance documents; further describe how pivotal science would be selected, made available, and peer reviewed; consider establishing an office on data sharing with the support of a peer review panel; provide greater clarity in the definitions of key terms, including data, independent validation, and publicly available; develop specific criteria for the Administrator's exemptions under the rule; consider applying rule requirements only to information developed after the effective date of the rule; seeking input from experts to identify best practices to ensure efficiency, utility, and protection of data that are made available; and consider funding reanalysis work. 
As noted in the preamble for the final rule, this rule sets the overarching structure and principles for transparency of pivotal science in significant regulatory actions and influential scientific information and technical considerations, including those noted by the SAB, could be addressed in subsequent implementation guidelines and/or statute-specific rulemakings. Further, in response to comments from SAB and the public, the EPA clarified and modified several key definitions, including data, independent validation, and publicly available, in the final rule. The EPA also provided criteria for the Administrator's exemption in 40 CFR 30.7 of the final rule. The EPA would like to clarify that this rule does not direct or require any outside entity or the EPA to establish data sharing mechanisms and does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science. Comments related to making data available and future funding opportunities are not within the scope of this rulemaking. Finally, the EPA is maintaining that the rule provisions apply equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. However, in consideration of instances where data cannot be made available, the EPA amended the rule to provide technical considerations (40 CFR 30.5(d)) that can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability in the absence of maximum transparency and included an exemption criterion -- that the development of the dose-response data was completed or updated before the effective date of the rule -- to 40 CFR 30.7 to account for the concerns raised by the SAB and other public commenters.
Comment: Commenter (12430) notes that congressional directives already require U.S EPA to consider whether scientific information is reasonable, clear, and complete and whether it has been peer reviewed. The commenter states that existing mechanisms complement these directives, including: the SAB, established by Congress in 1978 to advise on scientific matters; existing independent peer review of much of the scientific research relied on for regulatory action and publication; legal requirements that prohibit U.S. EPA's adoption of regulations that are arbitrary, capricious, or unsupported by substantial evidence; and procedures to request reconsideration of Agency decisions and correction of Agency information. Additionally, the commenter contends that the process for setting the NAAQS illustrates the scientific review procedures that are already in place for EPA's regulatory actions. The commenter provides that after determining weight by independent peer review, EPA staff and their contractors review the literature and subject it to further internal review, they develop a staff report. The commenter states that CASAC, which consists of internationally recognized experts in their scientific disciplines, reviews the EPA staff report, and in turn provides advice to the Administrator regarding the adequacy of current standards and recommendations for revisions, if necessary, for the protection of public health. Additionally, the commenter states that the reports receive public comment, including from independent and industry scientists, and the CASAC review includes consideration of those public comments. According to the commenter, the EPA NAAQS documents typically receive multiple rounds of expert scientific review before they are finalized. 
Commenter (12715) states that EPA never explains, in either the initial proposal or the SNPR, how its new approach will improve the Agency's scientific decision-making and references a 2019 joint statement of the editors of leading scientific journals. The commenter contends that the scientific bases of EPA risk and policy assessments are already peer reviewed by panels such as the EPA SAB panels, CASAC, FIFRA Science Advisory Panels, and the TSCA Science Advisory Committee to ensure that the evaluation of studies is performed using appropriate scientific criteria. According to the commenter, these reviewers have ample tools to evaluate the merit and robustness of scientific studies and information, including assessing: the quality of a study's design; the reasonableness of any assumptions; whether sample sizes are sufficient; whether the evidence is strong enough to support a conclusion; and how the study compares with other studies in the same subject matter area. The commenter asserts that EPA does not explain why these existing review criteria are insufficient or how its new, arbitrary requirements will add "transparency" or improve EPA's decision-making.
Response: The EPA agrees that existing mechanisms within the EPA, the federal government, and in the scientific community are increasingly pushing towards greater data sharing and increasing scientific rigor. The transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency. Although current methods and processes do allow for public comment and, where appropriate, peer review in the development of significant regulatory actions and influential scientific information, subject matter experts and peer reviewers (including the SAB and other science advisory panels) generally have not had access to the underlying data necessary to fully evaluate scientific conclusions. This rule provides that the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA believes that the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings
Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information. The promulgation of these internal Agency procedures in this rulemaking highlights the Administration's commitment to ensuring transparency in the EPA's most impactful regulatory actions and influential scientific information.
Comment: Commenter (11391) states that in its notice announcing the 2018 proposed rule, EPA claimed that it is needed to ensure the validity of the scientific information relied upon by the Agency, and enhance the transparency of regulatory processes, and while the earlier comments submitted by the commenter and others clearly show that is not the case, EPA has doubled-down on its claim, but again offered no evidence to support it.
The commenter (11391) notes that it is well established that Agency actions must be based on a consideration of relevant evidence and accompanied by a clear statement of how that evidence supports the action taken. The commenter states that as the Supreme Court explained in Federal Commissions Commission v. Fox Television Stations, Inc., an Agency must supply "good reasons" for reversing a previously held position, and that those reasons must be especially compelling where, as here, the agency's new position "rests on factual findings that contradict its prior policy." In such cases, the commenter provides that the Agency must provide "a more detailed justification than what would suffice for a new policy created on a blank slate," including a "reasoned explanation" for disregarding the facts and circumstances on which its prior policy was based. According to the commenter, in the absence of such an explanation, the Agency's action must be considered arbitrary and capricious in violation of the APA. 
As such, the commenter (11391) states that EPA has not provided any justification for its newfound view that the validity of scientific research can only be assessed if the underlying data are disclosed. The commenter contends that this view is at odds with the widespread consensus within the scientific community that existing peer review processes, which do not require data disclosure, are sufficient to ensure the validity of scientific research. The commenter notes that recognizing this, in 2016 EPA concluded that the availability of underlying data "does not affect the validity of the scientific conclusions from peer-reviewed research publications." Now, less than four years later, EPA has suddenly changed its view. According to the commenter, neither the 2018 Notice nor the 2020 SNPRM acknowledge the change, much less provide a reasoned explanation for it.
Furthermore, the commenter (11391) states that in the 2020 SNPRM, EPA baldly asserts that the Proposed Rule is needed to ensure Agency practices are consistent with new guidance from the OMB, but does not explain why, nor could it, since the referenced guidance -- an April 2019 Memorandum on Improving Implementation of the Information Quality Act -- does not require EPA to adopt the Proposed Rule or even support its stated reasons for doing so. The commenter asserts that whereas EPA claims that access to underlying data is needed to ensure the validity of scientific research, the memorandum indicates that this can be achieved through peer review.
The commenter (11391) also points to EPA's reference to OMB's 2002 Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies but, again, they do not support adoption of the Proposed Rule. The commenter contends that while the Guidelines do call for increased transparency in science, the provisions in the proposed rule that require disclosure are inconsistent with provisions in the Guidelines that indicate that transparency does not necessarily require disclosure of the data underlying research studies, that such disclosure can raise ethical and/or other issues, and that balances the need for transparency against "other compelling interests such as privacy, trade secrets, intellectual property, and other confidentiality protections." The commenter asserts that rather than attempt such a balance, EPA has declared a blanket standard, under which it must ignore or give lesser weight to studies for which the underlying data are not available, even if those studies have been independently verified (e.g., through peer review) and the researchers have valid reasons for withholding the underlying data.
Finally, the commenter (11391) notes that the 2018 Proposed Rule applied only to dose-response data and models underlying pivotal regulatory science that is used to justify significant regulatory decisions and that EPA is now proposing to expand the scope of the Proposed Rule in two ways:
      1. the Proposed Rule would apply not only to the science relied upon in promulgating significant regulatory decisions, but also in finalizing influential scientific information; and 
      2. the Proposed Rule would apply to all data and models underlying such science, instead of only dose-response data and models. 
The commenter (11391) contends that EPA offers no explanation for the first change. With respect to the second, the commenter provides that EPA asserts that "[t]ransparency of EPA's science should not be limited to dose-response data and dose-response models, because other types of data and models will also drive . . . significant regulatory decisions and influential scientific information." The commenter asserts that EPA has still not explained why its version of "transparency" is necessary or appropriate.
Response: The EPA agrees that existing mechanisms within the EPA, the federal government, and in the scientific community are increasingly pushing towards greater data sharing and increasing scientific rigor. The transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency. Although current methods and processes do allow for public comment and, where appropriate, peer review in the development of significant regulatory actions and influential scientific information, subject matter experts and peer reviewers (including the SAB and other science advisory panels) generally have not had access to the underlying data necessary to fully evaluate scientific conclusions. This rule augments, but does not replace, existing Agency procedures for evaluating science underlying significant regulatory actions and influential scientific information. As further discussed in the preamble to the final rule, the EPA is focusing on dose-response data in this rule to pursue an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. When implemented, the rule provides that the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA believes that the implementation of this final rule will increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information. The promulgation of these internal Agency procedures in this rulemaking highlights the Administration's commitment to ensuring transparency in the EPA's most impactful regulatory actions and influential scientific information.
In addition, the EPA clarified 40 CFR 30.5 to give equal consideration to dose-response data that are publicly available or available through restricted access. As discussed in the final rule, underlying dose-response data are considered available if they can be accessed through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. The EPA believes that the final rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making.
In the preamble to the final rule, the EPA clarifies that although the availability of dose-response data underlying a study in a manner sufficient for independent validation is an important component of determining the level of consideration to afford a study, the EPA agrees that availability by itself is not sufficient to determine study quality or validity. As explained in 40 CFR 30.5, the EPA will use existing factors (including soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review) to evaluate study quality. Subsequently, after identifying the highest quality, most relevant studies that would inform a dose-response assessment and identifying the availability of pivotal science, the EPA would consider additional applicable factors as described in 40 CFR 30.5(d) when determining the level of consideration to give pivotal science where the underlying dose-response data are not available for independent validation.
The EPA disagrees with commenters that peer review can achieve the goals of this rulemaking. Peer review does not typically include reanalysis of the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information) and, thus, peer review is not considered a replacement for the data availability requirements of this rule. Jamieson et al. (2019) state, "Journals play a key role in ascertaining the quality of the evidence in work submitted for publication through peer review. However, that practice is neither uniform nor free of exploitation. While it can select for trustable results, its existence alone is not sufficient to verify the quality of the evidence used to back claims." The authors go on to say that additional confidence could be gained from third-party reanalyses that independently validate the study results. In addition, as detailed in OMB's Final Information Quality Bulletin for Peer Review, "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." The scientific community has also embraced greater data transparency, as evidenced by data sharing and availability requirements for many high-impact journals and the emergence of organizations, such as the Center for Open Science, and international initiatives like FAIR data principles; FOSTER; and Guidelines for TOP in Journal Policies and Practices that incentivize greater transparency in research.[,] The EPA supports these efforts and believes that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety.
2.2.1.4 Basis for Proposal is Valid
Comment: Commenter (11401) states that they understand why EPA felt it necessary to supplement the rulemaking due to the Proposal's mischaracterization as an effort to block science. The commenter contends that this is not the case and that EPA is well within its authority to carry out the provisions of the SNPRM and it is a laudable effort to hold itself to scientific standards, as is only right and equitable as it holds companies and citizens to strict environmental standards that cost society billions of dollars every year.
Commenter (12692) contends that the Environmental Protection Network's assertion the rulemaking has no problem it is trying to solve is flawed. The commenter provides that it is a bit much to demand that EPA produce a list of its own "bad decisions" as any such determination would require a rulemaking of its own. The commenter states that the core thesis of the Transparency proposal is not that EPA knows the studies it relies on lead to "bad" decisions but that, in many cases, it cannot properly evaluate those decisions because the underlying studies incapable of independent validation. The commenter asserts that quality control via the "marketplace of ideas" has been precluded.
Additionally, the commenter (12692) states that the "replication crisis" is very real and that many studies published in prestige journals turn out to be junk. The commenter references The Economist, in an editorial cited by the Transparency proposal, that explained the problem as follows:
      A simple idea underpins science: "trust but verify." Results should always be subject to challenge from experiment. That simple but powerful idea has generated a vast body of knowledge. Since its birth in the 17th century, modern science has changed the world beyond recognition, and overwhelmingly for the better.
The commenter (12692) provides that success can breed complacency and that modern scientists are doing too much trusting and not enough verifying -- to the detriment of the whole of science, and of humanity. According to the commenter, too many of its findings that fill the academic ether are the result of shoddy experiments or poor analysis (see pages 26-30). The commenter asserts that a rule of thumb among biotechnology venture-capitalists is that half of published research cannot be replicated, even that may be optimistic. The commenter notes Amgen and Bayer have failed to reproduce many research projects, as well as the doubt of a leading computer scientist that frets that three quarters of papers in his subfield are bunk. The commenter also references that between 2000-10 roughly 80,000 patients took part in clinical trials based on research that was later retracted because of mistakes or improprieties.
According to the commenter (12692), The Economist identifies three main causes of replication failure: The publish-or-perish imperative of academic life; the associated career-related inducements to exaggerate, cherry pick results, and spin causality out of chance correlations; publication bias against studies finding negative results; and lax peer review. The commenter provides that for example, the article reports, "When a prominent medical journal ran research past other experts in the field, it found that most of the reviewers failed to spot mistakes it had deliberately inserted into papers, even after being told they were being tested." The commenter also reiterates Lutter and Zorn's concern over replicability.
Response: The EPA agrees with commenters on the importance of transparency and scientific scrutiny and debate in creating sound environmental policy. Because of the potential impact of the EPA's significant regulatory actions and influential scientific information on American lives and livelihoods, the American people deserve environmental decisions and policies that are based on the best information. Only through continuous improvement to its procedures, especially those focused on transparency, can the EPA fully demonstrate that commitment.
2.2.1.5 Prior Basis for Rule
2.2.1.5.1 Previous Environmental Rulemakings Necessitate Transparency
Comment: Commenter (9335) points to CASAC's 2019 Draft Review of the Particulate Matter Policy Assessment (PM PA) as a basis for the Proposed Rule. The commenter states that their comments reflect the need for transparency as follows: 1) there is NO causal relationship between PM2.5 and total mortality in the US, 2) the PM PA cites `positive authors' and omits `null authors' and their criticism, 3) the PM PA does not address the PM2.5 deaths controversy, 4) analyses of underlying data for four key US cohorts, including the Harvard Six Cities Study and the American Cancer Society Cancer Prevention Study II, support the need for the proposed EPA Transparency Rule, and 5) the PM PA must be revised to incorporate the CASAC Review and the criticisms by the commenter and others.
Response: Although the rulemaking is not related to any specific EPA rulemaking or particular study, the EPA agrees with the commenter that transparency and scientific scrutiny and debate is integral to creating sound environmental policy.
Comment: Commenter (9588) states that there is a very real need to review science and notes the FDA's five-step process for drug approval, EPA's chemical review process under the TSCA, and the OMB's requirement for review of all new regulations as a need for review. The commenter also provides that peer-review is a necessity at most legitimate science journals and considered the question of grantees serving on review committees, siding with the financial independence of advisory committee members. Additionally, the commenter notes several critiques ofgovernment transparency rules and their importance as they relate to science and policymaking.[,][,][,][,]
Response: The EPA agrees with the commenter on the importance of transparency and scientific scrutiny and debate in creating sound environmental policy. Because of the potential impact of the EPA's significant regulatory actions and influential scientific information on American lives and livelihoods, the American people deserve environmental decisions and policies that are based on the best scientific information. Only through continuous improvement to its procedures, especially those focused on transparency, can the EPA fully demonstrate that commitment. The EPA would also like to clarify that reviews conducted by other federal agencies and OMB are outside the scope of this rulemaking.
Comment: Commenter (11899) contends that past EPA rulemakings, including the Clean Power Plan and Endangerment Findings, were often not only secretive and arbitrary, but also adversely affected job creation and retention; the price and availability of the energy, food and consumer products needed in the daily lives of all Americans; and ultimately the health and welfare of millions of families, especially poor, minority and blue-collar families in states that rely on coal and natural gas for electricity, industries and factories for jobs and salaries  -  and on other activities that may have some adverse effects on environmental quality and human health and welfare, but also improve our environment, health and welfare in significant ways. The commenter provides that this negative impact provides a compelling case for the threat of EPA's rules on health, welfare, and pursuit of happiness, justice and civil rights progress of our members, the people we represent, and all Americans, than do the damages of any environmental contaminants being regulated, especially when based on secret science and pressure politics.
Response: Although the rulemaking is not related to any specific EPA rulemaking or particular study, the EPA agrees with the commenter that transparency and scientific scrutiny and debate is integral to creating sound environmental policy. Because of the potential impact of the EPA's significant regulatory actions and influential scientific information on American lives and livelihoods, the American people deserve environmental decisions and policies that are based on the best information. Only through continuous improvement to its procedures, especially those focused on transparency, can the EPA fully demonstrate that commitment.
2.2.1.5.2 Roots in Previous Legislation
Comment: Commenter (11392) states that the rulemaking's legacy lies in the failed Secret Science Reform Acts of 2014' and 20152 and the Honest and Open New EPA Science Treatment (HONEST) Act of 20173, bills that passed out of the House Committee on Science, Space, and Technology under Republican leadership but that the Senate did not see fit to consider. The commenter contends that this attempt to implement the provisions of failed legislation through rulemaking is an inappropriate circumvention of the authorities granted to the executive branch by the Constitution, and that the executive branch cannot exercise such legislative powers.
Response: The EPA disagrees that this rule is politically motivated, as transparency assumes no political ideology. The EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. The EPA is committed to furthering this effort with the implementation of this final rule. The EPA drafted this rulemaking independent of any external parties. 
Comment: Commenter (11890) notes that many of the same arguments were made in support of the Shelby Amendment of 1999. Following passage, the commenter states that broad demand for this public access never materialized. According to the commenter, the amendment and subsequent OMB Circular A-110 included provisions for public access to data from federally funded research. The commenter states that the mechanism providing access included procedures established under the FOIA. The commenter provides that an investigation into requests to EPA made between 2002 and 2012 for underlying study data (beyond published findings) found only two such requests. Both were granted through the FOIA mechanism provided by the Shelby Amendment. The commenter contends that this is hardly evidence of broad public demand for underlying study data.
Conversely, commenter (11915) states that the underlying intent of these reforms is grounded in the broadly agreed upon principle that citizens should have a right to the data and information that are used in the development of public policy. The commenter contends that this spirit of openness with respect to the regulatory process is found throughout government; it is enshrined in statute (known as "the Shelby Amendment") and countless federal directives and EPA memos reinforce the principle and detail guidance for implementing it. The commenter notes that publicly available data and information is also supported by experts of all political stripes, referencing a 2012 congressional testimony where White House science advisor Dr. John Holdren stated that "absolutely, the data on which regulatory decisions and other decision are based should be made available to the Committee and should be made public."
Response: The EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. This rule is designed to build upon OMB M-19-15, which highlights the need to characterize the sensitivity of an agency's conclusions to analytic assumptions, as well as other federal guidance documents that require greater data transparency. The EPA's attention to data transparency is also responsive to the broader interest in greater data and model transparency observed in the numerous transparency initiatives in the scientific community and federal government, as well as the criticism the EPA has received from members of the public, scientific community, and Congress on the transparency of the scientific basis for EPA's decisions in previous influential scientific information assessments and regulatory actions. The EPA's continued progress toward maximizing transparency is vital to building and maintaining trust with the public and credibility in the Agency's decisions.
2.2.2 Effects
2.2.2.1 Limits Available Science
Comment: Several commenters (9372, 10045, 10055, 10740, 10748, 10749, 10751, 10752, 10753, 10865, 10944, 10969, 10973, 10974, 11176, 11183, 11192, 11330, 11339, 11340, 11347, 11349, 11350, 11354, 11359, 11360, 11377, 11387, 11390, 11392, 11395, 11396, 11402, 11403, 11404, 11408, 11479, 11484, 11489, 11490, 11495, 11496, 11497, 11498, 11576, 11894, 11903, 11911, 11918, 11919 , 11921, 12404, 12405, 12412, 12415, 12422, 12427, 12430, 12431, 12435, 12442, 12443, 12456, 12458, 12464, 12466, 12474, 12688, 12540, 12645, 12701, 12707, 12712, 12715, 12716, 12720, 12727, 13549, 13788) that oppose the proposed rule note that the proposed rule would limit the use of science (and/or "best available science") valuable research from being considered in the EPA rule making process and that the supplemental proposal compounds the problem by expanding the scope of science that the rule would apply to. Additional comments and responses to comments on the implications for health and environmental protections are included in Section 2.2.2.3 of this Chapter. Commenters claim that most of the best research does not use publicly available data. Several examples of specific comments follow:
Commenter (11403) notes that providing raw data is not often a requirement for scientific publication. The commenter states that this does not mean that scientists should ignore any study that does not provide raw data; it simply means the peer review process ensures that data analysis is done properly. The commenter provides that this has been the scientific standard for more than 100 years. As noted by many commenters, the commenter states that raw data from epidemiological studies are often not made public in order to protect patients' identities as required by federal law. The commenter notes that many epidemiological studies that measure exposure to and harm from pollutants look at vulnerable populations in order to get a sense of how these populations respond compared to the general population. For instance, the commenter states that exposure to sulfur dioxide in the air is particularly worrisome for young people who have asthma, and exposure to pesticides tend to have a greater effect on the farmworker population due to disproportionately high chronic exposures.
Commenter (12442) contends that the EPA's supplemental rule proposal unnecessarily modifies which studies can be considered and unscientifically reduces the consideration of valuable and necessary studies that aid the Agency in setting policy to protect public health and the environment.
Commenter (11484) states that while they agree in principle that the increase of transparency in regulatory science is beneficial, that increase cannot come at the expense of EPA's statutory requirement to rely on the best available science.
Commenter (12435) contends that past, current and proposed research would be stifled by the SNPRM, due in part to state privacy rules. The commenter states that the proposed regulations force a false, illegal and unreasonable choice between protecting privacy and protecting public health.
Commenters (12435, 12456) express concern that, if the current language proposed by the SNPRM is adopted, studies conducted by researchers in Colorado would no longer be given weight because the underlying data would not be publicly available in accordance with Colorado Board of Public Health Rules, regardless of the quality of the study and the relevance of the information. The commenters state that, although these studies can be useful and even essential when EPA considers new rules or changes to existing regulations, current Colorado Board of Public Health Rules limit the availability of data collected by health departments to protect privacy. 
Commenter (12435) states that the proposed rule may unjustifiably exclude water sampling data collected by the Water Quality Control Division and entities within Colorado due to conflicts with the Colorado Open Records Act. The commenter states that the EPA should have access to all sampling data in Colorado while developing its guidance and rules, and the use of this data should not be at the discretion of the EPA Administrator, or excluded from consideration because the data was collected from drinking water system intakes or private wells.
Commenter (12456) states that high-quality research by Colorado Parks and Wildlife has been used by EPA to develop cadmium, copper, lead, silver and zinc criteria to protect aquatic life. The commenter states that the EPA should have access to all high-quality data, studies, and models while developing its guidance and rules, and the use of this data should not be at the discretion of the EPA Administrator, or excluded from consideration because all or part of study took place on private lands in Colorado. The commenter urges the EPA to change the 2020 proposed rule to promote the use of all high-quality data and studies in EPA's decision making, including studies that may include data that is not publicly available.
Commenter (11498) notes that the widening of the scope won't just limit EPA's use of science underpinning regulation, but also EPA's use of science underpinning other significant outputs of the Agency. According to the commenter, the changes mean EPA will likely be unable to cite important studies on topics relating to the levels of contaminants in water, air and land; epidemiological studies that describe clinical markers of exposure or effect; and many other studies that are fundamental in understanding and protecting human health.
Commenter (11894) states that, in short, adopting this proposal would do even more than adoption of the initial proposal to bar the Agency from basing its decisions on the best available science as the law requires. 
Commenter (12707) states that the changes provided in the supplemental rule fail to clarify how EPA will approach long standing scientific studies  -  it only creates further confusion and broadens the scope of science that will be covered. According to the commenter, the initial proposal and this supplemental proposal are a next step in a very transparent effort to discredit and make unavailable certain seminal studies that establish the connection between exposure to air pollution and adverse public health impacts.
Commenter (12464) notes that in their comments on the initial proposal, they explained that the proposed rule would prevent EPA from relying on studies involving confidential human health data, thus violating the Agency's duty to rely on the best available science. The commenter states that the supplemental proposal offers two alternatives that purportedly address this concern, one allowing EPA to consider studies without publicly-available data "if there is tiered access to these data and models in a manner sufficient for independent validation," 85 Fed. Reg. at 15,399, and the other giving lesser weight to studies whose data are not publicly available, id. The commenter contends that while the supplemental proposal presents the proposed and alternate versions of 40 C.F.R. 30.5 as solutions to concerns that they and many others raised regarding the initial proposal, the new provisions are in fact no improvement.
Commenter (11330) states that the 2018 proposed rule would have allowed the EPA to selectively exclude studies with conclusions they found unfavorable. The commenter notes that he was joined by over one hundred members of Congress, a thousand scientists, and the leading scientific advocacy organizations in America in condemning the proposal. Clearly, according to the commenter, the EPA did not heed their call. The commenter contends that the path the EPA chose given this blowback was to release a supplemental rule that effectively does the same thing. The commenter adds that the supplemental proposal allows the EPA to prioritize studies, and not just for rulemaking, for all EPA activities.
Commenter (11489) states that the proposed rule and SNPRM serve only to limit the studies the EPA can consider to those in which the data is publicly available, regardless of quality and value. The commenter states that these efforts limit the EPA's own ability to engage in evidence-based policymaking and do not address a legitimate need, as the EPA already has the authority to determine which studies it will consider during rulemaking. 
Commenter (11903) states that under its first option, the EPA is proposing rules forbidding the use of scientific evidence, including evidence from "high-quality studies", without regard to relevance, in a way that will obviously impose costs on others, by narrowing the indicia that scientists routinely use when determining what evidence is reliable. Under its second option, EPA proposes to bind itself to a complex decision-making contraption, requiring EPA and the public to interpret and apply at least ten newly promulgated regulatory definitions, all as a predicate to allow EPA to give "greater consideration" to certain data.
Commenter (12431) expresses concern that in its desire to ensure that the data underlying its significant regulatory actions and influential scientific information are publicly available, the EPA may exclude information that is most relevant to its assessment of risks to human health and the environment. In particular, the commenter states that excluding information that cannot be made publicly available would slow the development and implementation of alternative test methods and strategies to replace animal testing. The commenter adds that expanding the scope of this rulemaking would obviously compound the issue of potentially excluding information relevant to EPA's regulatory actions and scientific information.
Commenter (11183) states that the supplemental notice extends its reach. The commenter notes that the EPA's Risk Assessment paradigm has 3 components: Hazard Identification, Dose Response Analysis and Exposure Assessment, that then feed into the final Risk Characterization. The commenter notes that the proposal highlighted only the dose response data for limitation; the supplemental notice expands it to include all the components of the Risk Assessment. The commenter contends that limiting the use of the best science, viz human exposure and health data, throughout the Risk Assessment will eviscerate the science. Similarly, the commenter states that the supplemental notice expands the application of this policy from only regulatory decisions to include other `influential scientific information', that is, all the business of EPA.
Commenter (12427) states that the combination of the proposed rule and supplemental notice will have the effect of restricting the use of rigorous scientific studies with the following features:
 Health studies with PII
 CBI
 Intellectual property
 Older studies for which the data and models may no longer be available.
 International studies (e.g., because privacy and confidentiality restrictions in other countries may not enable full disclosure of raw data)
Commenter (10055) specifically asserts that the proposed supplement would throttle the science that supports standard-setting for the future and for the renewal of current standards. The commenter adds that it would greatly reduce the number of high-quality studies that could support environmental policy and regulation because these studies rely on confidential medical data and cannot be made publicly available or even posted on a tiered-access program.
Commenter (12701) expresses concern that if this rule goes into effect, the EPA might disregard premier peer-reviewed studies when revising air quality or drinking water regulations because confidential health information from study participants cannot be verified. Ultimately, the commenter is concerned that if what they believe is an this overly broad rule goes into effect, future EPA information and regulations intended to protect public and environmental health will not be informed by the best science.
Commenter (11402) states that the proposal would undermine EPA's ability to use the best available science in its policy-making process and thus impede EPA's mission to protect human health and the environment. The commenter states that the proposed supplemental rule would significantly undermine the ability of the EPA to use the best available science in setting policies and regulations if implemented. The commenter adds that it would result in a significant reduction in science that could be used by EPA, risking direct harm to our citizens and their environment. Therefore, the commenter contends that the effect of the rule would be to weaken the scientific underpinnings of federal policy and that standards created and enforced by the EPA should be based on ALL of the best available published science, data, and models. The commenter states that peer-reviewed literature is considered to be the gold standard for high quality research and that the use of such peer-reviewed scientific studies and conclusions in policy deliberations should not be restricted to studies with publicly available underlying data. 
Commenter (12715) states that the supplemental proposal would eliminate or de-emphasize consideration of relevant, peer-reviewed scientific studies where the underlying data cannot all be made public or cannot be made available via the proposed complicated tiered access system. In addition, the commenter notes that the proposal gives complete discretion to the EPA Administrator to decide what studies with non-public data are subject to the rule. The commenter contends that this would likely eliminate from consideration any research studies that, for valid reasons, do not have underlying data that can be made publicly available, including public health studies that form the basis of many critical nationwide protections.
Commenter (12715) states that applying the public availability requirement to all studies, regardless of when they were generated would severely restrict the body of scientific literature that could be considered as the basis for a "pivotal" science decision. According to this requirement, the commenter expresses concern that studies based on data that is not available or obtainable -- which is the case for most older studies, many of which are crucial to the scientific issues considered by EPA -- would no longer be used by EPA in its decision-making processes unless the Administrator used his or her discretion to grant an exemption under section 30.9. For example, the commenter notes that the EPA is required to review its NAAQS for criteria pollutants every five years and, if necessary, revise them to protect public health and the environment. See 42 U.S.C. § 7409(d). The commenter provides that the NAAQS review process builds on the administrative record from prior rulemakings, including historic studies that are part of that record. Under the supplemental proposal, the commenter states that the EPA could refuse to consider these studies and others because they rely on data pertaining to the personal medical histories of participants that cannot, by the studies' terms or by law, be divulged. The commenter contends that restricting the use of such studies would significantly undermine current and future NAAQS reviews and other rulemakings and would flout EPA's duty to use the best available science when protecting public health and the environment.
Additionally, commenter (12715) notes that given that the requirements set forth in the supplemental proposal might also be applied to historical studies considered by the Agency in any actions it takes going forward, many groundbreaking studies conducted in accordance with accepted scientific research principles could be either excluded from consideration or accorded less weight under EPA's tiered hierarchy for consideration. The commenter states that environmental research also may rely on CBI or data collected on private land that owners may not want made public. In short, the commenter asserts that the proposed rule will limit the scientific research that can be used to protect public health and the environment, which in turn will have a material adverse effect on State, local, and tribal communities
Commenter (12715) states that the supplemental proposal's requirement that underlying data be available in a manner sufficient for independent validation is technically unworkable. The commenter notes that the potential problems with this approach include that the actual feasibility of making data and models available in a manner sufficient for independent validation is unclear, as studies may be formatted in a manner that makes the data difficult to share, the lab that has generated the data may not be willing or able to collate and release the data, data from studies generated in other countries may be difficult to obtain, and some important conclusions are drawn from meta-analysis of numerous other studies. The commenter concludes that these issues would preclude potentially relevant studies from being considered or would give them less weight without a valid scientific reason for such a restriction.
Commenter (12415) states that helpful, representative tribal data will be more difficult to generate and make available to inform regulatory processes if the rule is adopted. The commenter states that the rule would dismantle decades of precedent and sound practice involving acceptable science, studies, and their use in assisting with regulatory development. The commenter states that thousands of rigorous, peer-reviewed studies and analyses rely on private, confidential patient data -- medical studies, personal health data, and clinical reports -- which by law cannot be made public. 
Commenter (11921) states that their experience makes clear that the EPA should use the broadest possible range of science for making decisions on risk, causality, and other important policy. The commenter notes that, while reproduction of the results of single studies is sometimes helpful in such systematic reviews, limiting evidence synthesis and assessment to only studies that have accessible data would seriously circumscribe the Agency's decision making and the robustness of such decisions. 
Commenter (11390) states that even as the EPA expands the proposed rule to capture nearly all science the Agency considers --  while still exempting without explanation Agency adjudications and enforcement activities  -- it presents among these revisions an "alternative" approach that would, if adopted, allow EPA to consider studies based on non-publicly available data under certain circumstances. In reality, the commenter contends that the revised proposed rule further hamstrings EPA's ability to use the best science when it regulates. The commenter asserts that the proposed rule is not only unnecessary and antithetical to the goal of achieving regulatory transparency but would also severely constrain EPA's ability to use the best quality science to make critical decisions. According to the commenter, eliminating the consideration of relevant science from the decision-making process based solely on the availability of underlying data would compromise the EPA's ability to effectively carry out its mission to protect public health and the environment. 
Commenter (11192) notes that while the supplemental proposal resolves some of the ambiguities of the original proposal, it exploits the concept of transparency, and provides an avenue for unwarranted dismissal of high-quality science. As a result, the commenter contends that the supplemental proposal further clarifies that this proposed rule would undermine science-based decision-making at the EPA. The commenter adds that, from ethical, legal, financial, and logistical considerations, it is clear that compliance with this proposed rule is untenable for existing public health studies, and the rule could cause the EPA to disregard studies crucial for the protection of human health.
Commenter (11192) asserts that it is an ill-conceived move by the EPA to put current researchers in the position of either complying with privacy laws or allowing their research to have real-world impacts by informing regulations. The commenter states that the effect of this is readily apparent: it will dramatically shift the balance of the kinds of studies that will be utilized in Agency decision-making, reducing the proportion of public health studies and increasing the proportion of assay-based and computational modeling studies. According to the commenter, this is likely to make the link between regulations and health protections less clear, not more. 
Commenter (13788) provides that the expansion of the rule to apply to influential scientific information could either arbitrarily erode the usefulness of its peer-reviewed influential scientific information and limit the Agency's ability to rely on influential scientific information in situation where it is required to use the best available science, or, if the Agency chooses to rely on the influential scientific information despite the exclusion of the best available science, potentially allow the Agency to circumvent statutory requirements. According to the commenter, neither of these outcomes is justified.
Commenter (11491) states that both options presented in the SNPRM, which allow for prohibiting, discouraging, or differentially weighing agency use of non-public data, are inconsistent with 2019 guidance from the OMB. The commenter states that an approach most consistent with the OMB guidance would be to continue to use the best available science regardless of underlying PII or CBI, and then explore the extent to which that science (possibly through tiering) could be made more accessible. The commenter states, in contrast, the EPA is proposing to give lesser or no consideration to science where data are confidential. The commenter states that, rather than maximizing access to data supporting influential science, the EPA's proposed approach would limit the science on which it can rely to make decisions, capriciously favoring not the most relevant or carefully designed studies, but those with public data.
Response: As more fully discussed in the preamble to the final rule, the EPA is concentrating its efforts in the final rule to increase transparency on dose-response data, as the dose-response data are discrete and the dose-response assessment is a well-defined and impactful step in the quantitative assessment of risk. Specifically, the scope of dose-response data in this final rule is those studies consisting of the data integral to characterizing dose-response relationships. In some instances, this group will consist of a handful of studies. In other instances, where there are multiple toxicity endpoints, there may be more studies that are crucial to characterizing dose-response relationships. In some other cases, there may be a large number of studies that are used to characterize a dose-response relationship (e.g., where the dose-response is based on a meta-regression of epidemiology studies). However, not all of these studies would be considered pivotal science. This final rule provides an important step in furthering the progress toward maximizing transparency and will provide insight for future statute-specific requirements. 
In addition, the EPA clarified in the final rule how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating significant regulatory actions or developing influential scientific information. As described in the preamble to the final rule, the EPA will continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies --  based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements or quantitative analyses of an EPA significant regulatory action or influential scientific information will be identified as pivotal science. When evaluating those scientific studies considered to be pivotal science -- in either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. Because the final rule does not categorically exclude any studies, including public health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations. 
In addition, the EPA is maintaining that the rule provisions apply equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. However, in consideration of instances where data cannot be made available, the EPA amended the rule to provide technical considerations (40 CFR 30.5(d)) that can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability in the absence of maximum transparency and included an exemption criterion -- that the development of the dose-response data was completed or updated before the effective date of the rule -- to 40 CFR 30.7 to account for the concerns raised by the SAB and other public commenters.
Moreover, although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges, and agrees with commenters, that there may be instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7. Again, because the final rule does not categorically exclude any studies, including public health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations. Because of these considerations and exemption criteria, the EPA disagrees that implementing this rule will reduce the proportion of public health studies or increase the proportion of assay-based and computational modeling studies. The EPA maintains that this rule is designed to build upon OMB guidance.
With this final rule, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
The EPA disagrees with the commenter that the rule is unworkable. Specifically, the commenter specifically stated that there was a feasibility issue with making data and models available. As discussed in the preamble to the final rule, the scientific community has already embraced greater data transparency, as evidenced by data sharing and availability requirements for many high-impact journals and the emergence of organizations, such as the Center for Open Science, and international initiatives like FAIR data principles; FOSTER; and Guidelines for TOP in Journal Policies and Practices that incentivize greater transparency in research. Data sharing mechanisms, like those provided for in the rule, already exist and provide a way to allow a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. Further, in consideration of instances where data cannot be made available, the EPA amended the rule to provide technical considerations (40 CFR 30.5(d)) that can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability in the absence of maximum transparency and included an exemption criterion -- that making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, CBI, or national security -- to 40 CFR 30.7 to make the rule more flexible. Again, because the final rule does not categorically exclude any studies, including public health studies, from EPA's consideration, the Agency will not be precluded from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
The EPA believes that the approach described in the final rule will result in significant regulatory actions and influential scientific information based on the highest quality, most relevant studies and ensure adequate human health and environmental protections while maximizing transparency and maintaining consistency with the statutes the EPA administers. The rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making.
Comment: Commenter (9372) states that it would be foolish not to consider the best research in the field on pollution, nearly all of which uses restricted data on individuals. For example, the commenter notes that the growing literatures in economics, public health and epidemiology have documented the contemporaneous health (Anderson and Thundilyil, 2011; Ransom and Pope, 1995; Pope and Dockery, 1999; Friedman et al., 2001; Moretti and Neidell, 2011; Schlenker and Walker, 2015) and mortality-related impacts among infants and the elderly (Anderson, 2016; Deryugina et al, 2016) of pollution, as well as linked it to bronchitis (Beatty and Shimshack, 2011), diabetes (Bowe et al, 2018) and missing school (Currie et al, 2008). The commenter adds that a growing literature also documents how in utero exposure to pollution affects health outcomes at birth such as infant mortality (Currie and Neidell, 2005; Chay and Greenstone, 2003), birth weight (Currie, Davis, Greenstone, and Walker, 2015) and the development of congenital anomalies (Currie, Greenstone, and Moretti, 2011). See also Jayachandran (2009), Currie and Walker (2011), Knittel, Miller, and Sanders (2015), ArceoGomez, Hanna, and Oliva (2015), and Almond and Currie (2011). Additionally, the commenter provides that some recent research also investigates how pollution exposure during gestation affects later human capital outcomes (Almond, Edlund, and Palme, 2009; Bharadwaj, Gibson, Graff Zivin, and Neilson, 2017; Black, Bütikofer, Devereux, and Salvanes, 2013; Ferrie, Rolphe and Troeskin, Persico, Figlio and Roth, 2016; Rau et al., 2015; Sanders, 2012). For example, Persico and Venator (2020), who find that pollution from local industrial sites opening affect students' academic achievement and behavioral incidents in nearby schools, as well as school rankings showing that there are community-wide effects of environmental policies. The commenter also adds that a few studies also document how acute, short term exposure to air pollution on testing days affects test score performance and performance in high-skilled occupations (Archsmith, Heyes, and Saberian, 2017). For example, Marcotte (2017) uses the variation in air quality on different testing days and finds that children who take tests on days on worse days for pollen and fine airborne particulate matter have worse outcomes. Similarly, the commenter provides that Roth (2016) finds that pollution on testing days affects college students' performance in the United Kingdom, and Ebenstein and Roth (2014) find that pollution affects performance on high school exit exams in Israel. Hernstadt and Muehlegger (2015) find that crime is higher downwind of highways than on the opposite upwind side, implying that pollution might have cognitive effects. As Almond, Currie and Duque (2016) point out in their recent literature review, even mild health shocks in early life can lead to substantial long-term negative outcomes. 
Response: The EPA agrees that the Agency should consider all relevant scientific information when developing significant regulatory actions and influential scientific information and has clarified this point in the preamble of the final rule. As described in the preamble to the final rule, the EPA will continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies --  based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements or quantitative analyses of an EPA significant regulatory action or influential scientific information will be identified as pivotal science. The EPA intends to issue implementation guidelines and statute-specific rulemakings to further clarify and describe how the EPA will identify pivotal science in its assessments and rulemakings. The specific criteria for determining "pivotal science" may necessarily be specific to the authorizing statute, as well as the significant regulatory action or the influential scientific information. The EPA intends to explain in each significant regulatory action and for influential scientific information how the pivotal studies were identified. Once studies considered to be pivotal science are identified, the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations. 
The EPA believes that the approach described in the final rule will result in significant regulatory actions and influential scientific information based on the highest quality, most relevant studies and ensure adequate human health and environmental protections while maximizing transparency and maintaining consistency with the statutes the EPA administers.
Comment: Commenter (10740) cites that Professor Steinzor and Professor Wendy E. Wagner have written cogently on how science is normally considered during the regulatory process, when agencies must evaluate the scientific evidence that informs a significant policy decision about health or environmental hazards. The commenter provides that they outline four sequential steps that are normally taken and note that an elaborate version of such steps is mandated by the CAA for EPA to use when setting NAAQS.
The commenter (10740) asserts that the proposed rule, instead, advances a process that Steinzor and Wagner have called "deconstruction" of the evidence. The commenter provides that deconstruction is accomplished through a myriad of techniques, including undermining the validity of studies for nonscientific reasons, encouraging the reconsideration of raw data using unreliable models, and adding research engineered to confound a finding of harm.
Response: The EPA disagrees with the commenter and finds that evaluating the availability of underlying dose-response data for pivotal science is critical to the EPA's progress toward increased transparency and providing increased opportunity for scientific reanalysis and review by subject matter experts. The rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. Additionally, the EPA believes that the approach described in the final rule will result in significant regulatory actions and influential scientific information based on the highest quality, most relevant studies and ensure adequate human health and environmental protections while maximizing transparency and maintaining consistency with the statutes the EPA administers.
Comment: Commenter (12715) states that although EPA rightly recognizes that it may not be possible to make data and models publicly available for many older studies, focusing the exemption solely on models and data developed before the effective date of the rule ignores the fact that academic researchers and others whose research is relevant to EPA's work may not conduct and report their studies in a way that satisfies the rule's requirements, and may not have the resources to change their protocols simply to comply with the rule's requirements. Moreover, the commenter notes that academic researchers who are focused on publishing their studies in peer-reviewed journals and on obtaining grants and other funding sources for their research have little reason or incentive to even consider whether their studies would qualify for EPA's use under this rule. The commenter adds that members of the academic community may have conducted and published a particular study with grant money that is long gone, may have moved on to other projects or endeavors, and may have utilized research participants who can no longer be found. By contrast, the commenter states that industry-generated studies are almost always funded and conducted for the purpose of being submitted for consideration by EPA and the industry sector has a strong incentive, as well as sufficient resources, to provide whatever is required to address any modified or additional "data transparency" requirements imposed by EPA in this new rule. It is therefore unreasonable, according to the commenter, for the EPA to expect that data generated and models used in academic research will be available in ways that comply with the proposed rule simply because they were developed after its effective date.
Response: The EPA is maintaining that the rule provisions apply equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that are unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator, as detailed under 40 CFR 30.7. One of the criteria the Administrator may use in his/her determination to grant a waiver is that the development of the dose-response data was completed or updated before the effective date of the rule. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
These considerations, the Administrator's exemption, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data are intended to minimize potential unintended consequences, such as that noted by the commenter. In addition, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. The EPA believes the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making.
Comment: Commenter (12718) states that the EPA elected to broaden the applicability of the proposed regulation through the supplemental proposal. The commenter notes that one effect could be that restrictions may be placed even on industry-produced scientific evidence required under the country's chemical and pesticides laws, which has long been viewed as credible and necessary by EPA officials during both Republican and Democratic administrations. For example, how the SNPRM requirements align with the FIFRA and the TSCA expectations remains unspecified in the SNPRM.
Response: With this final rule, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
Comment: Commenter (11495) states that, in at least two instances the D.C. Circuit Court of Appeals has recognized that studies for which the underlying data are not publicly available may constitute the "best available science." In American Trucking Associations, Inc. v. EPA, the commenter notes that the court held that the CAA did not require EPA to make underlying data public where EPA relied on the study itself and not the raw underlying data. The commenter provides that the court agreed with the EPA's argument that requiring the Agency to obtain and publicize the relevant data "would be impractical and unnecessary." In Coalition of Battery Recyclers Association v. EPA, the court again upheld EPA's reliance on non-publicly available data, reasoning that EPA relies on studies and their results rather than the raw data underlying those results. 
Response: As described in the preamble to the final rule, the EPA will continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies --  based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements or quantitative analyses of an EPA significant regulatory action or influential scientific information will be identified as pivotal science. When evaluating those scientific studies considered to be pivotal science -- in either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations. 
Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges, and agrees with commenters, that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7.
The EPA would like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science
Comment: Commenter (12720) expresses that the EPA commonly relies on studies conducted outside of the United States, including environmental epidemiology studies of air pollution and health in Canada. Such studies provide relevant information to EPA but are unlikely to meet stringent data requirements in the proposal. The proposal does not engage with this issue at all or identify any ways in which the significant reduction in research input from international community would affect EPA's science activities. The Proposal does not address these difficulties and should be withdrawn.
Commenter (12430) states that the supplemental proposal could potentially screen out "high-quality studies" developed by a mixture of academic, governmental, and private researchers over several decades and skew the scientific record in favor of newer industry-funded studies. The commenter contends that existing studies would be less likely to comply with the supplemental proposal, and unless the Administrator applies an ad hoc and discretionary exemption, would be discounted relative to newer studies created after the adoption of the supplemental proposal. The commenter provides that academic and government researchers generally prefer, and only receive funding, to focus on expanding scientific knowledge and do not have the resources to revisit established scientific conclusions. The commenter points out, however, that industry-funded groups, may have an incentive to invest in additional research to disprove those scientific conclusions. The commenter contends that the potential effect of the supplemental proposal's new and arbitrary criteria for assessing the relative value of scientific studies and information, would unscientifically skew the weight of the evidence toward industry research and interests.
Response: As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. 
Although the EPA is prioritizing transparency in pivotal science, the EPA also acknowledges and agrees with commenters, that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
The considerations in 40 CFR 30.5, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data are intended to minimize potential unintended consequences, such as the potential skew toward newer, domestic, and/or industry-funded studies suggested by the commenters. In addition, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. The EPA believes the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making.
Comment: Commenter (11396) states that the Agency has not articulated whether and how it will keep track of studies that are excluded from regulatory decision-making. In the interest of scientific transparency, the commenter suggests that the Agency at least develop a protocol for documenting and disclosing the reasons specific studies are excluded.
Response: As described in the preamble to the final rule, the EPA intends for the final rule to be implemented in a transparent manner. The EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. Additionally, the final rule requires the EPA to document the rationale for exemptions granted by the Administrator in the significant regulatory action or influential scientific information. 
Comment: Commenter (11176) states that the category of "non-federally funded and non-EPA data" likely makes up the largest category informing EPA's many actions. The commenter states that it is also the category that poses the most challenges for making data available. The commenter states that such studies generally provide important scientific contributions and should not be discounted because their data are not readily available. The commenter states that, while such work will not necessarily be excluded through the proposed rule with the SNPRM, it is likely to be de-emphasized to the detriment of evidence-based policymaking at EPA.
Response: As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges, and agrees with commenters, that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7.
The considerations in 40 CFR 30.5, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data are intended to minimize potential unintended consequences, such as the potential for non-federally funded and non-EPA studies to be de-emphasized as suggested by the commenter. In addition, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. The EPA believes the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making.
2.2.2.1.1 Specific Studies That Would Not Comply with Rule
Comment: The commenter (12465) provides the example of the Westat Survey. The commenter notes that the report entitled "Household Solvent Products: A National Usage Survey" (or the Westat Survey) is a survey of consumer usage for a wide variety of products to determine various attributes and inform exposure assessments. According to the commenter, while the report was completed in 1987, it remains in use today informing the EPA's Consumer Exposure Model and the Westat Survey informed several of the first 10 chemicals currently being evaluated under reformed TSCA. The commenter states that it is unclear in the proposal how the Westat Survey would be treated, let alone the multitude of exposure models that rely upon it.
Further, the commenter (12465) states that it is unclear that the supplemental proposal has adequately considered the numerous types of data and models EPA relies upon to support its regulatory decisions. The commenter provides that the 1987 report "Household Solvent Products: A National Usage Survey" remains in use today informing the EPA's Consumer Exposure Model and several of the first 10 chemicals currently being evaluated under reformed TSCA. According to the commenter, the supplemental proposal is unclear how this information would be treated, let alone the exposure models that rely upon it. Additionally, the commenter notes that this study is likely worthy of being updated, however it is not clear that respondents would be afforded sufficient privacy protections to elicit responses.
Response: As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges, and agrees with commenters, that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7.
With this final rule, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
Comment: Commenter (11894) states that the EPA's NAAQS established under the CAA were the primary target of EPA's original proposal. The commenter adds that the EPA's new proposal would further increase that impact in at least two ways: First, as noted it would extend the bar on the use of undisclosed data from dose-response models to all models. That would affect the many other models - for example, economic models, energy use models, and atmospheric models - on which NAAQS analysis relies. Second, many of the documents prepared in the course of setting a NAAQS are themselves "influential scientific information" or rely on other studies that would or could qualify as "influential scientific information."
Commenter (11894) suggests that the application of the bar on the use of undisclosed studies to these documents would further increase the cost and complexity of establishing NAAQS and diminish the quality of the decision-making record. The commenter provides that this would be particularly clear in the case of the Regulatory Impact Analysis that is prepared for every NAAQS. By law, the commenter notes that these documents play no part in EPA's decision, so the original proposal would not have covered them. However, the commenter states that they would unquestionably qualify as "influential scientific information." According to the commenter, the EPA's proposal does not analyze any of these issues. The commenter adds that it is ironic then, that developments since EPA's original proposal have further underlined the complete lack of justification for anything like EPA's proposal where NAAQS establishment is concerned.
Response: The rulemaking is intended to codify internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions.
Comment: Commenter (11921) provides that EPA should recognize that expectations for data access for studies have evolved over time and in many cases older studies that did not and are unable to comply after a prolonged interval still have substantial scientific merit and value for informing regulations. According to the commenter, some studies, e.g. the National Institute for Occupational Safety and Health/National Cancer Institute Diesel Exposed Miners Study (DEMS), have made their data available through federal RDCs, however other critical studies, e.g. the North American Rubber Workers Study (Delzell, 2006), which has played and continues to play a key role in assessment of risk from 1,3-butadiene and styrene, was designed and implemented at a much earlier time and would not be readily available. The commenter states that this supplemental proposal would require a significant delay in the use of a study that has been vetted and peer-reviewed by a wide range of scientists (including  -  in both these cases - the most recent update subjected to HEI project oversight and intensive independent peer review).
Response: Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges, and agrees with commenters, that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7.
Comment: Commenter (12715) states that while the initial proposal applied only to studies that provide the dose-response data for human health risk assessment, the supplemental proposal expands the types of studies to which the proposed rule would apply and now includes "data and models underlying pivotal regulatory science and pivotal science used to support significant regulatory decisions and influential scientific information." 85 Fed. Reg. at 15,400-01. The commenter provides that the expanded scope would thus include the data from many other types of studies such as environmental fate and transport, bioaccumulation, and ecological toxicity studies and will severely curtail EPA's ability to consider studies that provide the most relevant data in many types of regulatory and non-regulatory scientific decision-making. For example, the commenter notes that while risk assessment evaluations performed under the EPA's IRIS are not regulatory in nature, the toxicity factors provided by IRIS are used as the basis for many regulatory standards developed by the EPA and numerous states, as well as federal and state guidance values and cleanup decisions. In fact, the commenter notes that some states' regulations require that IRIS toxicity factors be used as the basis for human health standards for environmental contaminants. The commenter states that the supplemental proposal would significantly limit IRIS's consideration of and reliance on many key studies that are crucial for the assessment of human health effects and development of toxicity factors for environmental contaminants.
Commenter (12456) states that the systematic review process in place for IRIS and other review processes already includes consideration of the quality of a research study and methods. The commenter asserts that the best professional judgement of data quality should be left to subject matter experts and not assigned solely on the basis of data availability.
Commenter (11349) provides that the EPA and National Institute for Environmental Health Sciences' (NIEHS's) Children's Environmental Health and Disease Prevention Research Centers may certainly be excluded by this supplemental proposal and that the expansion increases the studies that can be excluded even further.
Response: Based on a consideration of these and other similar comments, the preamble to the final rule describes the EPA's rationale for modifying the applicability of the rule to focus on dose-response data and clarifies how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating significant regulatory actions or developing influential scientific information. Briefly, the EPA is concentrating its efforts in the final rule to increase transparency on dose-response data, as the dose-response data are discrete and the dose-response assessment is a well-defined and impactful step in the quantitative assessment of risk. This final rule provides an important step in furthering the progress toward maximizing transparency and will provide insight for future statute-specific requirements. In addition, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges, and agrees with commenters, that there may instances where the underlying dose-response data are not publicly available or not available in a manner sufficient for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7.
Including this review of dose-response data availability for pivotal science is critical to the EPA's progress toward increased transparency and providing increased opportunity for scientific reanalysis and review by independent third parties.
Comment: Commenter (11482) states that the inherently interdisciplinary and connected nature of the ocean means that more information, from more sources, is critical to gaining an understanding of ocean issues and being prepared to act from a regulatory perspective. The commenter notes that the proposed rule, which would limit rather than increase actionable knowledge, is likely to harm ocean and coastal ecosystems. Additionally, the commenter states that given the vast amounts of ocean data held by the U.S. Navy and shared with ocean researchers  --  but with important redactions and aggregation of data for reasons of national security  --  this rule could limit the information available to the EPA from such sources, despite that information being trustworthy and the findings vetted through extensive peer review. Similarly, the commenter provides that the EPA would be barred from relying on information from the National Oceanic and Atmospheric Administration that may not withstand these overly burdensome standards. For example, the commenter states that many environmental studies, including those intentionally conducted in the ocean and those that are the result of accidents like the Deepwater Horizon, simply can't (or shouldn't) be replicated in any meaningful way -- yet they provide key insights that should be taken into consideration by federal regulators. The commenter suggests that the rule may also hamper the use of climate models.
In short, the commenter (11482) contends that public health studies related to the ocean would suffer under this rule; this is particularly worrisome at a time when warming waters and deoxygenation are combining to exacerbate the problem of harmful algal blooms, with potentially negative, far reaching health consequences.
Response: In the final rule the terms replicate and reproduce (or other forms of the word) are no longer used. The EPA is clarifying that the intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze." In addition, the EPA is modifying the definition of "independent validation" in the final rule by replacing "capable of being substantially reproduced" with "produced." The EPA will not finalize the proposed 40 CFR 30.2 definition of "capable of being substantially reproduced" because the term is not used in the final rule's definition of "independent validation" or elsewhere in 40 CFR 30.
Based on a consideration of these and other similar comments, the EPA clarified in the final rule how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating significant regulatory actions or developing influential scientific information. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that are unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator, as detailed under 40 CFR 30.7. One of the criteria the Administrator may use in his/her determination to grant a waiver is that making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, CBI, or national security (emphasis added). Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
The considerations in 40 CFR 30.5, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data are intended to minimize potential unintended consequences, such as the potential unintended impact on ocean studies suggested by the commenter. In addition, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. The EPA believes the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making.
Comment: Commenter (11392) notes that past products designated influential scientific information, such as chemical draft risk evaluations, ISAs, IRIS toxicological reviews, and a scientific assessment of the impact of climate change on human health, would be undermined by imposing arbitrary limitations on the scientific studies that can be considered in the drafting of these products.
Response: The EPA disagrees that scientific assessments would be undermined by this rule. The approach outlined in this rule will result in significant regulatory actions and influential scientific information that are based on the highest quality, most relevant studies and ensure adequate human health and environmental protections, while maximizing transparency and maintaining consistency with the statutes the EPA administers. Based on a consideration of these and other similar comments, the EPA clarified in the final rule how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating significant regulatory actions or developing influential scientific information. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges, and agrees with commenters, that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7.
EPA would also like to clarify that the final rule applies prospectively to significant regulatory actions and influential scientific information and has no retrospective effect on existing significant regulatory actions or influential scientific information. Past products will not be impacted.
Including this review of dose-response data availability for pivotal science is critical to the EPA's progress toward increased transparency and providing increased opportunity for scientific reanalysis and review by independent third parties.
Comment: Commenter (11576) states that the inclusion of "influential scientific information" will impede Agency scientists from relying on the best available science while also compromising scientific integrity. The commenter notes that EPA's Peer Review Agenda describes different types of highly influential scientific assessments developed by the Agency. and includes a list of influential assessments, including the ISA for Ozone and Related Photochemical Oxidants (Office of Research and Development), the Peer Review of EPA's Biologically Based Dose-Response Model for Perchlorate (Office of Water), and the EPA's Draft Risk Evaluation for 1-Bromopropane (Office of Chemical Safety and Pollution Prevention), all of which rely on studies that may be arbitrarily excluded if the proposed rule is finalized.
Response: The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods.
As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges, and agrees with commenters, that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7.
Including this review of dose-response data availability for pivotal science is critical to the EPA's progress toward increased transparency and providing increased opportunity for scientific reanalysis and review by independent third parties.
Comment: Commenter (11890) provides that the rule would impair EPA's science assessments by excluding or downgrading high quality studies of human health impacts of environmental chemicals or agents. The commenter asserts that this is at odds with EPA's mission and duty to use the best available science and that critical evaluation programs impacted by the inclusion of "influential science" include drinking water health advisories, IRIS assessments, FIFRA re-registrations, children's lead hazard evaluation, and TSCA risk evaluations.
Response: The EPA does not agree that this rulemaking is counter to the Agency's mission. The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. Based on a consideration of these and other similar comments, the EPA clarified in the final rule how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating significant regulatory actions or developing influential scientific information. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges, and agrees with commenters, that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7.
In addition, as clarified in the preamble to the final rule, this rulemaking is not intended to modify the Agency's interpretations of "best available science." The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. The EPA will then identify and consider pivotal science in accordance with the provisions of this rule, unless the implementation of the rule conflicts with statutory requirements and associated implementing regulations.
The EPA believes that the approach described in the final rule will result in significant regulatory actions and influential scientific information based on the highest quality, most relevant studies and ensure adequate human health and environmental protections while maximizing transparency and maintaining consistency with the statutes the EPA administers.
Comment: Commenter (11894) notes that EPA has issued about 170 separate documents to establish scientific guidance for water quality. According to the commenter, these do not have regulatory effect but provide background for regulatory decisions that set actual water quality standards. The commenter states that these criteria are, therefore, an essential part of the water quality standards program which is, in turn, an essential part of the CWA, and taken by themselves, they would clearly qualify as "influential scientific information." As such, the commenter contends they would often rely on studies and models that would not meet the new requirements. The commenter questions whether these would be "agency actions" exempt from the new requirements despite their lack of legal effect, and if not, what would be the impact of the new requirements on human and environmental health, program functioning, and resource needs? The commenter also considers EPA and the National Highway Traffic Safety Administration's Heavy Duty Rule's inclusion of hundreds to thousands of references, each with sub-references, all of which appear to be "influential scientific information". The commenter questions whether or not these references are or would be covered by the rule, why, what the effects on EPA's authority to control emissions of pollutants under CAA sections 202 (a)(1), (2) and (3) are or would be, and EPA's reasoning for these findings. Additionally, the commenter suggests EPA should analyze how references in various significant rules would be impacted by this proposal.
Response: As described in the preamble to the final rule, the requirements in the final rule set the overarching structure and principles for transparency in the EPA's significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes EPA administers.
In addition, the preamble discusses that the EPA is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. The EPA is focusing on the underlying dose-response data for this rulemaking because of the influence these data have on particularly impactful decisions at the Agency. Risk assessments and regulations that target emissions and risk reduction of one or more pollutants, contaminants, or substances are integral to the Agency's mission and the underlying dose-response data that inform the quantitative value used to evaluate and mitigate potential risk are critical to understanding the assessment or regulatory action. In addition, the data underlying the dose-response assessment are more distinct than the broad range of data informing an entire risk assessment. Therefore, the EPA is concentrating its current efforts to increase transparency on a well-defined step in the quantitative assessment of risk supporting specific Agency actions. This final rule provides an important step in furthering the progress already being made toward maximizing transparency and will provide important insight for developing future statute-specific requirements.
The EPA is also finalizing regulatory text in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict.
Comment: Commenter (12464) states that many assessments, reviews, and reports of EPA cite thousands of scientific papers. According to the commenter, it is unclear how many of those papers would be subject to the proposal. Additionally, the commenter notes that because the supplemental proposal would systematically exclude studies that do not meet EPA's criteria for reanalysis and independent validation, critical studies that would otherwise inform influential scientific information would not be incorporated into key documents, such as the Integrated Scientific Assessment for Particulate Matter. The commenter asserts that many of the studies cited in the assessment are epidemiological papers analyzing the numbers of patient hospital visits or other sensitive medical information. As such, the commenter provides that these studies cannot be made publicly available due to laws and contracts designed to protect patient and human subject privacy. Thus, the commenter contends that the supplemental proposal's expansion in scope would in many instances prohibit EPA from relying on the best available science when assessing and determining highly influential scientific information, depriving regulators, researchers, and academics of a valuable and objective source of information.
Response: Based on a consideration of these and other similar comments, the EPA clarified in the final rule how pivotal science would be selected. As discussed in the preamble to the final rule, rather than considering all studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact as "pivotal science," the EPA is balancing transparency and feasibility by focusing on those studies that inform the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect. Thus, "pivotal science" includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). These are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. In the context of this rule, studies informing the dose-response modeling (e.g., benchmark response selection), studies identifying data for toxicokinetic adjustments, or studies informing the selection of uncertainty factors do not drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. 
The final rule also clarifies how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating significant regulatory actions or developing influential scientific information. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges, and agrees with commenters, that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
Comment: Commenter (11894) notes that, as detailed in Appendix B of their comment letter, the supplemental proposal would either bar a majority of the key long-term cohort studies in the 2018 PM ISA from consideration, or perhaps merely downgrade them (the proposal is to too opaque to allow any certainty as to its intended result) -- all without reference to the studies' methodological quality, consistency with prior studies, or any other individual consideration.
Commenter (12435) provides that the current ozone standard relies on "controlled human exposure studies conducted over many decades" (Ozone Standards ISA). The commenter notes that these studies were conducted with the informed consent of human participants and most studies were published more than ten years ago. While the underlying raw data may not be available, the commenter states that these studies have gone through peer review and the relevant information is now retained in the published journal articles. The commenter adds that the consistency of findings across decades contributes to the weight of evidence used to set this standard to be protective of public health. The commenter expresses concern that, in Colorado, 3.9 million people live within the ozone nonattainment area. Without this standard, the commenter states that Colorado would be hindered in its ability to regulate emissions that contribute to ozone and the health of many would suffer without going through incredible expense and time of setting standards under State law that should be done properly by EPA under the CAA. The commenter provides that epidemiology studies find that ozone concentrations "are associated with a range of respiratory effects, including asthma exacerbation, chronic obstructive pulmonary disease (COPD) exacerbation, respiratory infection, and hospital admissions and emergency department (ED) visits for combined respiratory diseases" (Ozone Standards ISA, page ES-8). The commenter states that Colorado's health agencies rely on policy based on the types of studies that could be dismissed under this proposal. 
Commenter (12720) expresses that, under the Supplemental Proposal, the EPA would not be able to rely on the best available science for its ISAs of air pollution which inform the NAAQS-setting process, while industry-funded research calling into question the air pollution-health link, would not be subject to similar data release requirements, or even peer-review and independent reevaluation. 
Response: Based on a consideration of these and other similar comments, the EPA clarified in the final rule how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating significant regulatory actions or developing influential scientific information. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges, and agrees with commenters, that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
The requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making.
Comment: One example of the historic studies that would have difficulty meeting the "open data" criteria, several commenters (11408, 11495, 11396, 12715) point out that the researchers who originally conducted the Harvard University's seminal, peer-reviewed "Six Cities Study," completed in 1993 (and reanalyzed in 2000) would have difficulty meeting the "open data" criteria the supplemental now proposes for inclusion in EPA's decision-making, even under the proposed tiered system. 
Commenter (11396) states that industries and their allies have been pushing to exclude studies for decades, using the same arguments found in EPA's proposal, targeting research that shows harm to public health from their arguments found in their emissions. The commenter highlights the history of how (outlined in their comment letter and by several comments on the 2018 proposal), in 1996, attorneys working for tobacco industry giant R.J. Reynolds recommended a similar approach requiring review of documents "because, at some point in the future, EPA will most likely be ordered to re-examine ETS [Environmental Tobacco Smoke]." The commenter adds that the tobacco attorneys recommended expanding this approach to other industries, which quickly happened. The commenter provides that two of the early industry targets were landmark air pollution studies completed in the 1990s that found solid evidence that particulate matter air pollution could cause premature death. The two long-term studies - the 1993 Harvard Six Cities Study and the 1995 ACS Study - looked at large population in multiple locations. The commenter notes that both studies found the particulate matter in the air was linked to increased risk of premature death. Instead of blocking the studies, as this proposal would do, the commenter notes that the EPA took a logical step and referred both studies to an independent third party, the HEI, for a deep-dive review. There, the commenter notes, autonomous reviewers examined the data and developed a report on the two studies that confirmed their original findings. Since these studies, the commenter notes that other research has confirmed their findings as well, including some studies that used publicly available datasets. The commenter suggests that similar third-party reviews could readily address concerns about existing or future studies as needed.
Response: Based on a consideration of these and other similar comments, the EPA clarified in the final rule how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating significant regulatory actions or developing influential scientific information. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available (either publicly or through restricted access) in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges, and agrees with commenters, that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7. One criterion for the Administrator's consideration of an exemption is that a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis. 
Although independent validation by a third-party can be informative, the presence of a third-party validation is provided as a case-by-case exemption criterion instead of an automatic rule consideration in order to more fully realize the benefits of open data. As discussed in the preamble to the final rule, when data are widely available, researchers can validate research results and help identify and correct unintended errors, as well as reanalyze the data for new and different purposes, examine novel questions, provide new scientific insights, and improve model development. The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. 
The EPA also agrees that data sharing mechanisms, such as that used by the Health Effects Institute in conducting their reanalysis study, already exist. The final rule gives equal consideration to studies with underlying dose-response data that are available publicly and studies with underlying data that are available through restricted access. The preamble to the rule clarifies that restricted access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. The rule does not prescribe a particular mechanism but does require the EPA to independently verify that such a mechanism exists for dose-response data underlying pivotal science. As discussed in the preamble to the final rule, including this review of dose-response data availability for pivotal science is critical to the EPA's progress toward increased transparency and providing increased opportunity for scientific reanalysis and review by independent third parties.
Comment: Commenter (11479) contends the SNPRM and draft rule would prevent critically important research from being used to inform the EPA's decision making on public health protections. The commenter states that, for example, the EPA funded four university-based Clean Air Research Centers between 2011 and 2016 to better understand complex air quality and health issues -- research that directly informed the Agency's air pollution standards, required under the CAA. The commenter states that the four centers have generated hundreds of research publications and helped answer crucial scientific questions that inform how the EPA can best protect public health, including that of at-risk subpopulations, from ambient air pollution, with an adequate margin of safety. The commenter states that the centers' contributions include understanding of multipollutant mixtures on health outcomes; stratification of health effects across geographies and populations; characterization of air pollution measurement error; and improved measurement and characterization of traffic emissions and their health outcomes. The commenter states that Dr. Gretchen T. Goldman, Research Director at the Union of Concerned Scientists' Center for Science and Democracy, contributed to this research with several studies that informed current and past EPA decisions on air pollution standards on ozone and particulate matter. 
Commenter (11403) states that, by deliberate design, the proposed rule would have the EPA ignore some of the most rigorous, long-term, peer-reviewed studies conducted by leading independent researchers. For example, the commenter provides that a recent study found that for children younger than age two even extremely short exposure to soot particles, like those created by burning fossil fuels such as coal, oil, and natural gas, increases the kinds of respiratory infections that are the leading cause of sickness and death. This is the kind of study EPA would ignore or deprioritize because HIPAA correctly makes it illegal for researchers to release the identities and health histories of the nearly 150,000 people in the study, making the data incapable of being disclosed.
Response: Based on a consideration of these and other similar comments, the EPA clarified in the final rule how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating significant regulatory actions or developing influential scientific information. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges, and agrees with commenters, that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations
Comment: Commenter (12442) states that together, the expansion of the rule and the tiered-access approach brings into question whether or not a study such as the 2013 report, "National Park Service Visitor Values & Perceptions of Clean Air, Scenic Views & Dark Night Skies 1988-2011" (Exhibit A in their comment letter), which followed standard scientific norms for gathering information from groups of people visiting parks, could be considered by EPA to inform internal scientific assessments or rulemakings in the future. The commenter notes that if this kind of study were to be devalued or disregarded for the protection of clean air in national parks because individuals weren't named, it would be a great loss of information that shows the clear co-benefits of the economic value that people place on clean air -- for many different reasons -- when visiting parks. 
Response: Based on a consideration of these and other similar comments, the EPA clarified in the final rule how pivotal science would be selected. As discussed in the preamble to the final rule, rather than considering all studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact as "pivotal science," the EPA is balancing transparency and feasibility by focusing on those studies that inform the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect. Thus, "pivotal science" includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). These are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. In the context of this rule, studies informing the dose-response modeling (e.g., benchmark response selection), studies identifying data for toxicokinetic adjustments, or studies informing the selection of uncertainty factors do not drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. 
The final rule also clarifies how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating significant regulatory actions or developing influential scientific information. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges, and agrees with commenters, that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
Comment: Commenter (12716) states that many of the ongoing programs and policies in place that will restore the Chesapeake Bay are based upon regulations promulgated by EPA and supported by the Agency's consideration of the best available science. In 2010, the commenter notes that the EPA issued a total maximum daily load (TMDL) for the Chesapeake Bay region. It "sets Bay watershed limits of 185.9 million pounds of nitrogen, 12.5 million pounds of phosphorous and 6.45 billion pounds of sediment per year -- a 25 percent reduction in nitrogen, 24 percent reduction in phosphorous and 20 percent reduction in sediment." In doing so, it set a cap on Nitrogen Oxides (NOx) from air deposition to 15.7 million pounds and "committed to reducing nitrogen deposition to the Bay and its surrounding waters by a total of 3.7 million pounds between 2009 and 2025, the year all practices are to be in place to meet the Bay TMDL goals." The commenter states that the Bay TMDL based these limits on years of peer-reviewed science and relied on the implementation of various federal regulations to achieve these reductions. 
In addition to regulatory programs upon which the success of the Chesapeake Clean Water Blueprint relies, the commenter (12716) states that the Agency's proposed changes could also impact other CAA and CWA programs needed to meet the goals of the Chesapeake Bay Watershed Agreement and reduce the effects of climate change. The commenter provides that the restoration of the Chesapeake Bay is dependent on the implementation of numerous federal CAA and CWA programs that are based upon the best available science. The commenter contends that a narrowing of the use of science -- under the guise of transparency -- in the promulgation of rules threatens the implementation of those programs, and therefore threatens the health of the Chesapeake Bay.
Response: The EPA is finalizing the applicability of this rule to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods. The EPA is not applying this rulemaking to permit proceedings, site-specific actions, or enforcement actions because these actions are typically focused on individual regulated entities.
As described in the preamble to the final rule, the requirements in the final rule set the overarching structure and principles for transparency in the EPA's significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
Comment: Commenter (12466) states that while they support greater transparency in EPA actions around the use of antibiotics in animal agriculture, they do not believe that the proposed rule actually provides the needed transparency and add that it may make it more difficult for EPA to properly take into consideration some of the limited information that is available related to use of antibiotics in vegetable and fruit production. The commenter explains that, in the realm of crop use of antibiotics, there is very limited data available and the data that is available covers mainly apples and pears not other crops such as citrus fruits. Because approvals for citrus use in the U.S. are fairly recent, the commenter states that studies on antibiotic use in citrus are primarily available from researchers in other countries, which increases the risk that the underlying data will not be available. The commenter states that the raw data from studies on impacts of worker exposure to antibiotic pesticides might not be available both due to the confidentiality of medical records and due to the need to protect workers from retaliation by crop production companies. Despite these limitations, the commenter asserts that this type of data is often the best available. Given the low number of studies related to crop use of antibiotics and the real risk that underlying data will not be available for studies published in the peer reviewed literature for the reasons described above, it is inappropriate for EPA to move forward with the proposed rule.
Response: Based on a consideration of these and other similar comments, the EPA clarified in the final rule how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating significant regulatory actions or developing influential scientific information. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7.
Comment: Commenter (11386) states that, in the case of air quality, privately-owned satellites play an increasingly important role in the observation of concentrations of atmospheric constituents. The commenter states that satellites take years to develop and launch, and private companies currently dominate the field of high-resolution atmospheric observations from satellites. The commenter states that MethaneSAT, a subsidiary of the Environmental Defense Fund, may provide publicly available data at coarser resolution than those provided by satellites like GHGSat-D, complicating the estimation of methane emission rates for point emission sources such as oil and natural gas facilities. The commenter states that there may be no plans to launch a satellite that would provide publicly available methane observations at similar resolution to those produced by GHGSat's instruments. The commenter states that excluding these observations from all regulatory science, as per the expanded scope of the proposed rule, would harm the improvement of emission inventories and the resulting policy.
Response: Based on a consideration of these and other similar comments, the EPA clarified in the final rule how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating significant regulatory actions or developing influential scientific information. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7.
2.2.2.1.2 Epidemiological Studies
Comment: Commenters (10040, 11182, 11359, 11360, 11400, 11403, 11404, 11492, 11497, 11576, 11890, 11911, 12405, 12422 12430, 12431, 12435, 12720, 12715) express concern that, under this rule, many epidemiological studies that are pertinent to the EPA's work could not be considered during the rulemaking process due to data confidentiality obligations. 
Commenter (11359) contends that crucial epidemiological studies would be excluded from consideration under the transparency requirements. The commenter highlights the value of using epidemiological studies for chemical risk assessment as they provide human data to accurately evaluate health impacts. To that end, the commenter states that epidemiological data should be considered essential for all chemical risk assessments, with key examples of important human studies conducted on the pesticides paraquat, atrazine, glyphosate, chlorpyrifos, and other chemical substances used in industrial and agricultural applications, such as PFAS. Additionally, the commenter provides a cancer risk study that would likely be excluded from consideration to further demonstrate the value of using human health data to better protect human health.
Commenter (11360) states that data privacy laws largely prevent information from studies involving humans from being made public in the manner being considered by the EPA. As a result, the commenter asserts, if the EPA proceeds with downgrading or dismissing such science from consideration, the Agency will be excluding from their policy and rulemaking process epidemiology studies linking air pollution to serious outcomes such as death, heart attacks, and strokes; disinfection byproducts to birth outcomes; and, toxic exposure to cognitive function.
Commenter (12405) provides that human-based clinical trials or epidemiology (observational) studies form the bedrock of knowledge for determinations about the environment's impact on human health. The commenter states that exclusion of these studies stands to affect every decision made at EPA from NAAQS to chemical registration and regulation in consumer products and pesticides. In effect, the commenter asserts the rule would restrict access to science rather than make it more transparent.
Commenters (10040, 11360) state that this proposal would result in the exclusion of epidemiology studies linking air pollution to serious outcomes such as death, heart attacks, and strokes; disinfection byproducts to birth outcomes; and toxic exposure to cognitive function in the EPA regulatory rulemaking. Commenter (11360) states that the fact that this information must be kept confidential to protect research participants does not make the data any less valid. 
Commenter (11182) disagrees with the proposal to include studies containing PII in this rule. The commenter states that epidemiologic studies are the foundation of many, if not most, environmental and occupational regulations to protect workers and the public. The commenter notes the studies of asbestos-exposed insulators and lung cancer (Selikoff et al., 1979), benzene-exposed rubber workers and leukemia (Rinsky et al., 1987), and in children, low level lead exposure and brain damage (Lanphear et al., 2002) and states that these epidemiologic studies were pivotal research in the subsequent regulations that have saved many workers' lives and improved the health of millions of children. The commenter states that these studies would likely not have been considered under this supplemental proposal; the simple but profound reason is the principle of confidentiality.
Commenter (11400) contends that, under the proposed rule, large epidemiological studies like the effect of air pollution and correlation with public health in regard to respiratory, cardiac, and neurological impacts would be discounted, to the detriment of people living close to polluting sources. The commenter states that the problem is that this will exclude studies where the patient information cannot be made public, even where the study is of higher quality than studies with no PII. The commenter states that environmental and epidemiological studies often consider where people are as they are exposed to pollutants: their residence and their subsequent health issues are germane to the studies. The commenter states that residence, cause of death on death certificates, older age, are PII. The commenter states that excluding these studies because they contain PII of the patients' proximity to sources of pollution means that the most relevant studies will not be given appropriate weight. 
Commenter (11403) asserts that, without these epidemiological studies, it would be impossible to put in place necessary protections to adequately protect vulnerable populations. The commenter states that studies done on animals are simply not protective of these populations. The commenter states that lab animals lack genetic heterogeneity; they live in tightly controlled environments, are fed an optimal diet, and lack any pre-existing health conditions. The commenter states that this is not reflective of the human race. The commenter states that, unlike animal studies, epidemiological studies look at human subjects, deliver results that reflect real-world exposure conditions, and are able to tease out populations that may be more vulnerable than the general population. The commenter states that these studies often have personal data associated with them that cannot be made public. The commenter states that forcing mass storage of this PII in an EPA system so that it can be released through a tiered access program is a security risk and puts members of these communities even more in danger.
Commenter (11404) expresses that the rule might greatly reduce the number and quality of health studies available to support the development of air quality standards or regulations and hinder the health protection work the EPA is entrusted to do. The commenter notes that a reduction in availability of epidemiological studies would affect not only the EPA, but also other environmental and health protection agencies - such as the Air District - via the work of our sister agencies, the California Air Resources Board (CARB) and the Office of Environmental Health Hazard Assessment (which independently establish California's ambient air quality standards and health values for toxic compounds). The commenter contends that the proposal undermines, rather than enhances the scientific studies, data and models that are needed to develop meaningful, cost-effective air quality regulations. As drafted, the commenter states that the proposal would stifle efforts to make decisions based on the best scientific information and ultimately impact our ability to protect the health of Bay Area residents.
Commenter (11492) states that the supplemental proposal assigns greater value to dose-response studies, thereby reducing the use/value of epidemiological studies. The commenter notes that EPA has acknowledged that "there is frequently a lack of dose-response data available for human subjects. When data are available, they often cover only a portion of the possible range of the dose-response relationship." As epidemiological studies form the bases of many human health exposure studies, the commenter contends that EPA must continue to use epidemiological studies; to do so otherwise is to compromise the health of all Americans.
Commenter (11497) contends that while much of the data showing a connection between chemicals and breast cancer risk comes from laboratory studies, epidemiological studies, in particular longitudinal studies, provide critical insights and important corroboration for these findings. Additionally, the commenter states epidemiological studies allow researchers to look at numerous factors in the lived experience of women to understand and parse out some of these complexities. Once such study provided by the commenter is the National Institute of Environmental Health Science's Sister Study that has and will continue to provide data on breast cancer and disease prevention. The commenter notes that most of these critical epidemiological studies, are funded by the NIH, and that their strong confidentiality policy would force EPA scientists to disregard the studies, resulting in less scientifically sound conclusions and diminished public health benefits.
Commenter (11576) states that epidemiological and public-health studies often involve confidential data related to people, and researchers must protect the privacy of study participants, rendering disclosure of the underlying data impossible and in some cases unlawful. The commenter states that the rule could have the effect of removing valid, credible, peer-reviewed studies of health effects as sources to support the Agency's regulatory efforts: "The proposed rule does not acknowledge that the epidemiologic science community, for example, has been making significant efforts to make data available where possible and to develop studies based on publicly available data where appropriate." The commenter states that excluding studies where it is not possible to make the underlying data publicly accessible would unnecessarily and irrationally exclude reliable science for no legitimate reason.
Commenter (11890) states that, if enacted, the proposal would disable the EPA's ability to use U.S. and most other countries' epidemiological studies that relied on private medical information. 
Commenter (11911) states that epidemiological studies enjoy the support of the scientific community and provide the main justification -- in some cases the only possible justification -- for some of our most important environmental protections. The commenter states that these protections have significantly improved the wellbeing of the American people. The commenter states that excluding studies simply because the raw data cannot be made publicly available would result in inadequately informed regulatory decisions that could subject people to real harm. 
Commenter (11911) states that many epidemiological studies that are pertinent to the EPA's work could not be considered during the rulemaking process due to data confidentiality obligations. The commenter states that, for example, in a study published in the American Journal of Respiratory and Critical Care Medicine in June 2018, researchers discovered that fine particulate matter exposure was more strongly associated with respiratory emergency hospital visits for children than for adults, while ozone exposure was more strongly associated with respiratory emergency hospital visits for adults than for children. The commenter states that the researchers concluded that in light of this finding, relying on Medicare data and other studies that restrict their analysis to populations over age sixty-five "could underestimate population respiratory health impacts of PM2.5 or ozone." The commenter states that, because the study was based on confidential emergency room visit records aggregated by state agencies to protect the patients' privacy, the proposed STRS Rule would bar the EPA from considering it.
Commenter (11911) asserts that the proposed rule is likely to have particularly harmful effects for regulations that rely on studies of low probability but high impact events, such as natural disasters, environmental catastrophes, wars, or terrorist attacks. The commenter states that some of these studies have unavailable data due to their age and changing data transparency norms. The commenter states that the data cannot be newly collected again as the disastrous event cannot be experimentally replicated. The commenter states that studies of the health effects from nuclear plant failures in Chernobyl or Fukushima, or fallout from deployment of nuclear weapons in Hiroshima and Nagasaki, likely fall in this category. 
Commenter (12430) asserts the SNPRM would dramatically impede the EPA's ability to consider epidemiological studies because of the type of data collected. The commenter states that epidemiological studies rely on confidential information -- with geographical location, and birth and death dates to determine the effects of specific hazards -- to link health and lifestyle information. The commenter states that, for example, had the proposed rule been in place 25 years ago, when the Harvard Six Cities and American Cancer Society studies were first published, air pollution today would be much worse. The commenter states that these two studies laid the foundation for the 1997 PM NAAQS that set the first national standard for PM2.5. The commenter states that they also were used in all subsequent updates to the federal PM2.5 standard as well as development of the national PM10 standard. The commenter states that, in addition, these studies were integral to the development of more than two dozen different rules adopted by the EPA to implement the NAAQS since 1997. The commenter states that key rules developed using these studies to evaluate particle pollution impacts on mortality and illnesses include the Control of Emissions from Non-road Diesel Engines and the Mercury and Air Toxics Standards. The commenter provides several examples of the types of studies and models that, according to the commenter, the SNPRM would force the EPA to reject or discount including studies related to (1) fecal indicator bacteria concentrations and water-content recreation at ocean and freshwater beaches; (2) recent U.S. EPA lifetime health advisories for PFOA and Perfluorooctane Sulfonate (PFOS); and (3) studies that document adverse health impacts associated with air pollutants as part of setting NAAQS.
Commenter (12430) discusses the reliance of EPA lifetime health advisories for Perfluorooctanoic Acid and Perfluorooctane Sulfonate on epidemiological studies, for which exposure may result in several adverse health effects. The commenter also states the importance of epidemiological studies in relation to setting NAAQS, highlighting the importance of the Harvard Six Cities study and American Cancer Society study in the first national PM2.5 standard.
Commenter (12430) states that the proposed rule is likely to preclude U.S. EPA from considering high-quality CARB-funded research that provides critical findings on air pollution exposures and health impacts. The commenter notes that a significant number of the studies that CARB supports are epidemiological/cohort studies that the SNPRM is likely to preclude from U.S. EPA consideration. The commenter provides the following examples:
 The 10-year Children's Health Study (CHS): Initiated in 1993, this was the first major study to assess the impacts of long-term air pollution exposure on the respiratory health of California's children. Following 5,500 students in 12 southern California communities from fourth grade through high school, this study revealed the extent to which ozone, nitrogen dioxide, acid vapors consisting of nitric acid and hydrogen chloride, and particulate matter affect children's lung development. The results of this study are evidence for classifying children as sensitive receptors to air pollution and have influenced research since and shaped California legislation addressing children's microenvironments. 
 The Los Angeles Family and Neighborhood Survey (LAFANS): This was a study of families in different neighborhoods in Los Angeles County. The researchers found that children more highly exposed to traffic pollution were 30-40 percent more likely to report wheeze symptoms. 
 The East Bay Kids Study and the California Health Interview Survey (CHIS): These studies sought to determine impacts of pollution levels and greater sensitivity in low income neighborhoods on asthma, including in the CHIS study, on whether the asthma burden disparity is due to exposure to higher levels of air pollutants, greater vulnerability, or both. Findings from these studies have helped to inform policy decisions on motor vehicle emissions control and enforcement, and asthma prevention, control, and education in low socioeconomic status populations. 
 Wildfire Impact: Studies currently in progress are examining the impacts of wildfire smoke on lost work days and on respiratory symptoms.
Commenter (12431) states that the proposed rule could force the EPA to exclude from its consideration epidemiologic studies that use confidential health data. The commenter states that such studies are frequently the most relevant to protecting public health as they assess outcomes in humans following real world exposures. The commenter states that the exclusion of this information may lead the EPA to increase its reliance on information from less relevant animal toxicity studies. 
Commenter (12435) notes that the retroactive application could include studies on toxic substances ranging from lead to particulate matter which were properly conducted in accord with health privacy regulations, including the Harvard Six Cities and American Cancer Society Cancer Prevention Study II that serve as major references to the PM NAAQS. Additionally, the commenter states that a majority of the cost-benefit analyses for air regulations is based on these studies because they present the best epidemiological data available. According to commenter, if these studies are excluded from future regulatory action, this would also prevent their use for the EPA's cost-benefit analyses. The commenter provides that foundational studies that have gone through peer review contribute to the body of knowledge on a topic and should not be excluded.
Commenter (11497) states that a likely outcome of the proposed rule is that many epidemiological studies showing the effects of environmental exposures would be disregarded; studies which cannot now be replicated, and whose contribution to and critical role in the pool of "best available science" is undisputed.
Response: In the final rule the terms replicate and reproduce (or other forms of the word) are no longer used. The EPA is clarifying that the intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze." In addition, the EPA is modifying the definition of "independent validation" in the final rule by replacing "capable of being substantially reproduced" with "produced." The EPA will not finalize the proposed 40 CFR 30.2 definition of "capable of being substantially reproduced" because the term is not used in the final rule's definition of "independent validation" or elsewhere in 40 CFR 30.
Based on a consideration of these and other similar comments, the EPA clarified in the final rule how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating significant regulatory actions or developing influential scientific information. As described in the preamble to the final rule, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information, consistent with existing Agency practice. The EPA will continue to use the following, established factors to assess the quality of studies used to develop significant regulatory actions and influential scientific information: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. Only later in the process, after study quality has been assessed, the EPA will then identify those studies considered to be pivotal science. When evaluating those scientific studies considered to be pivotal science -- in either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges, and agrees with commenters, that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
Although independent validation by a third-party can be informative, the presence of a third-party validation is provided as a case-by-case exemption criterion instead of an automatic rule consideration in order to more fully realize the benefits of open data. As discussed in the preamble to the final rule, when data are widely available, researchers can validate research results and help identify and correct unintended errors, as well as reanalyze the data for new and different purposes, examine novel questions, provide new scientific insights, and improve model development. The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. 
The EPA also agrees that data sharing mechanisms, such as that used by the Health Effects Institute in conducting their reanalysis study, already exist. The final rule gives equal consideration to studies with underlying dose-response data that are available publicly and studies with underlying data that are available through restricted access. The preamble to the rule clarifies that restricted access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. The rule does not prescribe a particular mechanism but does require the EPA to independently verify that such a mechanism exists for dose-response data underlying pivotal science. As discussed in the preamble to the final rule, including this review of dose-response data availability for pivotal science is critical to the EPA's progress toward increased transparency and providing increased opportunity for scientific reanalysis and review by independent third parties.
The EPA would like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
In addition, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. 
Finally, the EPA would like to clarify that the final rule applies prospectively to significant regulatory actions and influential scientific information and has no retrospective effect on existing significant regulatory actions or influential scientific information.
Comment: Commenter (12720) asserts that it may not be feasible to identify and make available data in epidemiological studies and, as a result, the proposal would disallow use of an enormous body of carefully protected, de-identified health data from epidemiological studies large and small, for which IRBs have approved collection because patient privacy has been protected. The commenter states that this measure would hamstring the research community's ability to continue to produce foundational, health-protective research. The commenter states that not only would the rule destroy society's collective ability to benefit from studies of the causes of and potential cures for ill health, it also would veer dangerously toward compromised privacy during an era in which electronic data security is a nationwide crisis. The commenter states that the proposal flies in the face of decades of statutory, regulatory and institutional progress to simultaneously protect public health and privacy and should be withdrawn.
Commenter (12720) states that the supplemental proposal is even more expansive than the proposal and would seriously reduce the Agency's ability to characterize health impacts of proposed regulations in influential science products, including key regulatory impact analyses regularly produced by the Agency. For example, the commenter notes that in the last three years alone, EPA has regularly relied on a set of peer-reviewed epidemiology studies that relate long-term air pollution exposures to premature mortality risks (a list of these studies can be found in Table 1 of their comment letter). The commenter states that each of the environmental epidemiology studies identified in Table 1 of their comment letter does not meet the strict data release requirements identified in the supplemental proposal; nevertheless, EPA has identified these studies as providing robust dose-response information upon which to base regulatory impact analyses across a range of sectors.
Commenter (12720) asserts that the foundational research in air pollution epidemiology demonstrating a causal link between pollution exposures and adverse health outcomes  -- including early death, heart disease, lung cancer, stroke, and asthma exacerbations  --  is at risk of being excluded from the regulatory process - seriously undermining the efficacy of regulations on which this research has historically been based - if the Supplemental Proposal is finalized.
Response: Based on a consideration of these and other similar comments, the EPA clarified in the final rule how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating significant regulatory actions or developing influential scientific information. As described in the preamble to the final rule, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information, consistent with existing Agency practice. The EPA will continue to use the following, established factors to assess the quality of studies used to develop significant regulatory actions and influential scientific information: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. Only later in the process, after study quality has been assessed, the EPA will then identify those studies considered to be pivotal science. When evaluating those scientific studies considered to be pivotal science -- in either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation.
The EPA would like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that are unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator, as detailed under 40 CFR 30.7. One of the criteria the Administrator may use in his/her determination to grant a waiver is whether making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, CBI, or national security. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
These considerations, the Administrator's exemption, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data are intended to minimize potential unintended consequences, such as that noted by the commenter. In addition, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. The EPA believes the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making.
Comment: Commenter (12720) states that vinyl chloride (VC) is regulated in workplaces, and in drinking water, food, and air: 
 OSHA issued workplace regulations in 1974, forcing a reduction in the allowable level of the VC monomer by 500-times, from 500 ppm to 1 ppm averaged over an 8-hour workday. Despite predictions of dire job losses, virtually all U.S. manufacturing facilities met the new standard within a few years while reducing costs, largely through better containment of the unpolymerized monomer and improved exposure monitoring. 
 EPA regulates VC pollution under the SDWA (Maximum Contaminant Level (MCL)=0.02 mg/L based on increased risk of cancer), and under EPA's Ambient Water Quality Criteria (0.025 ug/L). 
 FDA regulations limit vinyl chloride in food contact materials and packaging. 
The commenter (12720) provides that studies that support these EPA safeguards, and particularly the identification of diseases in workers like acroosteolysis that were not identified in rodent studies, are critical to protecting human health and preventing adverse environmental impacts. The commenter adds that thanks to effective health-protective regulatory actions by EPA, OSHA and other federal agencies - regulatory actions that were based on these studies - the elevated exposure conditions suffered by industrial workers in the 1970s and earlier are no longer the industry norm. Thus, the commenter states that these studies cannot meet the standards of transparency and replicability set out in the Proposal. Because these studies cannot meet the "standards" set out in the supplemental proposal, the commenter expresses concern that they may no longer form the basis of regulatory action, seriously compromising the safeguards on which we rely to protect against exposure to VC.
Response: Based on a consideration of these and other similar comments, the EPA clarified in the final rule how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating significant regulatory actions or developing influential scientific information. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
Comment: Commenter (12720) provides that Congress recognized that pesticides are designed to be poisonous, and thus requires them to be registered by EPA, under FIFRA. The commenter states that Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA) requires that when used according to the label, a pesticide will not cause unreasonable adverse effects on the environment or human health, which is commonly referred to as FIFRA's safety standard. The commenter adds that FIFRA was amended by the Food Quality Protection Act (FQPA), which was passed by Congress unanimously in 1996.
Due to risks to children's health, the commenter (12720) states that in 2000 EPA banned household use of chlorpyrifos and most other organophosphate pesticides. The commenter notes that residential uses prior to the ban were causing very high exposures to pregnant women and young children and scientists have since learned that even much lower levels may be harmful to children.
However, according to the commenter (12720), scientists have since shown in longitudinal cohort epidemiologic studies, that even low levels of exposure -- too low to poison a pregnant mother -- can disrupt brain development in their prenatally exposed children, leading to developmental delays, lower IQ, learning disabilities, and ADHD-like behaviors. 
To protect these children, in October 2015, the commenter (12720) provides that the EPA proposed to ban chlorpyrifos because Agency scientists found contamination of drinking water. A year later, the commenter states that the EPA found that chlorpyrifos residues on food, including fruits and vegetables, are unsafe for pregnant women and children; residue levels were far above their target risk level -- in some cases, by up to 140 times. 
According to the commenter (12720), these epidemiologic studies can no longer be reproduced because -- thanks to FQPA and the ban on residential uses -- pregnant women and young children are no longer poisoned by indoor use of organophosphate pesticides at such high levels. Without these epidemiological studies, the commenter states that our regulators will no longer be able to use the best available science to ensure that pregnant women, fetuses, and young children receive protection from exposure to chlorpyrifos. The commenter contends that the exclusion of these studies and its corresponding impact reveals EPA's Proposal and Supplemental Proposal to be arbitrary, capricious, and an abuse of discretion. 
Response: In the final rule the terms replicate and reproduce (or other forms of the word) are no longer used. The EPA is clarifying that the intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze." In addition, the EPA is modifying the definition of "independent validation" in the final rule by replacing "capable of being substantially reproduced" with "produced." The EPA will not finalize the proposed 40 CFR 30.2 definition of "capable of being substantially reproduced" because the term is not used in the final rule's definition of "independent validation" or elsewhere in 40 CFR 30.
Based on a consideration of these and other similar comments, the EPA clarified in the final rule how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating significant regulatory actions or developing influential scientific information. As described in the preamble to the final rule, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information, consistent with existing Agency practice. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. 
Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that are unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator, as detailed under 40 CFR 30.7. One of the criteria the Administrator may use in his/her determination to grant a waiver is whether making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, CBI, or national security. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
These considerations, the Administrator's exemption, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data are intended to minimize potential unintended consequences, such as that noted by the commenter. In addition, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. The EPA believes the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making.
Including this review of dose-response data availability for pivotal science is critical to the EPA's progress toward increased transparency and providing increased opportunity for scientific reanalysis and review by independent third parties.
Comment: Commenter (12720) asserts that lead regulations and reduction measures resulting from the implementation of science-based EPA regulations are essential for reducing lead poisoning effects in the U.S. population. According to the commenter (12720), since 2001, life-saving EPA standards under TSCA have protected children and families from exposure to lead in paint, dust, and soil, in and around homes and childcare facilities. The commenter also notes that the EPA Lead and Copper Rule (LCR) of 1991 established drinking water protections by requiring tap water monitoring and triggering a public alert and some protective action such as corrosion prevention measures or service line replacement if lead levels exceed 15 ppb. 40 C.F.R. Part 141 Subpart I. The LCR rule applies to water utilities, and the companion Reduction of Lead in Drinking Water Act sets standards for pipes, solder, and other plumbing fittings.
The commenter (12720) provides that the lead rules are based on risk analyses conducted by EPA using epidemiology studies published in the 1990s that correlate childhood blood lead levels with impaired brain function and adverse behavioral effects. The commenter notes that many of the published studies are longitudinal cohort studies that include measurements of lead in blood from children decades ago, and then follow them out over time to observe lasting effects. The commenter states that if these studies are excluded from consideration under the Supplemental Proposal, the integrity of regulations designed to limit exposure to lead would be seriously undermined.
Further, the commenter (12720) adds that precisely because of important EPA regulations and effective lead-reduction measures in gasoline and paint, overall blood lead levels have been reduced in many people. As a result, the commenter states that it is impossible to replicate the exposure conditions at the time the original children in the study cohort had their blood lead levels measured, such as the Port Pirie cohort study population living near a lead smelter in the 1980s. The commenter contends that studies like these -- longitudinal cohort studies, particularly those that capture exposures that may no longer occur -- are not reproducible.
Response: Based on a consideration of these and other similar comments, the EPA clarified in the final rule how the Agency will evaluate the quality of scientific studies when promulgating final significant regulatory actions and finalizing influential scientific information. As described in the preamble to the final rule, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information, consistent with existing Agency practice. The EPA will subsequently consider data and model availability when evaluating pivotal science and give greater consideration to studies where the underlying data and models are available for independent validation. The EPA acknowledges that there may be pivotal science where the underlying data and/or models are not available in a manner sufficient for independent validation. In these cases, the EPA may still use these studies in final significant regulatory actions and influential scientific information after either giving lesser consideration to the study or studies or if the Administrator grants an exemption under 40 CFR 30.9.
Comment: Commenter (12720) provides that mercury is a powerful neurotoxic agent capable of adversely affecting fetus and childhood development in low concentrations. The commenter provides an extensive review of the scientific literature in their comments pertaining to health effects and costs of mercury exposure. 
The commenter (12720) notes that the mercury Reference Dose (RfD) is based on recommendations of the NRC of the NAS, that conducted an extensive analysis and calculations derived from three longitudinal epidemiologic studies: the Seychelles Islands, the Faroe Islands, and the New Zealand studies. The commenter states that the studies measured neuropsychological effects in children that were exposed prenatally to methylmercury as a result of pregnant mother's consuming contaminated seafood. The commenter adds that the use of these studies to set EPA exposure limits was the result of a years-long transparent process of expert scrutiny, public engagement, inter-agency cooperation, and publication in scientific journals. 
However, according to the commenter (12720), the studies can no longer be reproduced, particularly the Faroe Islands study in which the exposure to the community was a result of eating whales, a practice that has since declined due to public alerts about the hazards of eating the mercury-tainted meat particularly for children and pregnant and breastfeeding women. In addition, the commenter states that it would take decades to repeat the studies, which took decades to conduct in the first place. The commenter notes that if EPA may no longer rely on these studies under the Supplemental Proposal, the Agency will not have access to the information it needs to ensure its regulations are sufficiently protective of public health. 
Response: Based on a consideration of these and other similar comments, the EPA clarified in the final rule how the Agency will evaluate the quality of scientific studies when promulgating final significant regulatory actions and finalizing influential scientific information. As described in the preamble to the final rule, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information, consistent with existing Agency practice. The EPA will subsequently consider data and model availability when evaluating pivotal science and give greater consideration to studies where the underlying data and models are available for independent validation. The EPA acknowledges that there may be pivotal science where the underlying data and/or models are not available in a manner sufficient for independent validation. In these cases, the EPA may still use these studies in final significant regulatory actions and influential scientific information after either giving lesser consideration to the study or studies or if the Administrator grants an exemption under 40 CFR 30.9.
Comment: Commenter (12431) states that excluding information that cannot be made publicly available would slow the development and implementation of alternative test methods and strategies to replace animal testing. The commenter states that such approaches can provide information of equivalent or better scientific quality and relevance for risk assessment, and their development and implementation are mandated by the TSCA. The commenter states that this potential conflict between public accessibility and continued progress in regulatory science has been recognized by EPA's Office of Pollution Prevention and Toxics (OPPT). The commenter states that, in its Strategic Plan to Promote the Development and Implementation of Alternative Test Methods Within the TSCA Program, OPPT states that public access to the datasets used to build models that predict human outcomes may not be possible in some cases, because OPPT uses CBI to update and refine its models, and these activities are critical to its mission of evaluating chemicals under TSCA. The commenter states that, under the proposed rule and SNPRM, OPPT could be limited to using only publicly accessible information and open source models. The commenter states that, in some cases, results from commercially available models can be used without the need to release proprietary code, and requiring commercial model developers to release this intellectual property would likely restrict their ability to continue developing and supporting their models. The commenter states that sponsors of new industrial chemicals and agrochemicals sometimes submit chemical and toxicological information derived from proprietary alternatives in lieu of certain animal tests. The commenter states that, under the proposed rule and SNPRM, the EPA may be unable to consider this information, resulting in animal testing that would, in the case of industrial chemicals at least, be conducted in violation of TSCA provisions to use available nonanimal test methods and strategies. 
Commenter (12720) states that the epidemiologic science and associated studies that are the basis of adherence to the current EPA's radiation protection standards are likely to be expressly excluded from consideration by EPA by the terms of this Proposal. The commenter notes that the EPA relied on radiation dose-response models to develop its radiation protection reports and regulations.
The commenter (12720) provides that, for the past several decades, these models have been used when developing practical and prudent guidance on ways to protect the public from potentially harmful effects of radiation while, at the same time, balancing the beneficial, justified and optimized use of radiation. The commenter states that these models are derived from studies by authoritative scientific bodies, including the U.S. NAS, and the National Council on Radiation Protection and Measurements (NCRP). According to the commenter, the EPA's proposal, if implemented, would limit EPA's staff from basing regulatory actions on precisely these types of studies by requiring that all the underlying data be publicly shared.
The commenter (12720) states that these are profoundly important studies that have been peer-reviewed for decades and the science that has emerged from them has been validated multiple times. According to the commenter, however, these are not studies where the entirety of the public data can be shared or independently replicated. The commenter expresses that there are no other radiation epidemiologic studies of health and longevity on a large population (example: more than 120,000 individuals in the atomic-bomb survivor studies) that have continued for more than 60 years. Thus, the commenter states that the replication of the studies is impossible and unethical as this data comes from individuals exposed to significant acute and a protracted dose of radiation. The commenter contends that the implementation of the rule would effectively block the use of such key scientific studies and allow for radiation standards to be weakened.
The commenter provides examples of regulations that could be adversely impacted by this rule include:
 Cancer Risk Coefficients for Environmental Exposure to Radionuclides 
 EPA Radiogenic Cancer Risk Models and Projections for the US Population (the "Blue Book") 
 The Uranium Fuel Cycle (40 CFR Part 190)  -  a standard that sets generally applicable environmental limits for the entire uranium fuel cycle 
 Soil cleanup levels for Superfund sites
 The concept of ALARA (As Low As Reasonably Achievable) in radiation protection
Response: Based on a consideration of these and other similar comments, the EPA clarified in the final rule how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating significant regulatory actions or developing influential scientific information. As described in the preamble to the final rule, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information, consistent with existing Agency practice. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
With this final rule, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
2.2.2.1.3 Meta-Analyses
Comment: Commenters (9455, 11176, 11911, 12405, 12617, 12720) express concern that, while systematic reviews and/or meta-analyses are often extremely informative in the health and environmental sciences, the proposed rule may discount or exclude such studies.
Commenter (9455) asserts that, because only those studies with accessibility would be allowed into the highly considered level, evaluation of consonance among all high-quality, unbiased studies would not occur under this scheme. The commenter states that this is a glaring scientific gap that disallows full meta-analyses and comparison of all high-quality studies. 
Commenter (11176) states that meta-analyses and systematic reviews have become important tools in evidence-based policy making. The commenter states that meta-analyses present a special challenge for data sharing because meta-analyses often do not begin with raw data, but with data that have already been "processed" to some degree (for example, the data may be from the results of primary analyses in the studies contained in the meta-analysis). The commenter states that, in such cases, the definition of raw data underlying the meta-analysis is ambiguous and the author of the meta-analysis may not have access to the underlying data in each study used in the meta-analysis. The commenter states that, even if the data were available for meta-analyses, data repositories for meta-analytic data do not currently exist. The commenter states that systematic reviews, one particular form of meta-analysis, are regularly used by EPA to bring together the many relevant studies and develop a summary weight-of-evidence recommendation. The commenter states that de-emphasizing such analyses could greatly weaken the EPA's ability to develop regulations and make them harder to defend in court challenges.
Commenter (11176) notes that researchers are developing multi-level models and privacy-preserving distributed algorithms to draw scientific conclusions based on data stored in multiple locations. The commenter states that related work is also a major focus in medical informatics for data from different hospitals (e.g., electronic health record data and biobank data) while protecting patient privacy. The commenter states that making such data available in any way would be impractical if not impossible.
Commenter (11911) states that the EPA has not addressed the impact of the proposed rule on scientific meta-analyses. The commenter states that, if researchers conclude that the rule's exclusion covers meta-analyses that incorporate some studies with underlying, unavailable, data, this would threaten the reliability of the meta-analyses themselves. The commenter states that the entire purpose of a meta-analysis is to aggregate large numbers of scientific studies, incorporating different datasets and research methods that span years. The commenter states that designating a study as having no evidentiary value simply because the underlying data cannot be made publicly available is arbitrary, and would result in a less accurate, or biased, average estimate. The commenter states that it would reduce the sample size of a meta-analysis, making it more difficult to draw legitimate statistical conclusions. The commenter states that doing so is inconsistent with the EPA's own guidance for conducting meta-analyses, which states that studies should be weighted according to their sample size or standard error. 
Commenter (11911) states that the harmful effects of categorical-exclusion policies can be witnessed in the recent proposals to repeal and replace the 2015 Clean Water Rule. The commenter states that the Agency excluded studies for reasons such as study age, type of wetlands, sample size, and lack of summary statistics. The commenter states that each of those issues could be addressed by weighting the studies appropriately as part of a meta-analysis. 
Commenter (12405) states that the herbicide paraquat is currently undergoing reregistration review. The commenter states that over the past few decades, studies consistently show a correlation between exposure to pesticides and Parkinson's disease, but that full breadth of data may not be reviewable by EPA under the current proposal. The commenter states that, for example, a meta-review examined 40 studies and concluded, "epidemiologic studies suggest a relatively consistent association between exposure to pesticides and an increased risk of developing [Parkinson's disease], despite differences in study design, case ascertainment and definition, control selection, and pesticide exposure assessment." The commenter states that many of these studies would be excluded from consideration under the proposed rule.
Commenter (12617) expresses concern that, while systematic reviews and meta-analyses are often extremely informative in the health and environmental sciences, it is unclear how the SNPRM applies to such studies. The commenter asks if the data access requirements apply to every study considered in a review or meta-analysis and, if so, systematic reviews and meta-analyses would be, in effect, universally ineligible for consideration at EPA.
Response: Based on a consideration of these and other similar comments, the EPA clarified in the final rule how pivotal science would be selected. As discussed in the preamble to the final rule, rather than considering all studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact as "pivotal science," the EPA is balancing transparency and feasibility by focusing on those studies that inform the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect. Thus, "pivotal science" includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). These are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. In some cases, there may be a large number of studies that are used to characterize a dose-response relationship (e.g., where the dose-response is based on a meta-regression of epidemiology studies); however, not all of these studies would be considered pivotal science. In the context of this rule, studies informing the dose-response modeling (e.g., benchmark response selection), studies identifying data for toxicokinetic adjustments, or studies informing the selection of uncertainty factors do not drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. 
The final rule also clarifies how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating significant regulatory actions or developing influential scientific information. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges, and agrees with commenters, that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
The EPA also clarified in the preamble to the final rule that the requirements in the final rule set the overarching structure and principles for transparency in the EPA's significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes EPA administers.
2.2.2.1.4 Coronavirus Disease 2019 (COVID-19) Implications
Comment: Commenter (12411) states that the proposed rule will limit the EPA's ability to create scientifically informed clean air regulations, which is especially reckless, right now, as the world faces COVID-19. Scientists have made the link between poor air quality and increased COVID-19 related health risks.
Commenter (12458) states that a large body of research has linked air pollution to premature death over the past several decades but in the current COVID-19 crisis, several preliminary studies have linked air pollution specifically to higher rates of COVID-19 fatality. The commenter contends that functionally excluding these kinds of public health studies would undermine public health at any time and to do so during a public health crisis is starkly at odds with the Agency's obligation to protect human health.
Commenter (12430) provides that a recent report from the Harvard T.H. Chan School of Public Health studied data from 3,000 counties and found that the presence of particulate matter in the atmosphere could increase the severity of COVID-19 cases. Specifically, it concluded:
 A small increase in long-term exposure to PM2.5 leads to a large increase in COVID-19 death rate, with the magnitude of increase 20 times that observed for PM2.5 [fine particulate] and all-cause mortality. The study results underscore the importance of continuing to enforce existing air pollution regulations to protect human health both during and after the COVID-19 crisis. 
 Other studies will likely follow; indeed, U.S. EPA has already directed the Science Advisory Board to investigate "[e]nvironmental [f]actors affecting transmission and severity of COVID-19." More specifically, U.S. EPA has requested that the Science Advisory Board examine the following questions:
 Can particulate matter in the atmosphere serve as a vehicle for the transmission of SARS-CoV-2?
 Does exposure to air pollutants, including wildland fire smoke or other air pollutants (e.g. ozone, particulate matter, diesel exhaust, pollen) increase the susceptibility to respiratory viruses like SARSCoV-2? Or exacerbate existing COVID-19 infection?
 Does ambient or indoor temperature or humidity affect the transmission of SARS-CoV-2 and severity of the COVID-19 illness? 
To the extent that current or future studies on these issues rely upon restricted data or models, the commenter contends that the supplemental proposal would prevent U.S. EPA from considering them. As a result, it would interfere with U.S. EPA's long-term efforts to address this virus or potential future ones.
Commenter (12692) states that, as summarized by E&E, Stanford Children's Health doctor, Lisa Patel, warned that the Rule would "bar the agency from using the latest scientific research on air pollution and disinfectants amid rapidly evolving public health crises." In other words, the commenter provides that the Transparency Rule would somehow prohibit EPA from posting and updating a list of recommended disinfectants for use against COVID-19. The commenter contends that is beyond ridiculous. The commenter states that the Transparency Rule applies only to "pivotal regulatory science" and "influential scientific information" (85 FR 15405) and that nothing in either the original proposal or the subsequent supplemental would prevent EPA from recommending the use of hydrogen peroxide or ammonium chloride (the chemical in disinfectant wipes) to kill viral pathogens on contaminated surfaces.
Commenters (10045, 11378, 11386, 11403, 14397, 12617) express concern that studies that could be usefully relied upon during a pandemic or other crisis would be systematically excluded from being used in EPA's scientific and regulatory efforts if this rule is finalized.
Commenter (10045) states that COVID-19 provides an example of the public health risks of this proposal. The commenter states that placing time-consuming barriers to the use of scientific information could in some cases be more than a mere annoyance or ministerial task; it could be fatal. The commenter states that the following are three examples of research that would seem highly applicable to EPA's responsibilities to respond to the COVID-19 pandemic:
 A March 2020 survey of existing research that describes the interaction of several coronaviruses on surfaces with biocidal agents; the EPA is responsible for recommending disinfectants for use against SARS-CoV-2 and other pathogens. 
 A 2003 paper which describes a statistical correlation between SARS fatalities in China and higher exposure to air pollution, information that could be relevant to EPA air officials as new criteria pollutant standards or air quality advisories for those at risk of the most adverse effects of COVID-19 are developed. 
 A 2015 paper that describes the challenges associated with the sterilization and disposal of medical waste contaminated by Ebola; the EPA worked jointly with CDC to develop disposal and sterilization guidelines and could be called on to do the same for SARSCoV-2. 
Commenter (10045) states that, since the proposed rule would systematically exclude studies that do not meet the EPA's criteria for independent validation, critical studies that would otherwise inform influential scientific information will not be used or incorporated into key documents such as the Integrated Scientific Assessment for Particulate Matter. The commenter states that this document is informed by studies that show that air pollution exposure increases the risk of developing lower respiratory infection. The commenter states that an examination of Table 5-10 of this document reveals that many of these studies are epidemiological papers analyzing the numbers of hospital visits or other medical outcomes in locations all around the world on days when air pollution is high. The commenter states that it is highly likely that some or all of these studies would be presumptively excluded from being used by EPA's rule, which means that they could cease to be incorporated into updates of EPA's Integrated Scientific Assessment, and thus excluded from being used both to inform the EPA's criteria air pollutant regulatory efforts as well as to inform policymakers seeking to assess the influence of air pollution on and develop and implement measures to mitigate COVID-19 (or other future national or global challenges). 
Commenter (11378) contends that confidential and proprietary geospatial data, including remotely sensed satellite imagery, location tracking of individual mobility, and georeferenced demographic and health information (which can reveal identity), play essential roles in environmental and health research. The commenter states that, for example, the U.S. response to the COVID-19 pandemic depends on temporarily restricting human mobility and tracing the spatial contacts of infected individuals in order to mitigate disease transmission, which both rely on confidential, individual-level geospatial location data for monitoring and modeling. The commenter states that the proposed rule suggests that such research could be discounted unless the underlying data, containing real-time whereabouts of individuals, be made available to the public. 
Commenter (11386) states that the proposed rule discounts the use of study conclusions based on publicly unavailable data. The commenter states that an example of when this policy may be harmful is in the case of a quickly evolving health emergency such as the current COVID-19 pandemic. The commenter states that, in this case decisions may need to be made quickly based on the science that is available at the time, which may be incomplete or may not have had enough time to undergo the thorough cleaning, scrubbing, and organization that would be necessary to safely and lawfully make that data publicly available. The commenter states that, by not considering such data in these situations the EPA may respond poorly to important public health emergencies as a result of this rule. 
Commenter (11403) states that the current COVID-19 pandemic has highlighted just how important it is for government agencies to be able to use health data in decision making. The commenter states that future studies on the spread and effects of the virus would be limited by the availability of the raw data, which is rightly protected by HIPAA. The commenter states that, for example, any studies drawing a comparison between instances of predisposition to respiratory illness in areas of poor air quality to cases of COVID-19 would be excluded from consideration by EPA. The commenter states that studies of this nature are of critical importance for an environmental health agency, and restricting the degree to which they may be considered is irrational. 
Commenter (12617) expresses concern that the proposal places an extra burden on research that helps the world understand and fight the COVID-19 epidemic; the requirements to make all the raw data public while somehow attempting to protect people's medical information. The commenter states that one striking study showed a strong correlation between risk of death from the disease and exposure to particulate air pollution; and states that this work, at least after peer review, ought to be allowed to inform rules on particulate emissions.
Commenter (14397) states that the current coronavirus pandemic is an excellent example of a health crisis that could be exacerbated by this SNPRM. The commenter states that an article in The Independent quotes James Goodwin, senior policy analyst at the Center for Progressive Reform as saying: "A lot of the science needed to effectively respond to this pandemic would not be available if this rule existed today." The commenter states that these fears were echoed by Senator Tom Carper who stated "the ongoing COVID-19 pandemic has illustrated the importance of ensuring rapid access and response to scientific information, as well as the utilization of that information." The commenter states that it is difficult to see how the research community could, under this proposal, learn anything from this pandemic while still following the appropriate HIPAA regulations. The commenter states that those who survive this crisis can be thankful that this proposal was not in affect at the time of the initial outbreak, as virtually no medical data could be immediately shared with epidemiologists or researchers under that scenario.
Commenter (12701) asserts that the current pandemic underscores the need for decision-making of federal agencies, including the EPA, to be informed by the best available science. For example, the commenter provides that policymakers need to understand scientific evidence regarding environmental transmission of the coronavirus and how environmental factors such as air quality might impact COVID-19 outcomes. According to the commenter, the supplemental proposal would not adequately allow the EPA to prioritize the best emerging scientific studies when assembling influential scientific information necessary to inform the best policy.
Response: Based on a consideration of these and other similar comments, the EPA clarified in the final rule how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating significant regulatory actions or developing influential scientific information. As described in the preamble to the final rule, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information, consistent with existing Agency practice. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
The EPA believes that the approach described in the final rule will result in significant regulatory actions and influential scientific information based on the highest quality, most relevant studies and ensure adequate human health and environmental protections while maximizing transparency and maintaining consistency with the statutes the EPA administers.
The EPA would like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Finally, the EPA acknowledges that the COVID-19 situation is unprecedented. The EPA's most important environmental statutes provide the Agency with the authority to issue emergency orders or respond to address emergencies to protect human health and the environment. This final rule would not limit or impede the EPA's authority to undertake such responses. Moreover, the final rule includes sufficient flexibility in terms of the considerations in 40 CFR 30.5, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data in order to minimize potential unintended consequences, such as a potential impact during a crisis situation. In addition, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. The EPA believes the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making.
2.2.2.1.5 Comments on Other Commenters' Science Limitation Concerns
Comment: Commenter (12692) disputed/commented on several claims regarding the limiting of science/data as a result of the proposal made by a commenter (Deborah Wallace, Ph.D. in Ecology) as follows: 
      Claim: The proposal would thwart meta-analysis, or research that analyzes studies across disciplines, because many studies would be excluded.
      Comment: A meta-analysis is the use of statistics to summarize the results of other studies. Nothing in the Transparency proposal would prevent EPA analysts from conducting or reviewing meta-analyses. The new emphasis on independent validation might, however, make meta-analyses more useful by making the underlying studies more reliable.
Response: Based on a consideration of these and other similar comments, the EPA clarified in the final rule how pivotal science would be selected. As discussed in the preamble to the final rule, rather than considering all studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact as "pivotal science," the EPA is balancing transparency and feasibility by focusing on those studies that inform the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect. Thus, "pivotal science" includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). These are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. In some cases, there may be a large number of studies that are used to characterize a dose-response relationship (e.g., where the dose-response is based on a meta-analysis); however, not all of these studies would be considered pivotal science. In the context of this rule, studies informing the dose-response modeling (e.g., benchmark response selection), studies identifying data for toxicokinetic adjustments, or studies informing the selection of uncertainty factors do not drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. 
As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
Comment: Commenter (12692) states that the Environmental Protection Network (EPN) cites an article titled "All science should inform policy and regulation" by Stanford professor John Ioannidis, the most prominent replication crisis scholar. Ioannidis warns that EPA's plan to "ban the use of scientific studies for regulatory purposes unless all their raw data are widely available in public and can be reproduced," would have dreadful consequences: "science would be practically eliminated from all decision-making processes. Regulation would then depend uniquely on opinion and whim." 
In response to this, commenter (12692) states that Dr. Ioannidis' description of EPA's proposal is a caricature. The commenter notes that the EPA is not proposing to "ban" the use of studies unless the raw data are "widely available." The EPA is proposing a system of "tiered" access including the use of highly restricted "data enclaves."
The commenter (12692) provides that independent researchers seeking access to data involving PII would be subject to the same legal requirements and penalties that prevent the original researchers from publicizing confidential information. And that is only if such access is legally permissible. As noted, the commenter states that the EPA's goal is to "ensure that, over time, more of the data and models underlying the science that informs regulatory decisions . . . is available to the public for validation in a manner that honors legal and ethical obligations to reduce the risks of unauthorized disclosure and re-identification" (83 FR 18770). 
Additionally, the commenter (12692) states that banning studies with inaccessible data would eliminate most science used in rulemaking precisely because of the replication crisis Dr. Ioannidis has documented and brought to national attention.
Response: As described in the preamble to the final rule, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information, consistent with existing Agency practice. The EPA will continue to use the following, established factors to assess the quality of studies used to develop significant regulatory actions and influential scientific information: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. Only later in the process, after study quality has been assessed, the EPA will then identify those studies considered to be pivotal science. When evaluating those scientific studies considered to be pivotal science -- in either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation.
Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
2.2.2.2 Legal Validity of Ignoring Studies
Comment: Commenter (11894) states that if EPA plans to comply simply by ignoring studies that do not meet the new requirements, an assessment of the legal validity of that approach would also be required. According to the commenter, the EPA has not done this and apparently, as with many other issues, throws the burden on commenters to figure out what it meant and why.
Response: As described in the preamble to the final rule, the rule establishes how the EPA will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information. The rule does not provide for categorically excluding or ignoring studies. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
Further, the EPA intends for the final rule to be implemented in a transparent manner. The EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. Additionally, the final rule requires the EPA to document the rationale for exemptions granted by the Administrator to the requirements included in the final rule in the significant regulatory action or influential scientific information.
Comment: Commenter (10944) states that any use of the supplemental proposal to nullify research data used as the basis for existing regulations is in direct violation of the fourteenth amendment as the reductions in environmental regulations are more burdensome to communities of color, low income communities, and children.
Response: The EPA would like to clarify that the final rule applies prospectively to significant regulatory actions and influential scientific information; the final rule has no retrospective effect on either significant regulatory actions or influential scientific information. Further, the final rule does not violate the Fourteenth Amendment to the United States Constitution and does not deprive any person of due process or equal protection under the law. The EPA would also like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. The rule also does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
 Moreover, as described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7. 
2.2.2.3 Health and Environmental Protection Implications
Comment: Many commenters (9336, 9338, 9372, 9455, 10055, 10749, 10865, 10866, 10974, 10975, 11103, 11190, 11192, 11330, 11354, 11359, 11370, 11377, 11390, 11393,11403, 11407, 11479, 11482, 11487, 11496, 11576, 11895, 11907, 11920, 12405, 12411, 12412, 12422, 12423, 12430, 12435, 12463, 12474, 12688, 12707, 12716, 12727, 13549) express that, as a result of the rule limiting the use of science to inform regulatory and policy decisions, there will be adverse health and/or environmental protection implications. Several of these commenters also contend ￼ [See also comments and EPA's response to comments in Chapter 13, section 13.4 (Limits Use of Science When Making Policy and Regulatory Decisions).] A sample of the specific comments follow:
 Commenter (11920) states that while the original proposal was seriously flawed, the supplemental notice dramatically expands the reach and impact of this arbitrary and unlawful effort to restrict the EPA's use of valid and credible peer-reviewed science. By excluding high-quality, peer-reviewed, and validly published science from the regulatory decision-making process, the commenter contends that the proposed rule undermines the integrity of these laws and will result in under-informed and inadequate protections for vulnerable communities of people and for wildlife. According to the commenter, a plain reading of this supplemental notice reveals that through restricting the use of relevant and high-quality scientific information, the EPA is seeking to protect polluters and regulated industries, rather than carry out its statutory mandate to protect the environment and American citizens. 
 Commenter (12435) states that the degradation in air quality that would surely result from this proposal would undermine public health while we are fighting a pandemic respiratory virus for which conditions such as asthma and COPD are serious risk factors.
 Commenter (12435) states that the supplemental proposal asserts that the action "does not establish an environmental health or safety standard." According to the commenter, while there are no specific standards included in the proposed rule, it has a direct impact on the data and models that can be relied on as the basis of environmental health or safety standards that demonstrably affect minority and low-income populations.
 Commenter (12435) states that the supplemental proposal asserts that the action "does not concern an environmental health risk or safety risk." To the contrary, the commenter asserts that the proposed rule could limit the ability of the Agency to consider all relevant data and models when making decisions that could have a direct impact on environmental health or safety of children. The commenter notes that children are considered a vulnerable class under the Federal Policy for the Protection of Human Subjects or "Common Rule" and dissemination of data from studies of children are even more stringently regulated than for the general adult population. Therefore, the commenter contends that it is reasonable to expect that environmental health studies involving children would be disproportionately discounted under the proposed rule, leading to regulations that do not adequately protect this sensitive population as required by the CAA. Further, the commenter adds that the proposed rule could restrict states ability to protect public health and the environment at EPA-lead CERCLA sites if EPA fails to consider data or models that might allow a greater level of protection.
 Commenter (11393, 11407, 11895) expresses that this rule would preclude EPA from considering important and pertinent scientific health studies that contain personal medical information which cannot be publicly released. The commenter contends that the rule would create barriers to regulations and arbitrarily deny consideration of important findings, thereby restricting EPA's ability to protect human health and the environment. 
 Commenter (11482) states that the proposed rule has implications for environmental health associated with the oceans, a core advocacy objective of Ocean Conservancy's mission.
 Commenter (11330) states that this proposed rule will be used to erode landmark advancements -- achievements in public health and environmental safety. For example, the commenter notes that they know the Clean Power Plan would have led to reductions in pollution that were predicted to prevent some three -- thirty-six hundred premature deaths, ninety thousand asthma attacks in children, and three hundred thousand missed school and work days each year. The commenter states that many of these health benefits were partially determined by landmark clean air studies like the Harvard Six City study. The commenter contends that this is equivalent to telling CDC they can't use health data when fighting the coronavirus. 
 Commenter (11390) states that the proposed rule, if adopted, will impose serious costs to public health and safety. The commenter contends that eliminating the consideration of relevant science from the decision-making process based solely on the availability of underlying data would compromise EPA's ability to effectively carry out its mission to protect public health and the environment.
 Commenter (9455) contends that contrary to the assertion that this proposed rule has no environmental impact, it would open current regulations to dilution or even elimination and needed new regulations to dilution or failure to be promulgated. Thus, the commenter suggests that it potentially would return the country to the days of killer smogs, rivers catching fire, widespread poisoning of fetuses and young children with heavy metals, massive illegal dumping, and extinction of species from persistent pollutants. 
 Commenter (12411) states that not only will this proposed rule be detrimental to the development of environmental and public health regulations, but it will also hurt businesses across all sectors by intensifying the climate crisis and impacting air, water, and wildlife.
 Commenter (12716) states that, in addition to threatening environmental harm, the rule would jeopardize critical programs (e.g., NAAQS) that protect human health in the watershed and throughout the country.
 Commenter (11576) notes that EPA has relied on well-established studies conducted before mechanisms existed for sharing data publicly in its assessments of the environmental risks of pesticides. The commenter also mentions that bioaccumulation data's inclusion in the supplemental proposal and the potential for undermining established science on the accumulation of mercury, lead, and other metals in aquatic systems and wildlife.
Conversely, commenter (11899) argues that critics claim the rules will exclude so much research that the rules will actually endanger public health. In fact, the commenter states that only studies EPA will likely not see is what researchers know will not survive robust peer review, and thus do not submit in the first place. According to the commenter, the real danger comes from research that is based on shoddy data, algorithms, models and analyses that past researchers have been able to keep secret. The commenter contends that is precisely what the rules will ferret out and correct.
Commenter (11479) states that, with the proposed rule's implementation, underserved communities are likely to face exacerbated harms from hazardous chemicals like ethylene oxide (EtO), which is known to increase the risk of certain cancers. When conducting the 2014 National Air Toxics Assessment, the commenter provides that the EPA determined that a number of communities are now exposed to EtO levels higher than the level that EPA has deemed safe. Because EPA was able to conduct this science-based assessment, the commenter states that it could pursue policy options to alleviate harm. For instance, the commenter notes that the EPA's Office of the Inspector General recently highlighted the need for EPA to take prompt action to inform and act on concerns from residents living near EtO-emitting facilities. However, the commenter states that if the proposed rule, as modified by the supplemental notice, is implemented, the EPA will lose the ability to rely on important scientific evidence for its internal risk assessments, likely resulting in less protective policies. According to the commenter, communities like St. James, Louisiana -- plagued by EtO levels up to 765 times higher than the levels EPA considers safe -- would be unaware about the risks they face and lose the ability to advocate for policy mechanisms that reduce those risks.
The commenter (11479) notes that another example concerns the aforementioned chemical class Per- and PFAS. EPA is currently developing regulations for PFAS chemicals, like proposing maximum contaminant levels for PFOA and PFOS in drinking water. But because research from the C8 Science Panel studies relied on human health data from tens of thousands of people, the commenter is concerned that this research could be excluded or down-weighted under the proposed rule. According to the commenter, this would weaken the scientific legitimacy of EPA's regulatory decisions and endanger public health. Since evidence suggests that PFAS contamination occurs far more frequently near communities of color and low-income communities, the commenter adds that the proposed rule is likely to disproportionately affect underserved communities.
In addition, commenter (11479) states that, under the proposed rule, as modified by the SNPRM, the EPA would be prevented from using much of this crucial research, because it relies on personal health information that cannot legally or ethically be disclosed in many cases. The commenter states that such restrictions on EPA's ability to rely on relevant scientific research on air pollution and health -- even research that the EPA itself funded -- will hinder the Agency's ability to enact health-protective NAAQS and other public health and environmental protections that the Agency is charged with enacting.
Commenter (11576) refers to detailed examples of influential human-health studies that necessarily relied on sensitive human data. The commenter states that such studies have led to the crucial strengthening of public-health protections by, for example, linking exposure to chemicals to the risk of disease. The commenter states that the SNPRM expands the scope of studies that might be subjected to the rule, which could greatly weaken public-health protections. 
Response: Based on a consideration of these and other similar comments, the EPA clarified in the final rule how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating significant regulatory actions or developing influential scientific information. As described in the preamble to the final rule, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information, consistent with existing Agency practice. The EPA will continue to use the following, established factors to assess the quality of studies used to develop significant regulatory actions and influential scientific information: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. Only later in the process, after study quality has been assessed, the EPA will then identify those studies considered to be pivotal science. When evaluating those scientific studies considered to be pivotal science -- in either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation.
Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
The EPA does not agree that this rulemaking is counter to the Agency's mission or that it would weaken human health protections. The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. The approach described in the final rule will result in significant regulatory actions and influential scientific information based on the highest quality, most relevant studies and ensure adequate human health and environmental protections while maximizing transparency and maintaining consistency with the statutes the EPA administers.
Comment: Commenter (12411) states that, overwhelmingly, scientists agree that a man-made climate crisis is upon us, warranting the need for strong environmental regulations now more than ever. The commenter notes that the impacts from the climate crisis, such as rising oceans and disruptions to supply chains, pose existential threats to our entire economy, indeed to human life itself. Unchecked, the commenter adds that the climate crisis will impact gross domestic product (GDP) by $44 trillion in just the next 40 years, while a study from the Brookings Institute says by 2100, climate impacts will slash global GDP by more than 20 percent. Yet, the commenter states that this proposed change would require scientific studies to make all their raw data public, including confidential medical records, or else not be considered for future environmental rulemaking processes. According to the commenter, while public statements from the EPA have deemed this a step towards transparency, it will in fact do just the opposite by limiting legitimate research that goes into creating clean water, clean air, and other vital regulations for the environment and public health because of privacy requirements. The commenter adds that this proposed rule does not make long-term business sense, as environmental and public health regulations directly affect businesses and their workers.
Similarly, the commenter (12411) expresses that the proposed rule will weaken clean water protections. The commenter provides that clean waterways are vital for sectors like breweries, the seafood industry, and the hospitality industry. The craft brewing industry contributed $76.2 billion to the U.S. economy in 2017, more than 500,000 jobs, and depends heavily on clean water to create a high-quality product for its consumers. The commenter adds that consumers spend $887 billion annually on outdoor recreation overall, and nearly $175 billion on fishing, kayaking, rafting, canoeing, scuba diving and other water sports alone  -  all of which thrives on a visibly healthy water supply. Further, the commenter states that clean water is important for the real estate industry  -  home values can erode by as much as $85,000 each on land near water with high nutrient pollution levels. Lastly, the commenter states that a clean water source allows each of these industries to provide safe products and activities for their customers to enjoy, while also reducing the cost of water treatment for each sector.
Commenter (12707) contends that both safe recreational enjoyment of the Great Lakes and water quality are threatened by EPA's proposal. The commenter provides that the availability of scientific data has been key to addressing important public health issues in the Great Lakes, such as the development of the 2012 Recreational Water Quality Criteria (RWQC), "designed to protect the public from exposure to harmful levels of pathogens in water-contact activities." The commenter states that the RWQC sets recommended concentration thresholds of known pathogens that cause gastrointestinal illness from exposure in water, such as E. Coli and enterocci. The commenter adds that these recommendations are informed by "the latest research and science, including studies that show a link between illness and fecal contamination in recreational waterways." According to the commenter, these recommendations provide essential information for determining warnings and beach closures for recreational users in the Great Lakes region.
Finally, the commenter (12707) states that with the rise of toxic harmful algal blooms (HABs) in the region safe drinking water has become a primary concern. The commenter provides that the EPA's 2015 Drinking Water Health Advisories for Two Cyanobacterial Toxins provide necessary guidance based on scientific studies that indicate the health effects of exposure to cyanotoxins, such as gastroenteritis and liver and kidney damage. The commenter notes that cyanotoxins, similar to pathogens like E. Coli and enterocci, pose serious health risks from exposure in recreational settings as well. According to the commenter, this relatively new area of concern is still being studied and access and use of the latest research as it continues to evolve is critical to protecting the safety of drinking water and recreation in the Great Lakes region.
Response: The EPA does not agree that this rulemaking is counter to the Agency's mission or that it would weaken environmental protections. As further detailed in the preamble to the final rule, the rule will not limit research. The requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The EPA believes that the approach described in the final rule will result in significant regulatory actions and influential scientific information based on the highest quality, most relevant studies and ensure adequate human health and environmental protections while maximizing transparency and maintaining consistency with the statutes the EPA administers. 
Finally, as described in the preamble to the final rule, that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
Comment: Commenter (12430) states that the supplemental proposal would preclude U.S.EPA from researching and fully considering the health and economic impacts to disadvantaged communities. The commenter notes that this would include both the impacts of increased exposure to pollution from nearby sources and the increased rates of illness and death from heightened vulnerability to the effects of pollution. Studies related to environmental justice are often conducted at the community level. The commenter states that it would not be possible to organize the participants into larger and less identifiable groups and be able to fulfill the purpose of the studies, which is to identify the health impacts at the community level. The commenter adds that grouping in this way is costly, time consuming, and would result in a data set that would not have the same analytical power as the original study. The commenter further adds that grouping would also reduce the locational sensitivity of the data and make more local and community-based studies unusable.
The commenter (12430) states that community level studies by CalEPA agencies -- including those required under state law and important for environmental justice considerations -- would not be considered by U.S. EPA under the SNPRM, if finalized. The commenter provides that the proposal would, for example, exclude multiple studies of the effects on children of prenatal pesticide exposure. 
Response: Based on a consideration of these and other similar comments, the EPA clarified in the final rule how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating significant regulatory actions or developing influential scientific information. As described in the preamble to the final rule, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information, consistent with existing Agency practice. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
The EPA believes that the approach described in the final rule will result in significant regulatory actions and influential scientific information based on the highest quality, most relevant studies and ensure adequate human health and environmental protections while maximizing transparency and maintaining consistency with the statutes the EPA administers.
Comment: Commenter (9455) provides extensive background information in their comment letter regarding her and colleagues' research related to the examination of public health outcomes from structural stress, i.e. chronic stress that arises out of socioeconomic disempowerment that the affected individual and community can neither fight nor flee. The commenter states that structural stress such as job strain, economic insecurity, racism, and sexism gives rise to numerous health impairments such as increased coronary heart disease (Marmot et al, 1997, abstract attached), low-weight births (Geronimus, 1996, abstract attached), metabolic syndrome and diabetes (Rosmond and Bjorntorp, 1998, abstract attached; Bjorntorp, 2001), Alzheimer's Disease and other dementias (Elovainio et al, 2009; Wang et al, 2012 ), chronic obstructive pulmonary disease (Clark et al, 2015), high blood pressure (Michaels et al, 2019), disturbed sleep patterns (Johnson et al, 2016), and obesity and the consequences of obesity (Brunner et al 1997; Bjorntorp, 2001). The commenter adds that these outcomes of structural stress indicate deterioration of underlying health in the afflicted and that many outcomes are identical to those of environmental exposures and raise the probability that structural stress and environmental exposures interact to reinforce each other and increase community incidence and prevalence of health impairments well beyond what they would be if the community had to contend with only one of the two afflictions, the well-known non-linear synergistic effect. The following examples are provided:
      Example 1. Incidence of Low-Weight Births. My own work revealed that geographic and temporal patterns of low-weight births in New York City reflected the years and neighborhoods subjected to "planned shrinkage", an extra-legal mode of urban renewal (Wallace 2011, full text attached). The discovery of the temporal patterns depended on medically confidential data on individuals. Discovery of geographic patterns depended on administrative datasets that, in turn, depended on medically confidential data (birth certificates). Other researchers (examples: Laurent et al, 2016, abstract attached; Twum et al, 2016, abstract attached) have shown that exposure to air pollutants also results in raised incidence of low-weight births, findings that depended on medically confidential data. Are we, as a nation, going to experiment with babies born to women in low income communities and brutally segregated communities of color by failing to properly regulate air pollution and seeing whether these chemicals and particles interact with structural stress to produce enormous increases in incidence of low-weight births? That would be a consequence of implementing the proposed rule. 
      Example 2. Lead Body Burden Interacting with Structural Stress. Another example of how the structural stress/environmental interaction plays out involves body burden of lead. The literature on the harmful effects of lead stretches back for many decades. In the latter half of the 20th Century, effects of low exposures to lead on fetuses, babies and small children formed the basis of much regulation of air-borne, water-borne, and surface-borne lead (soil and painted surfaces). These publications depended on confidential medical data. Needleman et al (1996) studied a cohort of boys in Pittsburgh to document whether lead body burden associated with impulsivity and potential for violence. To the authors' surprise, lead body burden associated with these traits only in the African-American boys. The demographic data in the paper hint at different processes and stresses within the black and white populations in the study area: the black subcohort numbers remained fairly stable during the several years of study, but white numbers dwindled rapidly. White families moved out of the study zone but black families did not. The answer to the puzzle came, not from Needleman's team but from the social psychiatrist Mindy Fullilove in her 2005 book Root Shock which described urban renewal in several US cities and its impact on the affected communities and individuals. Pittsburgh was one of those cities. Additionally, the University of Pittsburgh, in the 1980's-1990's, bought up large tracts of "slums" and evicted the residents (personal communication, Mindy Fullilove, 2008). Pittsburgh and the University degraded and destabilized the so-called slum neighborhoods. The white families could flee but the black families had to shift around, even doubling up in the remaining housing, because of housing segregation practices and because of the need for survival to maintain community ties. Thus, Needleman, unknown to himself, was dealing with an afflicted black community and a disappearing white one. The dose/response pattern of lead body burden with impulsivity and potential for violence detected only in the black subcohort reflected the synergism of community destruction and lead body burden. Carol Stack (1974) documented how poor black families rely on a dense social network of women to survive. Ida Susser (2012) documented how poor white families rely on a dense social network of women to survive. Destruction of poor neighborhoods, especially deliberate destruction by public policies such as urban renewal and planned shrinkage (the withdrawal of essential services such as fire service and sanitation) shred these vital networks but have differing impacts on black and white families because of pervasive and illegal housing segregation maintained by such practices as racial steering and refusal to sell or rent to black families and individuals. Massey and Denton (1993) documented these practices in the late 1980's-early 1990's but they persist into the present. For example: the NYS Attorney General is now investigating racial steering by realty agents on Long Island (Newsday, 2019). Housing injustice, thus, imposes a broader and deeper structural stress on black than white communities, as well as the unraveling of social networks on which families rely for survival. It is but one of the many race-based disempowering outcomes of oppressive policies that erode the underlying health of individuals and communities of color. The bodies of residents of oppressed communities reflect the chronic and acute institutionalized violence directed against them and cannot cope with environmental exposures with the same efficacy that is shown by less-oppressed individuals and communities. Present and future regulations and policies on lead must rely on confidential medical data. The proposed rule would constitute yet another avenue of chronic and acute racial oppression. 
      Example 3. Traffic Fumes and Violence: Raised Incidence of Asthma. Rosalind Wright's team at Harvard studied a cohort of children in poor communities of Boston (Clougherty et al, 2007, full text attached). One important study result was that children exposed to both traffic fumes and violence had much higher incidence of asthma. Since 2007, similar studies in other venues have confirmed the Boston findings. The website Pubmed produces a long list of peer-reviewed publications when search terms are "asthma and violence". The list includes several papers documenting the greater incidence of asthma in the aftermath of the 9/11 World Trade Center attack among people exposed to the dust and suffering from PTSD. This 2007 paper is an example of one supported by consonant results from other studies.
The commenter (9455) contends that barriers to new regulations and policies imposed by the proposed rule would constitute environmental injustice by continuing unjust exposures that interact with socioeconomic factors to produce patterns of worse health and higher mortality rates in low-income communities and communities of color. 
Response: Based on a consideration of these and other similar comments, the EPA clarified in the final rule how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating significant regulatory actions or developing influential scientific information. As described in the preamble to the final rule, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information, consistent with existing Agency practice. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
As discussed more fully in the preamble of the final rule, the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The EPA believes that the approach described in the final rule will result in significant regulatory actions and influential scientific information based on the highest quality, most relevant studies and ensure adequate human health and environmental protections while maximizing transparency and maintaining consistency with the statutes the EPA administers.
Comment: Commenter (10749) states that, since this proposed rule will apply to all research that the EPA uses, many studies that have found strong associations between adverse health effects and environmental hazards would also be discarded, such as evidence that links mercury from power plants to impaired brain function and lead in paint dust to behavioral disorders in children. Commenter suggests that a consequent decrease in air quality standards would lead to more air pollution across the United States, resulting in a higher prevalence of asthma, respiratory diseases, and cancer, especially among children. The commenter states that these high cost-to-treat diseases will cause more people to seek out medical attention leading to higher rates of hospitalization and a higher burden on the healthcare system and hospitals.
Commenter (10865) states that, if the EPA excludes studies or diminishes the consideration of studies because the data cannot be made public, children and their families may be exposed to real harm. The commenter states that the resulting policy decisions affecting millions of children would be based on inadequate information that fails to include well-supported studies by expert scientists.
Commenter (12430) contends the SNPRM would allow public availability of data to supersede the evaluation of scientific quality by independent expert peer reviewers, to the detriment of public health in both routine regulatory actions and crises that rely on the federal government's use of science. The commenter states that it would reduce the consideration given to many important, high-quality studies (or preclude that consideration entirely) and could even distort the practices used to conduct certain studies in the first place.
Commenter (12707) asserts that the negative effects of the SNPRM on individual EPA programs would be far reaching and would undermine the EPA's ability to protect air and water quality across the Midwest. The commenter states that foundational studies about the impact of air pollution on public health are in the bullseye of this proposal --  indeed, they have been the target of legislative efforts to restrict Agency science for years. The commenter states that these studies have been reviewed numerous times and found to be sound and appropriate. The commenter states that, under the EPA's proposed rule, however, they could be out of bounds, compromising the Agency's ability to truly assess the impacts of air pollution and to set national air health standards at a level that will protect the public health, as required by the CAA. The commenter states that less protective standards will mean even more air pollution in our communities. 
Commenter (12707) contends that the elimination of these studies would skew the evaluation of costs and benefits, inappropriately minimizing the public health benefits. The commenter states that artificially suppressing the benefits will lead to less protective rules that will not be based on a true accounting of the cost to the public in terms of public health impacts.
Commenter (12720) states that the Proposal and Supplemental Proposal has clear adverse consequences for cost-benefit analyses that consider the substantial costs of health effects caused by exposure to air pollution. The commenter notes that this area of work includes efforts to address carbon dioxide (CO2) pollution and climate change, such as the Clean Power Plan. The commenter asserts that health and air quality-related monetized benefits from reducing PM2.5 pollution, a co-benefit of CO2 reductions, would be substantially reduced if EPA is unable to rely on the best available science for pollution-health impacts.
The commenter (12720) that in its proposed rule repealing the Clean Power Plan, EPA signaled this approach: the economic health benefits of PM2.5 reduction were zeroed-out by EPA after levels reached the current annual NAAQS (12 μg/m3), The commenter states that this approach of using the NAAQS or LML as a safe threshold directly contradicts the best available science, and EPA's own stance on the pollution threshold issue as recently as 2012, more than even the current EPA dose-response approach for PM2.5 exposure assumes.
Response: Based on a consideration of these and other similar comments, the EPA clarified in the final rule how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating significant regulatory actions or developing influential scientific information. As described in the preamble to the final rule, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information, consistent with existing Agency practice. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
The EPA disagrees that implementation of the rule would either eliminate key health studies or minimize public health benefits. The benefits of greater data transparency and the significance of reanalyzing and validating study results are well-documented in scientific literature. McNutt (2014) noted, "reproducibility, rigor, transparency, and independent verification are cornerstones of the scientific method." The National Academies of Sciences, Engineering, and Medicine (NAS) workshop on Reproducibility and Replicability in Science also noted that "certainly, reproducibility and replicability play an important role in achieving rigor and transparency." Munafò et al. (2017) state, "the credibility of scientific claims is rooted in the evidence supporting them, which includes the methodology applied, the data acquired, and the process of methodology implementation, data analysis and outcome interpretation. Claims become credible by the community reviewing, critiquing, extending and reproducing the supporting evidence. However, without transparency, claims only achieve credibility based on trust in the confidence or authority of the originator. Transparency is superior to trust." The 2019 NAS workshop on Reproducibility and Replicability in Science also concluded, "the scientific enterprise depends on the ability of the scientific community to scrutinize scientific claims and to gain confidence over time in results and inferences that have stood up to repeated testing."  The EPA agrees that data transparency is vital for individuals who have not contributed to the study to be able to verify the quality and strength of published studies and finds that the benefits of reanalysis are especially important for data that drive the quantitative requirements or analyses of EPA significant regulatory actions or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and, thus, increase the opportunity for independent subject matter experts to validate pivotal science, and as the dose-response data are better understood through independent reanalysis the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information. Further, the rule offers sufficient flexibility in terms of the considerations in 40 CFR 30.5, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data in order to minimize potential unintended consequences. 
As described in the preamble to the final rule, that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
As discussed more fully in the preamble of the final rule, the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The EPA believes that the approach described in the final rule will result in significant regulatory actions and influential scientific information based on the highest quality, most relevant studies and ensure adequate human health and environmental protections while maximizing transparency and maintaining consistency with the statutes the EPA administers.
2.2.2.4 EPA Programs
2.2.2.4.1 Need for Assessment of Impact on EPA Programs and Regulations
Comment: Commenter (11395) asserts that EPA did not evaluate the impact of the proposed rule on the existing regulations and programs that have been adopted to implement those laws. The commenter provides that EPA has not provided an analysis of how the requirements in this regulation would affect key studies that support the periodically updated IRIS risk assessments, which are used to support regulations adopted pursuant to several of those statutes and that it is irresponsible to proceed with adopting this rule without such an analysis.
Commenter (11894) contends that EPA has failed to address how it can refuse to apply these new requirements to data and models used to support Pre-manufacture Notices (PMNs) given express direction from Congress to the Agency to consider data quality. The commenter provides that EPA has not addressed the issue of applicability to this program at all, or addressed the impacts on the public health or the environment if it did. Furthermore, the commenter notes that EPA has not explored the resources that would be needed to respond.
Commenter (12403) contends that the proposed rule and the SNPRM do not provide meaningful assessments of the cost of implementing the rule or of the impact of the rule on existing regulations and programs. The commenter notes that on the other hand, a 2019 SAB draft report identifies a number of costly activities that implementation would require, including establishing an office for data-sharing, funding replication studies, promulgating guidance on methods and analysis, developing tiers of public access, and conducting data reanalysis. The commenter provides that these costs should be thoroughly assessed prior to finalization of the rule.
Response: The EPA agrees that neither the 2018 proposed rule nor the 2020 SNPRM included a characterization of costs or benefits of the proposed action. As discussed in the preamble to the final rule, this is a rule of internal procedure promulgated under the EPA's housekeeping authority and imposes no enforceable duty on any state, local, or tribal governments or the private sector. Additionally, this rule does not require the EPA or other parties to disclose or host data. Hence, this rule does not impose costs on the EPA or any other party to make such data available. The EPA recognizes there will be some incremental review and administrative costs to the Agency but anticipates these will be small. In addition, the Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule. 
The main benefits of this rule spring from greater transparency in significant regulatory actions and influential scientific information. By placing greater emphasis on the availability of dose-response data underlying pivotal science, the rule will allow for greater scientific scrutiny as EPA decision makers are developing significant regulatory actions and influential scientific information and increases the likelihood that any errors will be identified and corrected. Greater transparency is also inherently valuable as a principle of good government and provides benefits to the public at large, including reducing the risk of errors in EPA analyses and in the science such analyses rely upon. The ability for independent subject matter experts to validate pivotal science will facilitate more effective comment and engagement with the public during development of future significant regulatory actions and influential scientific information.
With this final rule, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
2.2.2.4.2 Superfund National Priorities List
Comment: Commenter (11894) discusses the Superfund National Priorities List (SNPL) and EPA's three-tier hierarchy to select human health toxicity values for use in risk assessments to guide those cleanups. The commenter notes that these Superfund risk assessments do not set absolute legally binding standards but rather presumptive levels. The commenter asserts that the methodology under which the assessments are conducted expressly recognize that EPA should take action to reduce public health risks using the best available science without waiting for further study to improve the certainty of the toxicity values. According to the commenter, insisting on certainty leads to delay, which has its own costs.
The commenter (11894) contends that if the final science transparency rule applies retroactively to every one of these toxicity values will have to be re-examined. The commenter states that if the human toxicity studies underlying these values fail to meet the new tests of transparency and reproducibility, the cleanup levels for air, water and soil based on these toxicity values will be discredited. Furthermore, the commenter provides that as a result, the cleanup actions underway at all 1335 Superfund sites will be undermined, as well as the cleanups completed at 424 sites that have been deleted from the SNPL. Meanwhile, the commenter notes that if the final science transparency rule applies only prospectively to new toxicity values, EPA will still need to revise the hierarchy of human health toxicity values to incorporate the new transparency and reproducibility requirements. The commenter asserts that EPA will incur significant costs and time replacing the hierarchy, the toxicity values, and the cleanup levels at many sites.
Commenter (12720) states that the expansion to "influential scientific information" would impact cleanup levels at the 1,335 sites on the SNPL. Should the human toxicity studies underlying toxicity values used at these sites be discredited due to the failure to meet the new requirements, the commenter asserts that cleanup actions at all sites will be undermined, as well as the cleanups at the 424 sites that have been deleted from the SNPL.
Response: This rule sets the overarching structure and principles for transparency of pivotal science in significant regulatory actions and influential scientific information. The final rule provides that if implementing the rule results in any conflict between this rule and the environmental statutes that the EPA administers, and their implementing regulations, this rule will yield and the statutes and regulations will be controlling. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods. The EPA is not applying this rulemaking to permit proceedings, site-specific actions, or enforcement actions because these actions are typically focused on individual regulated entities.
The EPA would also like to clarify that the final rule applies prospectively to significant regulatory actions and influential scientific information; the final rule has no retrospective effect on either significant regulatory actions or influential scientific information.
2.2.2.4.3 Safe Drinking Water Act Programs
Comment: Commenters (11894, 12720) provide that EPA's proposal will affect two major elements of EPA's SDWA program: drinking water health advisories and human health benchmarks for pesticides in drinking water. According to the commenters, neither the advisories nor the benchmarks are enforceable, but water utilities often use them when their source water is contaminated with a pollutant for which there is no federal drinking water standard. The commenters contend that if the final science transparency rule applies retroactively to influential scientific information, every one of these advisories and benchmarks will have to be re-examined. The commenters maintain that if the human and/or animal toxicity studies underlying these advisories and benchmarks fail to meet the new tests of transparency and reproducibility, the treatment provided by utilities for these pollutants will be discredited. The commenters also provide that if the final science transparency rule applies only prospectively to new advisories and benchmarks, EPA will still need to replace the current methodologies for these recommended values in order to incorporate the new transparency and reproducibility requirements. According to the commenters, EPA will incur significant costs and time replacing these methodologies and recommendations.
Response: This rule sets the overarching structure and principles for transparency of pivotal science in significant regulatory actions and influential scientific information. The final rule provides that if implementing the rule results in any conflict between this rule and the environmental statutes that the EPA administers, and their implementing regulations, this rule will yield and the statutes and regulations will be controlling. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods. The EPA is not applying this rulemaking to permit proceedings, site-specific actions, or enforcement actions because these actions are typically focused on individual regulated entities.
Comment: Commenter (6901) states that federal regulations are the basis for the actions of state drinking water programs in protecting public health. The commenter contends that the SDWA has clear statutory language that requires regulations to be based on sound science to appropriately protect public health. According to the commenter, while states may disagree at times with details of the final regulations, states are generally comfortable with the transparency of the regulatory development process as practiced by the Office of Ground Water and Drinking Water (OGWDW). The commenter does not recommend making significant changes in that process. In fact, the commenter suggests that, if other environmental programs do not currently have a robust science-based regulatory development process, the process used by OGWDW would be a good model.
Commenter (6901) states that the SDWA statutory language in §1412(b)(3) requires the use of "best peer reviewed science and supporting studies conducted in accordance with sound and objective scientific practices." Therefore, according to the commenter, a statutory requirement that the drinking water regulations have a strong scientific basis already exists. The commenter states that this section of the SDWA also outlines the way this information must be shared with the public, so the process is currently transparent. The commenter provides that the EPA has followed this statutory mandate since the 1996 SDWA Amendments and has relied not only on peer reviewed scientific studies but has also directly involved the scientific community in supporting rule development. The commenter adds that states have also been active participants in this science-based process. The commenter suggests that the rule development currently underway for perchlorate is a good example of the process at work. According to the commenter, recommendations from the SAB helped guide the methodology to develop the Maximum Contaminant Level Goal (MCLG) and the EPA has held two peer reviews to help refine the models and determine how best to apply the modeling to determine the appropriate MCLG. The commenter states that the SAB and the peer review process are all open and the recommendations are public, providing full transparency to the deliberations and decisions.
Response: This rule sets the overarching structure and principles for transparency of pivotal science in significant regulatory actions and influential scientific information. The final rule provides that if implementing the rule results in any conflict between this rule and the environmental statutes that the EPA administers, and their implementing regulations, this rule will yield and the statutes and regulations will be controlling.The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. The specific criteria for determining "pivotal science" may necessarily be specific to the authorizing statute, as well as the significant regulatory action or the influential scientific information. The EPA intends to explain in each significant regulatory action and for influential scientific information how the pivotal studies were identified
2.2.2.4.4 Aquatic Life Criteria
Comment: Commenter (11894) notes that aquatic life criteria have been published for 47 toxic chemicals. The commenter provides that these criteria are designed to protect plant and animal life in surface water bodies from both short and long-term exposures to pollutants. Additionally, the commenter states that human health criteria have been published for 120 toxic chemicals, two pathogen indicators, and two harmful algal bloom toxins. The commenter contends that if the final science transparency rule applies retroactively, every one of these criteria will have to be re-examined. The commenter asserts that if the studies and bioaccumulation models underlying any of these criteria fail to meet the new tests of transparency and reproducibility, the enforceability of state water quality standards will be undermined since most such standards are based on the EPA criteria. According to the commenter, if the final science transparency rule applies only prospectively to new criteria, EPA will still need to replace the current methodologies for both aquatic life and human health criteria in order to incorporate the new transparency and reproducibility requirements. The commenter states that EPA will incur significant costs and time replacing these methodologies and water quality criteria. Despite this major impact, the supplemental notice provides no estimation of the costs and benefits.
Response: As further described in the preamble to the final rule, based on public comments and other considerations, the EPA is concentrating its efforts in the final rule to increase transparency of dose-response data, as the dose-response data are discrete and the dose-response assessment is a well-defined and impactful step in the quantitative assessment of risk. The rule does not apply to dose-response models or other types of data and models, including the bioaccumulation 
Further, as discussed in the preamble to the final rule, the final rule applies prospectively to significant regulatory actions and influential scientific information; the final rule has no retrospective effect on either significant regulatory actions or influential scientific information. Therefore, existing water quality criteria would not be impacted by this rule. 
This rule sets the overarching structure and principles for transparency of pivotal science in significant regulatory actions and influential scientific information. The final rule provides that if implementing the rule results in any conflict between this rule and the environmental statutes that the EPA administers, and their implementing regulations, this rule will yield and the statutes and regulations will be controlling. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods. The EPA is not applying this rulemaking to permit proceedings, site-specific actions, or enforcement actions because these actions are typically focused on individual regulated entities.
The EPA would also like to clarify that the final rule applies prospectively to significant regulatory actions and influential scientific information; the final rule has no retrospective effect on either significant regulatory actions or influential scientific information.
2.2.2.4.5 Clean Water Act (CWA)
Comment: Commenter (12720) provides that the CWA requires each state to develop TMDLs for every waterbody the state identifies as having uses impaired because of pollution. According to the commenter, if EPA disapproves a state TMDL, EPA must develop a replacement TMDL. The commenter contends that if the final science transparency rule applies retroactively, over 65,000 TMDLs will have to be re-examined.
The commenter (12720) contends that if the model underlying a TMDL fails to meet the new tests of transparency and reproducibility, the enforceability of the state's point source permit limits and nonpoint source controls based on that TMDL will be undermined. The commenter states that if the science transparency rule applies only prospectively to new TMDLs, EPA will need to disapprove any new TMDL based on a proprietary model since such a model would not meet the new transparency and reproducibility requirements. The commenter provides that whether the science transparency rule applies retroactively or prospectively, EPA will incur significant costs and time replacing TMDLs.
Response: As further described in the preamble to the final rule, based on public comments and other considerations, the EPA is concentrating its efforts in the final rule to increase transparency of dose-response data, as the dose-response data are discrete and the dose-response assessment is a well-defined and impactful step in the quantitative assessment of risk. The rule does not apply to dose-response models or other types of data and models.
The water quality criteria that support TMDLs may be applicable to this rule if they are classified as influential scientific information. As discussed in the preamble to the final rule, the final rule applies prospectively to significant regulatory actions and influential scientific information; the final rule has no retrospective effect on either significant regulatory actions or influential scientific information. Therefore, existing water quality criteria would not be impacted by this rule. 
This rule sets the overarching structure and principles for transparency of pivotal science in significant regulatory actions and influential scientific information. The final rule provides that if implementing the rule results in any conflict between this rule and the environmental statutes that the EPA administers, and their implementing regulations, this rule will yield and the statutes and regulations will be controlling. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods. The EPA is not applying this rulemaking to permit proceedings, site-specific actions, or enforcement actions because these actions are typically focused on individual regulated entities.
2.2.2.4.6 NAAQS
Comment: Commenter (11479) asserts that the coverage of influential scientific information would be problematic for EPA programs with reoccurring rulemakings, such as NAAQS, because it would prevent adequate implementation of the review.
Commenter (12720) contends that many scientific interpretation activities at EPA, including legally-mandated reviews of the NAAQS, are projects that build upon a growing scientific evidence base. The commenter provides that for example, the recently-completed ISA and Policy Assessment for Fine Particulate Matter rely on scientific data and models and other information previously synthesized by the Agency for prior reviews. The commenter notes that in this case, the Agency is consulting thousands of peer-reviewed studies that already meet significant requirements established by EPA.
Response: Based on a consideration of these and other similar comments, the EPA clarified in the final rule how pivotal science would be selected. As discussed in the preamble to the final rule, rather than considering all studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact as "pivotal science," the EPA is balancing transparency and feasibility by focusing on those studies that inform the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect. Thus, "pivotal science" includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). These are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. In the context of this rule, studies informing the dose-response modeling (e.g., benchmark response selection), studies identifying data for toxicokinetic adjustments, or studies informing the selection of uncertainty factors do not drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information.
In addition, with this final rule, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
Comment: Commenter (12727) states that some actions EPA intends to cover with this rule, such as ISAs that inform NAAQS, are based on an assessment of overall confidence in an at-risk factor (i.e., a factor that potentially increases the risk of air-pollutant related health effects for individuals in certain subpopulations), and excluding one or more studies showing a relationship between that factor and adverse health effects could tip the balance toward, for example, recommending a change in the NAAQS. Thus, the commenter asserts that arbitrarily ignoring rigorous studies at one or multiple stages of the regulatory process could alter critical health and environmental protections and would be illegal.
Response: As described in the preamble to the final rule, the rule establishes how the EPA will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information. The rule does not provide for categorically excluding or ignoring studies. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
In addition, with this final rule, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
2.2.2.4.7 Toxic Substances Control Act Programs
Comment: Commenter (11894) states that TSCA §4 gives EPA authority to require chemical manufacturers to test their product for several reasons, one of which is that "there is insufficient information or experience to determine or predict the effects of these activities." The commenter contends that the new data availability requirement would logically imply that in assessing the "sufficiency" of effects information on a chemical, EPA will have to avoid reliance on studies without data disclosure, and instead order manufacturers to conduct new and conforming studies to replace the studies that do not meet the disclosure standards. The commenter queries whether EPA is legally obliged to apply its own view of high standards of data quality to the implementation of a statute for which Congress expressly made data quality a central implementation concern, and if not, why not?
Response: In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. 
Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science is unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation.
In addition, the EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods. The EPA is not applying this rulemaking to permit proceedings, site-specific actions, or enforcement actions because these actions are typically focused on individual regulated entities. The provisions of the final rule also do not apply to physical objects (like laboratory samples), drafts, and preliminary analyses, and influential scientific information and/or pivotal science that meet one or more of the exemptions identified in Section IX of the OMB Final Information Quality Bulletin for Peer Review.
2.2.2.4.8 Pesticide Restrictions and Approvals
Comment: Commenter (11576) states that the SNPRM lists environmental-fate studies and models as an example of the types of studies included in its new, broader scope. The commenter states that these data and models are critical for the EPA's assessments of the environmental risks of pesticides, for example, and the Agency has relied on well-established studies that were conducted before mechanisms existed for sharing data publicly. The commenter states that this requirement would place an uneven burden on data requirements for pesticide restrictions compared to that required for pesticide approvals, especially if implemented retroactively.
Response: Based on comments and other considerations, the EPA is concentrating its efforts in the final rule to increase transparency on dose-response data, as further described in the preamble to the final rule. The EPA is maintaining the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods. The EPA is not applying this rulemaking to permit proceedings, site-specific actions, or enforcement actions because these actions are typically focused on individual regulated entities. The provisions of the final rule also do not apply to physical objects (like laboratory samples), drafts, and preliminary analyses, and influential scientific information and/or pivotal science that meet one or more of the exemptions identified in Section IX of the OMB Final Information Quality Bulletin for Peer Review.
With this final rule, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
2.2.2.5 General Delay
Comment: Commenter (11360) contends that the current COVID-19 pandemic is showing the necessity of scientific expertise in decision-making. The commenter states that the speed of the release of the virus' genetic code was unheard of, and that making that information available to other researchers who wanted to study it, unleashed a massive collaborative effort to understand the mysterious new pathogen that rapidly spread across the world. The commenter provides that if EPA's proposed transparency rule was in place at that moment, it seems like there would be a need to determine a) whether this type of information sharing would meet the rule's requirements and/or b) whether they require independent validation or a case-by case exemption from the rule's applicability before they could be used for EPA decision-making purposes. The commenter asserts that even if these and other relevant studies were ultimately deemed to be eligible for use under this rule, this proposed rule would clearly establish barriers and result in fatal delays to using the best available science to inform EPA's actions.
Response: The EPA considered this and other similar comments when developing the final rule and does not believe that the final rule will result in any meaningful delay in promulgating regulations because of the limited application of this rule to pivotal science underlying significant regulatory actions and influential scientific information. Additionally, this rule does not require the EPA to disclose or host data, but to determine if the dose-response data underlying pivotal science are available and to give greater consideration to pivotal science for which dose-response data are available in a manner sufficient for independent validation.
Comment: Commenter (11408) states that the rulemaking would likely also be costly in terms of the forgone benefits that would occur as a result of the additional regulatory delays that it would create. The commenter contends that the "cost" of one year's delay of some of the EPA's biggest CAA rules could be measured in thousands of premature deaths and tens of thousands of asthma attacks and missed school and work days. The commenter asserts that as legal scholars across the political spectrum recognize, the rulemaking process has become extraordinarily slow, due in large part to the combination of agencies' dwindling resources and the proliferation of additional procedural and analytical requirements that agencies must satisfy before issuing new rules. The commenter contends that as a result, it already can take anywhere from five to ten years for the most complex rules to complete the rulemaking process. The commenter provides that in 2012, the Government Accountability Office found that OSHA spent an average of seven years on each of its major rulemakings. 
Further, the commenter (11408) contends that implementation of this rulemaking would only exacerbate this problem, leading to even more costly delays that harm public health, safety, financial security, and the environment. The commenter recommends that to better fulfill its mission of protecting public health and the environment, the EPA should be exploring opportunities to streamline its processes for issuing new safeguards  -  not making the problem of alarming delays even worse.
Commenter (11495) notes that in an era of limited EPA funding, the expanded scope of the supplement potentially would overrun the Agency's capacity to review data and the studies relevant to any review process. The commenter provides that just handling these challenges for a single Agency action, such as reviewing the Particulate Matter ("PM") NAAQS with its over 2,000 references, would be an enormous undertaking and it begs the question whether any newly adopted system hosted by the EPA can handle what it is requesting in the Supplemental Notice.
Commenter (11890) maintains that the cost of delays for EPA programs should be considered. For example, the commenter provides that it would take considerable staff time to track down data owners and obtain consent from all studies included in a typical IRIS review. The commenter questions several aspects of this process: Will all study authors need to consent? If the study was funded by a consortium, will all participating companies need to consent? What about companies that have since been purchased by others? Will the EPA be asking international and privately funded researchers to provide data access only when a specific requester submits a request or must they provide data to a data repository to comply? The commenter states that this rule will likely further deteriorate the EPA's ability to comply with statutory reassessment deadlines in a number of programs, causing delays that keep EPA from updating their assessments and meeting its mission of protecting human health and the environment.
Response: The EPA considered this and other similar comments when developing the final rule and does not believe that the final rule will result in any meaningful delay in promulgating regulations because of the limited application of this rule to pivotal science underlying significant regulatory actions and influential scientific information. Additionally, this rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which dose-response data are available in a manner sufficient for independent validation.
Comment: Commenter (12720) highlights concerns of Senator Thomas Carper's letter to the EPA Administrator that explains that the Supplemental Proposal's "time-consuming barriers to the use of scientific information could in some cases be more than a mere annoyance or ministerial task; it could be fatal" because it could significantly delay EPA's ability to make timely use of critical public health research. The letter notes that
      If EPA's rule was in place, before these studies could be used by EPA decision-makers there would, at minimum, be a time-consuming review needed in order to determine a) whether they meet the rule's requirements and/or b) whether they require independent validation or a case-by case exemption from the rule's applicability before they could be used for EPA decision-making purposes. Even if these and other relevant studies were ultimately deemed to be eligible for use under this rule, this rule would clearly establish barriers and delays to using available science to inform EPA's response.
Within the text of the Supplemental Proposal, the commenter (12720) provides that EPA has not explained what safeguards are in place so that new public data release requirements do not inhibit EPA's response to public health emergencies. The letter also notes that
      it is also possible that studies that could be usefully relied upon during a pandemic or other crisis would be systematically excluded from being used in EPA's scientific and regulatory efforts well before the next pandemic or other crisis occurs if this rule is finalized.
The commenter (12720) notes that the Supplemental Proposal includes a provision that gives the EPA Administrator the ultimate authority to decide which studies are exempt from new public data release requirements. As applied to the current crisis, the commenter contends that the Supplemental Proposal's requirements for public data release conflict with the Executive Branch's approach to releasing data and models within the context of the current COVID-19 response. For example, the commenter provides that a White House representative recently told reporters that the COVID-19 task force "has not publicly released the models it drew from out of respect for the confidentiality of the modelers, many of whom approached the White House unsolicited and simply want to continue their work without publicity."
The commenter (12720) contends that EPA runs the risk of discouraging researchers from working with the Agency to quickly and safely meet the challenges presented by the COVID pandemic. Additionally, the commenter states that EPA's failure to explain how the Supplemental Proposal would not obstruct the Agency in responding to emergencies is further evidence that the Supplemental Proposal is arbitrary and capricious, an abuse of discretion, and otherwise unlawful.
Response: The EPA considered this and other similar comments when developing the final rule and does not believe that the final rule will result in any meaningful delay in promulgating regulations because of the limited application of this rule to pivotal science underlying significant regulatory actions and influential scientific information. Additionally, this rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which dose-response data are available in a manner sufficient for independent validation.
Finally, the EPA acknowledges that the COVID-19 situation is unprecedented. The EPA's most important environmental statutes provide the Agency with the authority to issue emergency orders or respond to address emergencies to protect human health and the environment. This final rule would not limit or impede the EPA's authority to undertake such responses. Moreover, the final rule includes sufficient flexibility in terms of the considerations in 40 CFR 30.5, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data in order to minimize potential unintended consequences, such as a potential impact during a crisis situation. In addition, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. The EPA believes the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making.
Comment: Commenter (12692) contends that the independent validation of all data used in hundreds of decisions regarding permits and product approvals would be quite impracticable, bringing the daily work of the Agency to a screeching halt, and imposing unreasonable burdens on the private sector. The commenter notes that this "lack of consistency" in application is valid due to the difference in scope, number, and consequence. The commenter provides that if consistent application across all actions was sound logic, all NEPA proceedings would require full-blown environmental impact statements. According to the commenter, there would be no provision for shorter environmental assessments, much less categorical exclusions for actions an Agency has determined have no significant impact on the human environment.
Response: The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods. The EPA is not applying this rulemaking to permit proceedings, site-specific actions, or enforcement actions because these actions are typically focused on individual regulated entities. The provisions of the final rule also do not apply to physical objects (like laboratory samples), drafts, and preliminary analyses, and influential scientific information and/or pivotal science that meet one or more of the exemptions identified in Section IX of the OMB Final Information Quality Bulletin for Peer Review.
With this final rule, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
Comment: Commenter (10045) states that, even if relevant studies were ultimately deemed to be eligible for use under this rule, this rule would clearly establish barriers and delays to using available science to inform the EPA's response. The commenter states that, if the EPA's rule were in place, before these studies could be used by EPA decision-makers there would, at minimum, be a time-consuming review needed in order to determine a) whether they meet the rule's requirements and/or b) whether they require independent validation or a case-by case exemption from the rule's applicability before they could be used for EPA decision-making purposes. 
Commenter (10742) expresses concern that the EPA has not analyzed whether the additional time needed to upload information to this database would delay rulemaking, which, in turn, would allow additional pollution to continue, potentially costing years of human life. 
Commenter (11408) states that having the EPA acquire a collection of raw data would be burdensome to both the Agency and study investigators, and create major delays in rulemaking. 
Commenter (11576) has concerns with respect to the practicability of requiring all information to be made available, given unequivocal statutory deadlines embedded within environmental statues. 
Commenter (12412) suggests that, by not using the best available science, the SNPRM may delay regulations on emerging contaminants, such as PFAS. The commenter states that, with the exclusion of the best available science, regulations like this will likely be delayed much longer, due to the need for more studies which have all data publicly available, and may not be accurate, due to the exclusion of a portion of the best available science. The commenter states that this delay in proposing regulations could harm public health in the future, especially in communities, such as tribes, which are disproportionately exposed to environmental contaminants. 
Commenter (12430) asserts that, if a team of researchers investigated and published in a top medical journal, following a rigorous peer-review process, urgent information about the health effects of lead, the SNPRM could preclude or delay EPA consideration of this information. The commenter states that, if the study relied on extensive exposure analyses with county-level data on household wipe samples and vital signs in populations of concern, the SNPRM would prohibit or undermine EPA consideration of the study until the time-consuming process of redacting PII from the raw data was complete, if even possible. The commenter states that, further, another team of scientists could publish an inferior study without county-level or demographic data, albeit with underlying data made available, that does not help state and local officials identify hot-spots based on county results or populations of concern. The commenter states that, depending on the proposed alternative, the SNPRM would either favor the inferior study or prohibit EPA experts from considering the superior study. The commenter states that such an outcome would fall short of the Agency's statutory duties and would adversely affect public health. 
Response: The EPA clarified 40 CFR 30.5 that it will give equal consideration to dose-response data that are publicly available or available through restricted access. As discussed in the final rule, underlying dose-response data are considered available if they can be accessed through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. Therefore, a redaction of the PII in the hypothetical example may not be necessary in order for the study to be fully considered as the EPA promulgates significant regulatory actions or develops influential scientific information. The EPA would like to emphasize that such behavior would be purely voluntary on behalf of the researchers; the rule does not regulate the rights and obligations of any party outside of the EPA let alone have legal force and effect on them.
Further, although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science is unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7.
Further, the EPA considered this and other similar comments when developing the final rule and does not believe that the final rule will result in any meaningful delay in promulgating regulations because of the limited application of this rule to pivotal science underlying significant regulatory actions and influential scientific information. Additionally, this rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which dose-response data are available in a manner sufficient for independent validation.
The EPA does not agree that this rulemaking is counter to the Agency's mission. The EPA believes that the approach described in the final rule will result in significant regulatory actions and influential scientific information based on the highest quality, most relevant studies and ensure adequate human health and environmental protections while maximizing transparency and maintaining consistency with the statutes the EPA administers.
Comment: Commenters (11400, 11480, 12546) express concern that the proposal could result in substantial delays as studies are subjected raw data availability, reanalyzes, and independent validations. 
Commenter (11400) asserts that this rule would make the process of seeking to use well established studies onerous and lengthy, as persons are forced to re-analyze data and apply for consideration and access. The commenter states that this rule would likely result in a lengthy, bureaucratic process to use epidemiological studies. The commenter states that, following the suggested model for tiered access, researchers would have to submit a research proposal outlining the need for restricted use data. The commenter states that the government is still developing standard data repositories. The commenter states that studies would have to be reviewed for meeting the criteria; researchers would need to apply to have the study considered or exempted; interested parties would have to redact information or see if it could be reanalyzed and validated. The commenter states that these are burdensome and time-consuming requirements, that would keep scientists from evaluating and relying on relevant studies. The commenter states that this rule will impede timely actions to protect public and environmental health.
Commenter (11480) states that tactics to demand access to raw data and selective reanalysis is a key part of the strategy that corporations from the tobacco, diesel, chemical, and pharmaceutical industries, among others, have used for decades to question the science underlying regulation and other information disseminated by government agencies (regulatory and non-regulatory). The commenter states that lobbyists have attempted many legal maneuvers to formalize processes for raw data availability and reanalysis to undermining science involved in rulemaking over the decades, from the Shelby Amendment in Fiscal Year (FY) 1999 omnibus appropriations act to this SNPRM.
Commenter (12546) objects to the proposal's statement that "approaches to increasing access to data and models can often allow stakeholders to reanalyze the data and models and explore the sensitivity of the conclusions to alternative assumptions." The commenter asserts that this sets a dangerous precedent, opening the policy and rulemaking processes to potentially endless debate via "reanalysis", "alternative models", and "independent validation" by stakeholders who may have a vested financial interest (and capability) to impact or perpetually delay critical policies to protect public and environmental health. 
Commenter (12715) states that, although data may technically be "available," it is likely to be difficult to obtain and often may not be received by the requestor until after the comment period ends, if at all. The commenter states that, if this is the case, it is unlikely that interested parties would be able to obtain the information in a manner timely enough to allow for their own review and comment within any timeframe designated by EPA. 
Response: The EPA considered this and other similar comments when developing the final rule and does not believe that the final rule will result in any meaningful delay in promulgating regulations because of the limited application of this rule to pivotal science underlying significant regulatory actions and influential scientific information. Additionally, this rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which dose-response data are available in a manner sufficient for independent validation.
The EPA disagrees that implementation of the rule would result in a lengthy process in order to use epidemiology data. The final rule includes sufficient flexibility in terms of the allowance for restricted access, considerations in 40 CFR 30.5, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data to minimize such potential unintended consequences. The EPA recognizes there will be some incremental review and administrative costs to the Agency but anticipates these will be small.
2.2.2.6 Research Impacts
2.2.2.6.1 Burdens on Researchers Outside EPA
Comment: Commenters (10974, 11377, 11404, 11495, 12422, 12718, 12727) assert that the rulemaking will impose substantial burdens on researchers and that the supplemental notice adds to those burdens. According to the commenters, these burdens may result in the reduction of future studies necessary for policymaking due to the perceived difficulty of adjusting study designs to meet the new requirements with constrained resources. Additional comments and responses to comments on monetary costs to researchers are included in section 14.2.4.3 of Chapter 14. Examples of specific comments follow:
Commenter (11377) states the expansion of scope increases the already substantial burden that this rule would place on researchers to ensure the privacy and confidentiality of research subjects by including influential information rather than only regulatory decision-making information.
Commenter (11495) takes issue with the EPA's position that the proposed policy "exclusively pertains to the internal practices of EPA." The commenter contends that if implemented, the proposed policy would impact a wide range of external stakeholders, including scientists, research volunteers who participate in environmental health studies and the general public.
The commenter (11495) states that while the specific mechanisms regarding how this rule would be carried out has not been described in either the original proposed rule or this latest supplement, it is clear that many of the burdens of this Supplemental Notice would fall to the individual investigators and their institutions. The commenter contends that at a minimum, there would be a shared responsibility between individual researchers and EPA staff to carry out the requirements of the Supplemental Notice. The commenter provides that given that the burden of these new requirements far exceeds the EPA's capacity, most of these new responsibilities imposed by the Supplemental Notice would necessarily fall on individual researchers outside of the Agency to complete.
Additionally, the commenter (11495) asserts that contrary to the EPA's assertions in the supplement, this proposed rule would dictate the conduct of scientists outside the federal government and would place burdens on individual researchers that would go beyond just the time and resources required to adhere to these new requirements. The commenter states that if promulgated, the proposed rule would place an enormous strain on researchers as they decide whether they are able to comply with the rule. The commenter provides that on one hand, if a scientist does not share the data out of concern for violating consent agreements and breaching subject privacy, then the EPA will give less weight to the study in policymaking; on the other, if a scientist inadvertently discloses information in a way that violates data use agreements or contractual obligations, they may leave themselves open to legal action, dismissal, or have restrictions placed on their ability to conduct research in the future. The commenter notes that this is not an easy decision for scientists committed to generating and disseminating knowledge useful for the public good. 
Commenter (11495) states that it is possible that fewer scientific studies will be performed and made available to the scientific community due to an awareness by potential study participants or researchers that underlying data will be released to the public, potentially subjecting them to privacy intrusions or harassment by regulated industry.
Commenter (12405) asserts that, if the EPA's rule takes effect, it could introduce selection bias that may slow studies and alter results and thereby affect regulatory decisions. The commenter states that large-scale population studies rely on many people  --  often numbering in the thousands  --  to reveal sensitive or private information. The commenter states that studies may have difficulty recruiting or retaining volunteers if the researchers are required to make deidentified data publicly available as some may be more hesitant to share their information. The commenter states that those who are willing to participate may be different from others in ways that cannot currently be predicted or described, which could introduce confounding variables and biases that may question the study's results. 
Commenter (12422) provides that these new strictures will be particularly burdensome to the environmental health research community, ironically imposing substantial new costs during a time of drastic proposed research funding cuts. The commenter notes that epidemiologic research will be particularly challenged. According to the commenter, epidemiology is the foundation of public health, providing essential evidence of the distribution of disease, risk factors, and vulnerable populations. The commenter contends that while it remains to be determined how this historical evidence would be considered under this proposed rule, the implications are ominous for public health measures that have improved the health of millions of Americans.
The commenter (12422) asserts that many in the scientific and regulatory communities recognize the importance of transparency and access to the data underlying important studies. The commenter states that addressing these issues can be accomplished without injuring the scientific enterprise or discouraging scientists from pursuing pressing scientific questions. The commenter provides that mechanisms to accomplish this that do not require new regulation include development of public access databases, use of the FOIA, and reanalysis by third parties. The commenter notes that certain journals, for example require authors to agree to make their data available to editors and others upon request.
Commenter (11404) states that the concepts in the SNPRM would negatively impact future studies. According to the commenter, this proposed regulation could have the unintended consequence of tainting the epidemiological research process. The commenter notes that if this proposal were adopted, epidemiological researchers may feel pressured to modify their research processes to meet the additional standards imposed by the rule, so as to secure federal funding and / or ensure the research can be used in regulatory development. The commenter notes that giving "greater consideration" to studies that are able to make their data available for independent validation, via the SNPRM's alternative approach, would promote this as well. The commenter maintains that the risk would be a less-than-ideal research plan and less robust results.
Commenter (12718) contends that the application of restrictions for data not funded by government, even when such information can be useful for decision-making, is peculiar. The commenter provides that without grant-making powers or resources to support the scientific community or the provision of incentives for shifting expectations and requirements, there is only one logical conclusion about the impact of the regulation in practice  -  instead of transparency, the rule would likely discourage evidence-based policymaking.
Response: The EPA disagrees that this rule would impose costs on third-party researchers. This procedural rule does not regulate the conduct or determine the rights or obligations of any entity outside the federal government. This is a rule of internal procedure that does not impose requirements on any party other than the EPA. This rule imposes no costs on researchers or their institutions, and the EPA will consider and evaluate all relevant and appropriate science in its significant regulatory actions and influential scientific information. The EPA recognizes that researchers and other third parties may voluntarily consider the EPA's position on data availability, as described in this rule, as they make their own decisions about how to conduct research and the extent to which they make data and models available. Researchers may choose to make more data and models available, but the EPA recognizes that these parties will weigh their own benefits and costs and make choices that they deem appropriate.
The EPA agrees with the commenter that data sharing mechanisms already exist; however the rule is not intended to develop public access databases or encourage authors to share their data, as the commenter suggests. The purpose of the rule is to increase transparency by codifying the EPA's own internal procedures for determining the consideration to afford pivotal science according to the availability of underlying dose-response data. The preamble to the rule clarifies that restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. The rule does not prescribe a particular mechanism but does require the EPA to independently verify that such a mechanism exists for dose-response data underlying pivotal science. As discussed in the preamble to the final rule, the EPA cannot rely on data access statements made by authors to journals. Instead, in order to realize the benefits of open data, the EPA must conduct an independent evaluation of the availability of the data.
Comment: Commenter (10974) states that EPA, in this SNPRM, claims that the proposed rulemaking "does not regulate any entity outside the Federal Government...", and that "the [a]gency recognizes that any entity interested in EPA's regulations may be interested in this proposal." The commenter provides that although this language has not changed from the 2018 NPRM, the commenter still finds that this claim is incorrect, appearing purposefully deceptive. The commenter provides that at the least, this rule would impact how EPA scientists and policy makers are able to consider their own research; at worst, it would impact how epidemiological (and all other) research would be conducted in the future, thereby altering the entire environmental health science and policy landscape.
Objecting to both proposed alternatives at 40 CFR § 30.5, the commenter (10974) states that the EPA Administrator is inappropriately given carte blanche authority to judge research on a case-by-case basis. The commenter notes that when promulgating new standards or rules, EPA is expected to incorporate all available research into their findings  -  even when research results are contradictory, and/or demonstrate a broad range of possible environmental or health outcomes. According to the commenter, in either proposed version of 40 CFR § 30.5, the EPA Administrator could make the decision about what research is "worthy" of use based solely on the availability of underlying data. The commenter states that this decision would undermine, even ignore, factors that are more important for determining the validity of research  -  for example, how well a study was controlled, the power in the results used to make the conclusions, or the relevance of the study variables to the question being investigated.
Furthermore, the commenter (10974) asserts that the proposed change in the criteria for scientific studies' acceptance into decision making at 40 CFR § 30.5 would change how academia, private companies, state, local, and tribal governments, and others must conduct research in order for it to be considered relevant and useable by EPA. The commenter maintains that such a change would undoubtedly also impact how study participants perceive and ultimately decide to participate, or not, in environmental health research. Additionally, the commenter states that research would have to be conducted differently, and that is not merely an internal change in EPA processes and procedures.
Response: As discussed in the preamble to the final rule, the EPA is retaining the Administrator's exemption provision with some added considerations in response to public comments. To ensure that the Administrator's decision is appropriately transparent, in the final rule the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. This documentation would typically be provided as part of the proposed rulemaking given this will be part of the decision of what is the pivotal science for the rule. Regardless, the final rulemaking will provide clear documentation. The EPA is confident that the criteria provide sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
Further, this procedural rule does not regulate the conduct or determine the rights or obligations of any entity outside the federal government. The EPA disagrees that this rule would impose costs on third-party researchers. This is a rule of internal procedure that does not impose requirements on any party other than the EPA. This rule imposes no costs on researchers or their institutions, and the EPA will consider and evaluate all relevant and appropriate science in its significant regulatory actions and influential scientific information. The EPA recognizes that researchers and other third parties may voluntarily consider the EPA's position on data availability, as described in this rule, as they make their own decisions about how to conduct research and the extent to which they make data and models available. Researchers may choose to make more data and models available, but the EPA recognizes that these parties will weigh their own benefits and costs and make choices that they deem appropriate.
Comment: Commenter (12727) asserts that the costs that this proposal would impose on the research community would likely be significant and pervasive, thoroughly impacting the scientific process. According to the commenter, these costs are even higher than those that would have been imposed under the 2018 proposal because of the wider range of studies and EPA actions encompassed by the supplemental proposal. The commenter states that EPA cannot disregard these costs by claiming that the supplemental proposal does not directly impose requirements upon the research community. The commenter contends that researchers performing public health and environmental studies have an ethical, professional, and personal stake in how their studies are utilized. The commenter provides that it is untenable for EPA to ignore the impacts that this proposal would have on the research community, as if scientists would -- or even could -- conduct their research in a vacuum, with complete indifference to whether their work would be deemed unsuitable for consideration by the nation's top environmental regulator. Some of the specific issues/concerns highlighted by the commenter follow:
Commenter (12727) asserts that attempting to satisfy the rulemaking's parameters would be costly and convoluted for researchers. The commenter notes that the recent SAB report reveals several contexts in which costs for researchers could increase, including "researchers' time to collate data and work with EPA to make these data publicly available," and likely "additional costs that occur at an institutional level (i.e., IRBs) that would be substantial." Furthermore, the commenter provides that "there are legitimate legal, ethical, professional and financial reasons why researchers may be unable or unwilling to fully share `data.'" The commenter states that individual members of the SAB have also observed that study subjects may decline to participate in studies if researchers may ultimately share their data with others, raising serious concerns that researchers who attempt to comply with the requirements of this proposal will have difficulty persuading members of the public to participate in studies, or that participation in such studies may be biased or skewed in ways that would influence the results. The commenter contends that this proposal lacks any discernible benefit, so researchers' efforts to meet EPA's arbitrary strictures would be superfluous to any measures taken to ensure scientific rigor and valid, evidence-based outcomes; such efforts would increase costs and administrative burdens but provide almost nothing in return except checking off EPA's irrational requirements.
The commenter (12727) contends that researchers unable to obtain additional funding would have two options: (1) they could continue their study without meeting EPA's requirements (assuming funders would support a study that the Agency would ignore), with the bleak awareness that its benefit for public health and the environment -- not to mention the researchers' professional advancement -- might be arbitrarily thwarted by EPA or, (2) they might drop the study entirely in order to avoid the Agency's erratic minefield of requirements, which would deprive the entire public of the benefits of their study.
The commenter (12727) states that even if some researchers obtained additional funding to meet requirements, the overall pool of research funding cannot be expected to increase commensurately. Therefore, the commenter provides that if the cost of each study increases, and the total funding remains constant, then fewer studies could be funded; impeding not only studies incompatible with the proposal's arbitrary criteria, but even those that are compatible, drastically winnowing the available body of evidence upon which to base public health protections. The commenter asserts that organizations that devote resources to meet the proposal's requirements "will have to shift funding from ongoing, vital new research to fund this activity, at the net negative cost to the nation's health." The commenter notes that aside from constraining funds available for new research, the proposal would pose unique challenges when applied retrospectively to prior research, possibly precluding the use of past studies, with distortionary impacts on Agency analyses.
Commenter (12727) contends that the introduction of the tiered access alternative in the supplemental proposal raises major additional concerns about the costs to the research community. The commenter states that the tiered access approach would likely be difficult and costly to administer, but EPA has not explained, even in general terms, how this approach would be implemented, the degree of burden it would impose, or who would bear the costs. However, the commenter points to a recent briefing with staff on the House Committee on Science, Space, and Technology, where EPA staff revealed that "key implementation responsibilities [for tiered access] will fall on the research community." The commenter provides that EPA's apparent expectation that it will impose massive new costs on the research community without acknowledging as much in the Supplemental Notice is arbitrary and a violation of notice requirements. According to the commenter, it would further privilege EPA's reliance on studies from wealthy organizations or those with a financial incentive to expend the necessary resources for EPA to consider their work.
Commenter (12727) notes that EPA's arbitrary fixation on the public availability of underlying data would impair scientific research is the deterrent effect on study participants. The commenter asserts confidentiality pledges to study participants are often essential in order to collect accurate and complete information. The commenter maintains that this concept was reinforced by a SAB member, who advised EPA that the possibility of disclosure of PII would "[c]ertainly . . . cause[] some participants to decline participation." The commenter contends that the proposal could encumber the researcher-participant relationship by introducing several unnecessary concerns into the process: (1) some subjects in public health studies may lose confidence that their data will remain confidential; even if researchers withhold the names of study participants, the increased pressure on researchers to release information could instill a reasonable fear in subjects that their data could be traced back to them; (2) if researchers committed to keeping underlying data confidential, subjects might reasonably fear that EPA would arbitrarily disregard the findings of the study, lowering their incentive to participate, EPA seems to acknowledge that study participants are motivated in part by the potential impact of a study on public health protections; (3), by imposing new requirements for data availability that have no scientific basis, EPA may create a false impression that the public release of data is an indicator of study quality, this could deter participants from participating in studies for which data will be kept confidential.
The commenter (12727) contends that as a result, scientific research would become more arduous and expensive; fewer studies could be performed, to the detriment of public health policy at EPA and other regulatory bodies. The commenter provides that EPA's failure to consider these costs is arbitrary and unlawful and notes that while these concerns also applied to the 2018 proposal, the expanded scope of the Supplemental Notice greatly exacerbates them.
Response: This procedural rule does not regulate the conduct or determine the rights or obligations of any entity outside the federal government. The EPA disagrees that this rule would impose costs on third-party researchers. This is a rule of internal procedure that does not impose requirements on any party other than the EPA. This rule imposes no costs on researchers or their institutions, and the EPA will consider and evaluate all relevant and appropriate science in its significant regulatory actions and influential scientific information. The EPA recognizes that researchers and other third parties may voluntarily consider the EPA's position on data availability, as described in this rule, as they make their own decisions about how to conduct research and the extent to which they make data and models available. Researchers may choose to make more data and models available, but the EPA recognizes that these parties will weigh their own benefits and costs and make choices that they deem appropriate. Additionally, this rule does not require the EPA or other parties to disclose or host data. Hence, this rule does not impose costs on the EPA or any other party to make such data available. Analyses of proposals that impose such requirements are, therefore, not relevant for this rule. The EPA recognizes there will be some incremental review and administrative costs to the Agency but anticipates these will be small. The analysis noted by the commenter is not applicable to this rule because the analyses were not for this rule and the assessed requirements are not part of this rule. 
The EPA acknowledges that data sharing mechanisms already exist and that researchers, if they so choose, could select the mechanism that best suits their needs and effort. The preamble to the rule clarifies that restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. The rule does not prescribe a particular mechanism but does require the EPA to independently verify that such a mechanism exists for dose-response data underlying pivotal science. 
Further, although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges, and agrees with commenters, that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7.
2.2.2.6.2 Resulting Bias in Available Research
Comment: Commenters (10974, 11479, 11491, 11576, 11920, 12430, 12464, 12546, 12720, 12727, 12731) contend that the proposed rule will result in a bias in what information EPA is able to utilize due to capabilities of certain stakeholders to abide by the rule's requirements and the ease of some studies being able to adapt comparative to others. Additional comments and responses to comments on monetary costs to researchers are included in section 14.2.4.3 of Chapter 14. Examples of specific comments follow:
Commenter (11479) notes that EPA's proposed rule alludes to the use of good laboratory practices (GLP) and standardized test methods, where "available and appropriate." The commenter contends that this would bias EPA's literature review toward industry-funded GLP studies, even though GLP practices are more about recordkeeping than quality of research design. The commenter provides that for example, in a study reviewing the quality of GLP literature on the chemical bisphenol A (BPA), the authors concluded that "simply meeting GLP requirements is insufficient to guarantee scientific reliability and validity." The commenter maintains that while GLP studies should not be excluded from EPA's consideration, the highest-quality assessment that EPA can conduct would be a systematic review approach that considers all relevant data and establishes criteria for evaluating study quality, as recommended by NAS.
Commenter (12720) states that the potential for dangerous bias with adverse impacts on achieving quality scientific input at EPA is large. The commenter provides that for example, it may be relatively less burdensome for small epidemiologic studies with limited numbers of study participants to meet data release requirements-- but the results of such studies are generally less reliable and include the potential for more random error and larger confidence intervals than cohort studies that span larger population groups. The commenter asserts that the proposal threatens to bias Agency decision making towards smaller studies and anecdotal case reports rather than robust scientific information provided in larger epidemiologic cohort studies.
Commenter (10974) contends these proposed changes would bias research results and thereby erode the quality, dependability, and power of research used by EPA, state, local, and tribal governments when charged to develop programs designed to protect environment and public health. As such, the commenter does not support either proposed definition at 40 CFR § 30.5, nor do they support, generally, EPA's attempts to discredit research based on data availability or model reproducibility by either the oft-referred to "subject matter experts" or the public.
Commenters (11491, 11576, 11920) state that the EPA's own SAB highlighted that adoption of this rule would introduce bias by selectively excluding relevant data. The commenters states that, specifically, the Board found that "exclusion of segments of the scientific literature, with the possibility of inclusion of other selected information without pre-defined criteria, could allow systematic bias to be introduced with no easy remedy." 
Commenter (12430) states that in the SNPRM the EPA disclaims any responsibility for obtaining, redacting, or publishing the restricted data, and the SNPRM has no provisions for compensating researchers for this activity. The commenter states that developing and maintaining a tiered access system could prove onerous and expensive. The commenter states that large industrial actors would be able to absorb the costs more readily than other researchers could. The commenter states that, as a result, industry-funded science would be more likely to satisfy the requirements of the SNPRM and qualify for EPA consideration than other science would. 
Commenter (12546) states that it is well documented that financial sponsorship (i.e. source of funding) introduces a risk of bias in the results and conclusions in favor of the regulated industry's interests, including animal studies examining the effect of exposure to atrazine on reproductive or developmental outcomes. The commenter states that this has the impact of potentially reducing or eliminating regulations that apply to industry products and ultimately have the effect of increasing profits. 
Commenter (12464) asserts that the fact that the proposal places the burden of making data publicly available on researchers systemically favors industry data over academic data. The commenter states that academic scientists have no incentives to go back to old studies and -- at a considerable cost of time and money -- identify the relevant data and make it available to the general public or through a tiered access system. The commenter states that industry scientists, however, have a financial incentive to make available data from studies that support those industries' goals. The commenter states that such an approach does not evenhandedly promote transparency in science.
 Commenter (11497) states that, in reality, "public scrutiny" would only serve industry's interest, either by eliminating the use of studies that contradict their claims of chemical safety or by allowing them to attempt to discredit studies and researchers by "reinterpreting" the data in their favor.
Commenter (12415) contends that, if the rule is adopted, the EPA may rely less on valid information and more on potentially biased data and research from the regulated community (e.g., polluting industries) that would ostensibly meet the standard of "public" while not meeting the standard of independent science.
Response: As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges, and agrees with commenters, that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7.
The technical considerations in 40 CFR 30.5(d), the equal consideration of data available through restricted access, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data are intended to minimize potential unintended consequences, such as the potential for bias suggested by the commenter. In addition, this final rule sets the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. The EPA believes the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making.
The EPA received comments from the SAB on the 2018 NPRM and 2020 SNPRM on April 24, 2020. The EPA considered these comments in development of the final rule. In brief, the SAB consensus was that it supported the precept of enhancing public access to scientific data and analytical methods to ensure scientific integrity, consistency, and robust analysis and their consensus recommendations were that the EPA develop additional policy and/or guidance documents; further describe how pivotal science would be selected, made available, and peer reviewed; consider establishing an office on data sharing with the support of a peer review panel; provide greater clarity in the definitions of key terms, including data, independent validation, and publicly available; develop specific criteria for the Administrator's exemptions under the rule; consider applying rule requirements only to information developed after the effective date of the rule; seeking input from experts to identify best practices to ensure efficiency, utility, and protection of data that are made available; and consider funding reanalysis work. 
As noted in the preamble for the final rule, this rule sets the overarching structure and principles for transparency of pivotal science in significant regulatory actions and influential scientific information and technical considerations, including those noted by the SAB, could be addressed in subsequent implementation guidelines and/or statute-specific rulemakings. Further, in response to comments from SAB and the public, the EPA clarified and modified several key definitions, including data, independent validation, and publicly available, in the final rule. The EPA also provided criteria for the Administrator's exemption in 40 CFR 30.7 of the final rule. The EPA would like to clarify that this rule does not direct or require any outside entity or the EPA to establish data sharing mechanisms and does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science. Comments related to making data available and future funding opportunities are not within the scope of this rulemaking. Finally, the EPA is maintaining that the rule provisions apply equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. However, in consideration of instances where data cannot be made available, the EPA amended the rule to provide technical considerations (40 CFR 30.5(d)) that can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability in the absence of maximum transparency and included an exemption criterion -- that the development of the dose-response data was completed or updated before the effective date of the rule -- to 40 CFR 30.7 to account for the concerns raised by the SAB and other public commenters.
Comment: Commenters (12727, 12731) note that SAB member Robert W. Merritt commented that data from older studies may be located in obsolete storage media or data formats that are either impossible or very costly to convert to a shareable form. Moreover, the commenter states that grants funding older studies have likely lapsed and little to no funding would be available to academic and non-profit institutions to make data available, potentially skewing the pool of research towards for-profit industry studies. The commenter also asserts that EPA has disclaimed any responsibility for this cost and is instead opting to discard studies out of hand.[,]
Response: As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. 
Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges, and agrees with commenters, that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7.
Comment: Commenters (11576, 12720) notes that the proposal states that "the Agency does not intend to make all data and models underlying pivotal regulatory science and pivotal science publicly available (85 FR 15402). The commenters state that the rule fails to consider how the EPA plans to assess which studies and underlying data will be made publicly available, or how it plans to prevent potential bias in its decision to make certain data and models publicly available as opposed to other data and models. 
Commenter (12720) states that there is a significant potential that the EPA could selectively release only specific types of studies to members of the public, including biased handling of data and models involving CBI. The commenter states that CBI is already protected from release under FOIA. The commenter states that, in 2019, the EPA finalized an update to its FOIA Regulations. Under this rule, the Administrator and other officials are allowed to review all materials that fit a FOIA request criteria and can decide "whether to release or withhold a record or a portion of a record on the basis of responsiveness or under one or more exemptions under the FOIA, and to issue `no records'' responses." The commenter states that, given the vague language in this proposal and existing FOIA regulations at EPA, there is a significant potential for political interference and arbitrary determinations within EPA for public release of data and models.
Response: Based on a consideration of these and other similar comments, the EPA clarified in the final rule how pivotal science would be selected. In addition, the EPA intends for the final rule to be implemented in a transparent manner. The final rule requires the EPA to document the rationale for exemptions granted by the Administrator to the requirements included in the final rule in the significant regulatory action or influential scientific information.
Comment: Commenter (11176) expresses concern that the rule will hamper the use of evidence in EPA work because of the discounting of important research and introduce potential bias in the rulemaking process, which in turn may have very detrimental health and environment effects. 
Commenter (11186) asserts that the EPA should not pick and choose which studies go into laws based on public availability because this approach would not be telling all sides of the story: that would be biasing our laws to minimize the impacts of environmental exposures. 
Commenter (11495) states that, in the alternative Option, the EPA is proposing to "give greater consideration," but this is absent of any scientific justification. The commenter states that the "greater consideration" Option is not fully discussed with no details provided about how the Agency would go about providing a weight to an individual study based on the availability of underlying data. The commenter states that this presents a considerable potential to introduce bias. The commenter states that, for instance, studies which contain PII are most likely to reflect specific health outcomes at an individual level, where the existing detail promotes mechanistic understanding and generalizability by clearly defining the impacted population. The commenter adds that, without CBI and proprietary data, analyses are not sufficiently informed of the true impact on business practices, management and economic outcomes. 
Commenter (12430) contends the SNPRM would introduce a bias towards less sensitive studies. The commenter states that higher-quality study designs typically collect detailed demographic information and information on risk factors and medical conditions so that susceptible subpopulations can be identified. The commenter states that the more detailed the study, the more difficult it would be to remove PII and eliminate the risk of re-identification. The commenter states that, under the SNPRM, studies that omit personal information would be given greater weight, since this information can be released without linking to an individual. The commenter states that personal information is used to strengthen a study by providing a way to account for impacts due to characteristics other than the exposure effects, such as effects due to age, gender, smoking status, and preexisting medical conditions. The commenter states that it also allows an investigator to stratify the analysis by income and race/ethnicity to see whether any subgroups may be more susceptible to exposure to pollution. The commenter states that studies with individual personal information often are subject to agreements with the study subjects that limit who can view their personal information, making it impossible for this information to be released, even if it can be grouped in such a way as to not identify individuals. The commenter states that the studies that do not contain detailed personal level data are not as strong, since potential confounders cannot be corrected for, thereby limiting the ability of EPA to discover, and base its decisions on, the most accurate determination of the true causes and consequences of a health impacts.
Commenter (12458) contends that giving greater weight to studies where the underlying data is available will increase the relative weight of industry-funded research and research using non-human subjects, which are less likely to foreground questions about human health. 
Response: The EPA considered these and other similar comments when developing the final rule. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges, and agrees with commenters, that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7.
The technical considerations in 40 CFR 30.5(d), the equal consideration of data available through restricted access, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data are intended to minimize potential unintended consequences, such as the potential for bias suggested by the commenter. In addition, this final rule sets the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. The EPA believes the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making.
Including this review of dose-response data availability for pivotal science is critical to the EPA's progress toward increased transparency and providing increased opportunity for scientific reanalysis and review by independent third parties. This approach will result in significant regulatory actions and influential scientific information based on the highest quality, most relevant studies and ensure adequate human health and environmental protections while maximizing transparency and maintaining consistency with the statutes the EPA administers. The EPA does not believe that this final approach will lead to bias towards any party.
2.2.2.7 Retroactive Application Would be Burdensome
Comment: Commenter (11358) reiterates their 2018 comments that, regardless of the components of the final rule, EPA should seek to phase-in whatever requirements are justified to allow for sufficient time to respond and prepare for the implications of this rule. The commenter contends that this rule should also not apply to the previous record, and that trying to apply this proposal to models, rules, and research that has already begun or has concluded would only serve to set current work back and complicate work already done. The commenter provides that it makes sense to "grandfather" what has already been completed and to implement this rule in stages in order to not compromise or delay EPA's work.
Commenter (12720) states that the supplemental proposal "would apply to reviews of data, models, and studies at the time a rule is developed or influential scientific information is finalized, regardless of when the data and models were generated." According to the commenter, this requirement raises numerous serious implementation issues.
Response: The final rule applies prospectively to significant regulatory actions and influential scientific information; the final rule has no retrospective effect on either significant regulatory actions or influential scientific information. 
Comment: Commenter (11391) states that applying the Proposed Rule retroactively could undermine the scientific bases supporting many of our most important environmental protection programs. According to the commenter, retroactive application would mean that when EPA is, for example, conducting a review of NAAQS or considering modifying the list of Hazardous Air Pollutants pursuant to the CAA, or reviewing Water Quality Standards under the CWA, it could refuse to consider valid, peer-reviewed epidemiological studies that have supported existing standards if all the underlying data in those studies has not been made public. The commenter contends that this could potentially decimate many of the existing pollution standards that unquestionably protect both the environment and human health. 
Further, the commenter (11391) notes that in many cases, there will be valid reasons why the data underlying historic studies cannot be made public. In this regard, the SAB has noted that the underlying data "may have been discarded if they were deemed not necessary to maintain" and, even if they still exist, "the researchers may no longer be alive or in a position to assemble the data." The commenter provides that data assembly is time consuming and labor intensive; it is unclear whether EPA would perform the necessary work itself or require researchers to do it. The commenter states that either way, regulatory decision-making would be delayed, and scientific research hampered. 
Response: The final rule applies prospectively to significant regulatory actions and influential scientific information; the final rule has no retrospective effect on either significant regulatory actions or influential scientific information.
Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges, and agrees with commenters, that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7.
Comment: Commenter (11576) contends that the proposed rule must not be applied retroactively to existing data, models, and studies -- indeed, it must not be applied at all. The commenter provides that complying with the requirements of the proposed rule would represent a significant departure from standard scientific practices, as data availability typically has no bearing on the accuracy or robustness of scientific findings. The commenter asserts that if EPA finalizes the proposed rule, EPA must provide adequate notice to allow researchers to attempt to comply with the rule's requirements, which will impact not only data reporting but privacy agreements for data collection. Furthermore, the commenter notes that without ensuring that infrastructure exists to make such data sharing feasible, the EPA cannot expect previously generated data and models to be able to be shared. The EPA's SAB has rightly warned that "retrospective application of the requirement would be difficult to implement, could be expensive with no clear responsibility regarding who would cover the added costs, and could arbitrarily impact the conclusions drawn."
According to the commenter (11576), retroactively applying the proposed rule to existing science would arbitrarily and unjustly eliminate a large collection of established, peer-reviewed, and verified science generated prior to the establishment of the rule's expectations. Furthermore, the commenter states that the proposed rule would impose new challenges for researchers related to data handling, storage, and the protection of sensitive information, and studies conducted prior to this rulemaking would likely not have made arrangements to allow for sharing of sensitive data publicly in the ways required by the proposal. Additionally, the commenter (11576) asserts that retroactive application of the proposed rule would conflict with the practice of building scientific studies and models iteratively over time in the context of existing literature, as it would essentially wipe out consideration of older and established studies conducted prior to the establishment of mechanisms for sharing data publicly. The commenter states that effectively excluding older studies based on this arbitrary criterion would undermine the most critical and foundational studies and impose tremendous challenges for more recent studies to re-establish foundational findings for acceptance under the guidelines described in the rulemaking.
Response: The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods. The provisions of the final rule apply to science that serves as the basis for informing a significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science.
Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges, and agrees with commenters, that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7.
The EPA would like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
The EPA would also like to emphasize that the final rule applies prospectively to significant regulatory actions and influential scientific information; the final rule has no retrospective effect on either significant regulatory actions or influential scientific information.
Comment: Commenter (12720) contends that one important aspect of the retroactivity issue that EPA must consider is that it is not sufficient to apply the policy prospectively to new agency actions, because such actions may rely on scientific products that pre-date the new policy. The commenter refers to the CWA, which requires revision of standards every three years, and notes section 304(a)(1) of the CWA, where the water quality criteria that state standards include are commonly based on levels EPA determines to be safe enough to support various uses in reliance on available science. 
According to the commenter (12720), EPA's criteria guidance is -- as the statute provides -- periodically updated, but any new update will be based on then-existing scientific studies (especially if there is no evidence to suggest that those studies should be ignored). The commenter notes that accordingly, EPA may be issuing revised criteria documents and making approval/disapproval decisions on newly-submitted state standards that rely on science products that pre-date this rule for many years into the future. The commenter asserts that any final rule must not limit EPA's ability to use these pre-existing studies for new Agency actions.
Response: The final rule applies prospectively to significant regulatory actions and influential scientific information; the final rule has no retrospective effect on either significant regulatory actions or influential scientific information.
Further, as described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider underlying dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the EPA acknowledges, and agrees with commenters, that there may instances where the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7.
2.2.2.8 Benefits of Proposal
Comment: Commenter (11911) contends that the Agency does not adequately explain the expected benefits of the proposed rule. According to the commenter, applying this rule retroactively does not alter the incentive for papers that have already been published, while penalizing the Agency and public by excluding potentially valuable information from the regulatory process.
Commenter (10742) states that implementing this tiered system would likely require a significant investment of staff time and resources and it is not clear that the potential costs of the proposal outweigh the benefits.
Commenter (12727) contends that the Supplemental Notice does not cure EPA's failure in the 2018 proposal to articulate the benefits of its policy. The commenter provides that the error is even more glaring now that the scope of the proposal has significantly expanded. The commenter asserts that the SNPRM would have contained at least the seed of benefits that should be all the more obvious now that the proposal covers a wider range of studies and actions, yet EPA still has not characterized what positive outcomes this proposal would deliver. The commenter states that EPA's failure to characterize or analyze the benefits of its proposal -- or to acknowledge the absence of benefits -- is arbitrary and unlawful.
Response: The EPA agrees that neither the 2018 proposed rule nor the 2020 SNPRM included a characterization of costs or benefits of the proposed action. As discussed in the preamble to the final rule, this is a rule of internal procedure promulgated under the EPA's housekeeping authority and imposes no enforceable duty on any state, local, or tribal governments or the private sector. Additionally, this rule does not require the EPA or other parties to disclose or host data. Hence, this rule does not impose costs on the EPA or any other party to make such data available. The EPA recognizes there will be some incremental review and administrative costs to the Agency but anticipates these will be small. In addition, the Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule. 
The main benefits of this rule spring from greater transparency in significant regulatory actions and influential scientific information. By placing greater emphasis on the availability of dose-response data underlying pivotal science, the rule will allow for greater scientific scrutiny as EPA decision makers are developing significant regulatory actions and influential scientific information and increases the likelihood that any errors will be identified and corrected. Greater transparency is also inherently valuable as a principle of good government and provides benefits to the public at large, including reducing the risk of errors in EPA analyses and in the science such analyses rely upon. The ability for independent subject matter experts to validate pivotal science will facilitate more effective comment and engagement with the public during development of future significant regulatory actions and influential scientific information.

 Chapter 3: Capable of Being Substantially Reproduced, Independent Validation, and Reanalyze
In the notice of proposed rulemaking (NPRM), the Agency discusses reproducibility and ability to replicate findings as important qualities of information that fulfills the proposal's requirements.
The Agency is proposing a definition for the term "capable of being substantially reproduced" in §30.2 of the supplemental notice of proposed rulemaking (SNPRM) to read: 
      Capable of Being of Substantially Reproduced means that independent analysis of the original or supporting data using identical methods would generate similar analytic results, subject to an acceptable degree of imprecision or error. 
The EPA's interpretation of "capable of being substantially reproduced" as included in the proposed definition above builds on the description in the "Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies". The Agency states that these guidelines, which were issued by the Office of Management and Budget (OMB), are intended to help agencies ensure and maximize the quality, utility, objectivity and integrity of the information that they disseminate. 
Several comments were received regarding reproduction, replicability, reanalysis, and independent validation. In this section the public comments are organized according to the following topics under sections 3.1 NPRM Replicability, Reproducibility, and Reanalysis and 3.2 SNPRM Capable of Being Substantially Reproduced; Replication and Reproduction:
Section 3.1: NPRM Replicability, Reproducibility and Reanalysis
          Proposed Rule Requirement to Make Available Information Necessary for the Public to Replicate Finding (§30.5)
          Clarification of the Meaning of Terms Used in the Proposal 
          Replication versus Reproducible versus Reanalysis 
          Whether the Ability to Reanalyze, Replicate and Reproduce a Study Should Determine the Quality of the Study 
          Replicability-Specific Comments 
          Reproducibility-Specific Comments 
          Replication (or Reproduction) Crisis
          Reanalysis-Specific Comments 
          Recommends Further Consultation to Develop New Rule Addressing Reproducibility and Replicability Concerns 
Section 3.2: SNPRM Capable of Being Substantially Reproduced; Replication and Reproduction
 Capable of Being Substantially Reproduced; Replication and Reproduction
 Reanalyze
The specific comments provided in this Chapter reflect the comment letters that provide arguments supported by detailed rationale, examples, and references/documentation. The Docket ID numbers for these commenters are cited throughout this Chapter. In addition, some general comment letters may also include comments related to comments included and responded to in this Chapter. These comments are summarized in Appendix B. Note that points and issues raised in these general comment letters may also be raised by the more detailed comments summarized in this Chapter. The responses provided in this Chapter adequately address these more general comments.
3.1 NPRM Replicability, Reproducibility and Reanalysis
3.1.1 Proposed Rule Requirement to Make Available Information Necessary for the Public to Replicate Findings (30.5)
3.1.1.1 Comments Supporting the Proposed Replicability Requirement
Comment: Commenters (1941, 2421, 6105, 6365, 6864, 6921) support the proposed requirement to make available information necessary for the public to replicate findings. Commenters (0039, 0563, 0844, 1486, 2850, 4541, 6867, 6921) refer to a replication (or reproduction) crisis as evidence that supports the need for the proposed rule (specific comments related to replication (or reproduction) crisis provided in section XIV.G.1). Specific comments supporting study reproducibility are provided in section XIV.F.2.
Commenter (2421) expresses support for the proposed rule because they believe it is common sense to base administrative policies, procedures and rules on valid, verifiable information.
Commenter (6864) states the public's interest is best served when science is replicable, transparent, and explainable with other toxicology information. The commenter states that, when studies cannot be replicated or when such studies are not consistent or explainable with other information, using such studies then depends on having access to the underlying data for independent analysis.
Commenters (6105, 6864) state that for many chemicals regulated by EPA, these data are often from dozens to hundreds of studies conducted according to federal guidelines. The commenters report that studies that are not replicated, or for which the results appear to be outlier(s), or that are not consistent within an overall pattern, are still considered by EPA. According to the commenters, EPA's scientists will often contact the authors to obtain additional information or data in order to conduct their own analysis. When such data are forthcoming, independent analyses can be conducted by EPA scientists without the need to disclose confidential information, and public health is better served. The commenters express concern that, when such information is withheld by the authors, EPA's scientists are left with a dilemma - whether and how to use the study. 
Commenter (1941) specifically endorses the core precepts of section 30.5. The commenter reports that efforts to gain access to the NIOSH/NCI "Diesel Exhaust in Miners Study" ("DEMS") data, and the results obtained, serve as a clear-cut example of the vital importance of ensuring that pivotal scientific data are made available for review, replication and reanalysis by other independent scientists. The commenter states that, without that type of third-party expert review, the original results of pivotal regulatory science might not receive the scrutiny and reassessment that is warranted, and the appropriate corrections and amendments of those original results might not be made. The commenter asserts that in turn could lead to the implementation of mis-informed regulatory policy and inherently flawed regulations. According to the commenter, the notice of proposed rulemaking (NPRM), when properly implemented, can serve as a safeguard against that type of adverse outcome. The commenter provided additional information regarding the DEMS study, which is summarized below.
      The DEMS study covered more than 10-year period and published their results in 2012. DEMS' purpose was to assess the potential association between exposures to diesel engine exhaust (DEE) and lung cancer among a population of 12,315 mine workers employed in eight U.S. non-metal mines. The NIOSH/NCI investigators claimed to observe a positive exposure-response trend between estimated levels of respirable elemental carbon and lung cancer. There were a number of significant questions regarding the scientific robustness of the DEMS' investigators' methods and observations. Efforts to help facilitate independent investigators' access to the full DEMS data sets in a manner sufficing to allow for a thorough replication and reanalysis of the DEMS results took well over three years and well over $200,000. The investigators ultimately were able to complete their reanalyses, which in turn led to the publication of four peer-reviewed journal articles highlighting the significant questions and uncertainties that pertain to the original DEMS results. The independent investigators were able to demonstrate that: (i) when better estimates of exposure to DEE are utilized, the reported associations between occupational exposures to DEE and lung cancer are greatly attenuated, with most being statistically non-significant; (ii) the observed health impacts were associated principally with only one mine (the limestone mine); (iii) exposures to radon were significant cofounders, and appear to account for most of the remaining observed lung cancer associations; and (iv) the DEMS results are not sufficiently robust to serve as the basis for a quantitative risk assessment for occupational exposures to DEE and lung cancer. 
Commenter (6365) contends that the experience of the lead national ambient air quality standards (NAAQS) rulemaking demonstrates the salutary purpose that transparency would serve in the regulatory process. The commenter states that, while EPA relied heavily on the Lanphear et al. (2005) study in its analysis of the health effects of lead exposure, the underlying data for the Lanphear study were not available to EPA or to stakeholders for replication and validation at the time of publication. The commenter reports that it took stakeholders years, including litigation under the Freedom of Information Act (FOIA), to obtain that data. According to the commenter, due to the lack of availability, there was no complete reanalysis of that data for nearly a decade and EPA relied on the study in both the 2008 and 2016 lead NAAQS reviews. The commenter states that the lack of available data has been noted in the scientific literature as preventing alternative statistical analyses to be performed, which is particularly important given that key confounding factors such as parental education, intelligence, birthweight, smoking, and race were not controlled in several of the Lanphear study cohorts and may contribute to an overestimation of health effects. The commenter reports that when the data underlying the Lanphear study was reanalyzed in Crump, KS et al. (2013), numerous errors that were not discovered in the peer-review process and that were not previously known to EPA were uncovered and analyzed. The commenter notes that, although EPA ultimately determined that these errors did not materially affect the NAAQS review documents that relied on the Lanphear study, the Agency considered the errors and their potential impact on EPA's regulatory efforts to be sufficiently substantial so as to include a memorandum in the subsequent NAAQS rulemaking docket acknowledging those errors and considering their impact on the regulatory process. 
Commenter (6864) refers to an example supporting the need for replication where there was the publication of a series of studies on a single human group from exposure to a commonly used insecticide that showed an unexpected effect at extremely low exposures. The commenter notes that the finding of these studies has not been replicated in other human studies and is in contrast to extensive animal and other limited human studies. The commenter reports that, while EPA scientists asked the authors of the studies for the underlying data in order to confirm the results at the lower exposure, the authors refused the release of data citing confidentiality concerns. According to the commenter, since EPA had neither confirmatory studies, nor consistent information that made sense in light of the multitude of available studies of the chemical, nor the ability to conduct its own analysis, it chose not to use the incompletely published information from this single human group. The commenter contends that EPA's decision not to use studies with incomplete information is correct. The commenter asserts that when the underlying data are not provided to EPA scientists, especially those adept in the receipt and protection of confidential information, it is difficult to use these studies with confidence to make a credible risk judgment for public health, much less national rulemaking.
Response: The EPA found that comments regarding confusion around the definition of "replicate" in the 2018 proposed rule have merit and is clarifying that the intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze." Therefore, in the final rule the term replicate is not used.
3.1.1.2 Comments Opposing the Proposed Replicability Requirement
Comment: Commenters (1409, 1973, 4894, 6117, 6144, 6170, 6880, 6895, 6915, 8276, 9227) oppose the proposed requirement for information necessary for the public to replicate findings. Specific comments supporting their opposition and others' opposition to the proposed study replicability requirement, where specifics provided, are included under relevant topics under sections XIV.B through I.
Response: The EPA found that comments regarding confusion around the definition of "replicate" in the 2018 proposed rule have merit and is clarifying that the intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze." Therefore, in the final rule the term replicate is not used.
3.1.2 Clarification of the Meaning of Terms Used in the Proposal 
3.1.2.1 Recommendation that EPA Better Define Terms Used in the Rule
Comment: Commenters (6114, 6880, 6915, 6909, 8281-PH20) recommend that the EPA better define the terms used in the rule. Commenter (8281-PH20) states that the terms, "pivotal regulatory science", "replication", "reproducible", and "research data" are not defined or are problematic.
 
Response: In this final rule, in Part 30.2, the EPA is defining the following terms: "data," "dose-response data," "independent validation," "influential scientific information," "pivotal science," "publicly available," "reanalyze," "science that serves as the basis for informing a significant regulatory action," and "significant regulatory actions." The terms "pivotal regulatory science", "replication", "reproducible", and "research data" are not used in this final rule.
3.1.2.2 Request that EPA Clarify/Define Replicability and Reproducibility 
3.1.2.2.1 Clarify/Define Replicability and Reproducibility Differences
Comment: Commenters (6114, 6125, 6351, 6915, 6909) request that EPA clarify or include definitions to better inform the meaning and differences between "replicability" and "reproducibility" in the rule. Commenter (6351) requests clarification on what the EPA considers "replication". 
Commenter (6114) states that this proposal calls for "replication" and "reproducibility"' of studies, yet there are questions within the scientific community as to what these terms mean, and to what degree they are needed. 
Commenter (6125) asserts that, in context, EPA is using "replicate" not in the sense of conducting a new study to determine if the original conclusions also are supported by analyses of new data, but whether the results are "reproducible" by other analysts using the original data. The commenter reports that neither the proposal, nor the legislation it is based on, define "reproducible," or "replicable" as used in the rule and are in fact not clear on how broadly this goal might be applied (information must be sufficient to "replicate" findings). 
Commenter (6909) reports that EPA has seemingly used the words "reproducibility" and "replicability" interchangeably when in fact they do not mean the same thing. According to the commenter, the EPA's lack of clarity here is one example of the numerous ways in which this proposed rule is so vaguely written and uses terms which are so ill-defined that it is difficult to comment effectively  -  it is unclear whether EPA is concerned about only using studies that can be "reproduced" or only using studies that can be "replicated".
Commenter (6880) recommends EPA consider sources discussing the difference between "replicable" and "reproducible" which include:
 http://www.replicability.tau.ac.il/index.php/replicability-in-science/replicability-vs-reproducibility.html [Note: link does not work  -  double-checked comment letter-correct cite],
      https://jblevins.org/log/rep, and
      https://osf.io/tvyxz/wiki/6.%20Future%20Directions/. 
Response: In the final rule the terms replicate and reproduce (or other forms of the word) are no longer used. The EPA is clarifying that the intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze." In addition, the EPA is modifying the definition of "independent validation" in the final rule by replacing "capable of being substantially reproduced" with "produced." The EPA did not finalize the proposed 40 CFR 30.2 definition of "capable of being substantially reproduced" because the term is not used in the final rule's definition of "independent validation" or elsewhere in 40 CFR 30.
3.1.2.2.2 Interpretation and Definition of Replicability 
Comment: Commenters (1973, 6114, 6116, 6125, 6131, 6876, 6880, 6895, 6909, 6924, 9227) who note that the term "replicate" needs to be defined or clarified in the rule refer to literature references defining and interpreting "replication" and provide their interpretation of the term.
   Definitions/Interpretations in the Literature
 Commenters (6924, 9227) refer to a 2016 NAS workshop report and suggest that the EPA should use the term as defined in the 2016 NAS workshop report: 
            Replication means that a scientific experiment or a trial to obtain a consistent result is repeated. The repeated experiment uses exactly the same protocols and statistical programs but with data from a different population. The goal is to see if the same results hold with data from a different population.
 Commenters (6114, 6116, 6131) refer to an opinion piece by Dr. Bernard Goldstein (Goldstein, B., 2017)  which states that replication is best achieved, when the findings of the initial study are supported by other studies approaching the same question in different ways.
   Commenter Interpretation of the Term
 Commenter (6876) states that replication refers to the fact that consistent findings from studies in different populations in different places strengthens the likelihood of an effect.
 Commenter (6880) states that to "replicate" findings means to repeat an entire experiment, generate new data, and get the same results. 
 Commenter (6895) states that replicability is one of the principles of scientific research and in environmental exposure studies that is used to determine if linkages to health effects caused by environmental pollutants can be confirmed in similar or in other populations. The commenter provides that replication means that any scientist studying the same population-wide health effects and pollutant concentrations in an environment should come to the same conclusion regarding linkages between those two factors. 
 Commenter (6909) states that, to replicate a study is to repeat the entire study, independent of the original researcher, using the same methods (but without the use of the original data). 
 Commenter (6125) states that, under standard usage, a study can be replicated without access to the underlying data and this standard usage would apply. 
 Commenter (9227) states that replicating studies does not require access to underlying study data, but rather details regarding the methodological design.
 Commenter (1973) states that replication tests whether different populations recruited in different ways in different locations, with different data collected on them, or in a different manner, and potentially analyzed in a different manner, produces similar results. The commenter provides that this is why scientists believe that replication is the key approach to reaching conclusions.
Response: In the final rule the term replicate is not used. The EPA is clarifying that the intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze."
3.1.2.2.3 Interpretation and Definition of Reproducibility
Comment: Commenters (6117, 6362, 6868, 6875, 6880, 6909, 9224, 9227) who note that the term "reproducibility" needs to be defined or clarified in the rule refer to the NAS workshop report definition of "reproduction" and provide their interpretation of the term. 
Commenters (6875, 9224) express concern that, according to Goodman, Fanelli, and Ioannidis (2016), there is no scientific consensus for what "methodologically reproducible" is. The commenters suggest that, if EPA wants to build a rule around transparency in regards to methodology, "methodologically reproducible" must be defined. 
   Definition/Interpretation in the Literature
 Commenters (6924, 9227) refer to the 2016 NAS workshop report and suggests EPA should use the term as defined in the 2016 NAS workshop report: 
            Reproduction means producing something that is very similar to the original research, but in a different medium or context. A researcher who is reproducing an experiment addresses the same research question but may use different methodologies or experimental measurements than the original researcher did.
   Commenter Interpretation of the Term
 Commenter (6880) states that to "reproduce" findings, which requires publicly available data, means to analyze the data provided and come to the same conclusions.
 Commenter (6909) states that to "reproduce" a study is to take the original data from a study and to reproduce its findings by analyzing the data in the same way or using the same code. 
 Commenter (9227) states that reproduction refers to a body of evidence addressing the same hypothesis, but using different populations, methods, etc. 
 Commenter (6117) states that "methods reproducibility" is when a study provides enough information about the experimental and/or computational procedures so that future authors can repeat the study using the same data to obtain the same results.
Response: In the final rule the term reproduce (or other forms of the word) is no longer used. The EPA is modifying the definition of "independent validation" in the final rule by replacing "capable of being substantially reproduced" with "produced." The EPA did not finalize the proposed 40 CFR 30.2 definition of "capable of being substantially reproduced" because the term is not used in the final rule's definition of "independent validation" or elsewhere in 40 CFR 30.
3.1.2.3 Interpretation and Definition of Reanalysis 
Comment: Commenters (6924, 9227) refer to the 2016 NAS workshop report and suggest EPA should use the term as defined by the NAS:
      Reanalysis is conducting a further analysis of data. A person doing a reanalysis of data may use the same programs and statistical methodologies that were originally used to analyze the data or may use alternative methodologies, but the point is to analyze exactly the same data and see if the same result emerges from the analysis.
Commenter (9227) states that reanalysis is a tool to demonstrate the robustness of an effect to changes in the statistical model underlying an analysis of a single data set. The commenter contends that reanalysis does not validate or invalidate a study's findings. 
Commenter (1973) states that, compared to replication, reanalysis is a thin reed. The commenter provides that reanalysis can determine whether the same data, analyzed in a different manner (which may or may not be appropriate) produces similar results. 
Response: The EPA is clarifying that the intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze." In the final rule, Part 30.2, the EPA defines reanalyze to mean to analyze exactly the same dose-response data to see if a similar result emerges from the analysis by using the same methods, statistical software, models, or statistical methodologies that were used to analyze the dose-response data, as well as to assess potential analytical errors and variability in the underlying assumptions of the original analysis.
3.1.2.4 Interpretation and Definition of Validation
Comment: Commenters (0671, 6915) express concern that "validation" of scientific studies is mentioned in the proposed rule, but the term is not defined. 
Commenter (0671) states that the proposed rule is using a very narrow definition of "validation"; in reality the best validation is not from the same data already used but from new data on other populations using other methods. The commenter asserts that the best replication happens when there is agreement across populations and methodologies and study designs. The commenter reports that his broader type of replication is what EPA already highlights when making rules. 
 Response: As discussed in the preamble to the final rule, EPA defines "validation" as to substantiate or confirm. In the final rule, EPA defines "iindependent validation" as "the reanalysis of study dose-response data by subject matter experts who have not contributed to the development of the study to evaluate whether results similar to those reported in the study are produced."
 In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze." Given that proposed 40 CFR 30.5 also included the term "independent validation" and that this term directly relates to "replicate," the EPA also proposed a definition at 40 CFR 30.2 for this term. The proposed definition of "independent validation" included the term "capable of being substantially reproduced." The EPA also defined this term because it was an important component of the definition of "independent validation." The EPA modified the definition of "independent validation" in the final rule by replacing "capable of being substantially reproduced" with "produced." The EPA did not finalize the proposed 40 CFR 30.2 definition of "capable of being substantially reproduced" because the term is not used in the final rule's definition of "independent validation" or elsewhere in 40 CFR 30. The EPA is also modifying the definition of "reanalyze" to specify the use of the same methods because the proposed rule specified the use of the "same or different" methods. This change was made so that the definition would be consistent with the final rule's definition of "independent validation." 
3.1.3 Replication versus Reproducible versus Reanalysis
Comment: Commenters (1973, 4840, 6132, 6158, 6170, 6874, 6880, 6893, 6916, 6924, 6925, 8771-PH56, 9227) contend that while replication is important, replication is not done by making underlying data available to reanalyze but is by determining whether the same effects occur and that making data publicly available is insufficient to allow someone to replicate findings. Commenters explain that validation in science is usually accomplished not by reanalyzing published studies' data, as proposed by this draft rule, but instead by other scientists conducting separate studies in other populations to test if the study results are replicable elsewhere or not.
Commenter (1973) states that it has been argued that public data would allow reanalysis of data to ensure conclusions of the studies EPA relies on are valid. The commenter agrees that reanalysis has a role to play, however, the commenter states that the key determination of the consistency of scientific evidence comes from replication, not reanalysis. The commenter asserts that replication is not the re-analysis of previously published studies. It occurs when other researchers, independently conduct similar research on different populations in different locations and come to similar conclusions. Science reaches consensus when competing hypotheses have been tested in multiple independent replications with broadly consistent results, not when one data set has been reanalyzed many times. The commenter asserts that the only thing that would ostensibly be gained by EPA's rule would be, were investigators to comply with EPA's request, the ability for others to re-analyze the same data.
Commenter (4840) states that the scientific enterprise involves approaching the same question in different ways to determine if the results support each other. The commenter asserts that reanalyzing the same study over and over is little different from checking on a surprising newspaper article by buying additional copies of the same paper to see if it says the same thing.
Commenter (6158) states the proposed rule would only address the issue of reanalysis (i.e. analyzing exactly the same data and determining if the same results emerges from this new analysis), but this would do nothing to address replication (i.e. doing the same exact experiment on a different population to see if the same result occurs) or reproducibility (i.e. addressing the same research hypothesis in a different study system or with different methodologies to see if similar findings occur). The commenter provides that, to the extent that there is a replication/reproducibility issue in science generally, the solution is additional, different, independent studies on the same general issue, not essentially rechecking the math in previously published studies that have already undergone extensive peer-review.
Commenter (6170) states that it is much more important in our science to replicate results with new data and in new settings than to re-analyze existing data. This is the type of replication that needs to be encouraged and not the type of data sharing the proposed rule would seek. Many such replication studies have been and will be done outside the United States and these data would never become available to EPA  -  or being made publicly available  -  but some may represent the best studies worldwide, due to high level human exposures in such countries (arsenic in Bangladesh and Taiwan) and the national health system records that are available (in Denmark, Sweden, Norway and Taiwan for example) and much more comprehensive than anything currently available for conducting human studies of this kind in the US.
Commenter (6880) states that it is clear that making data publicly available is insufficient to allow someone to replicate findings and, therefore, EPA's insistence that data must be made public in a manner that allows the findings to be replicated is fundamentally unworkable. 
Commenter (6893) reports that reanalysis of data may be undertaken at times to increase confidence in particularly unexpected or consequential findings, but typically confidence in findings is strengthened by having evidence from different studies accumulate.
Commenter (6916) states that the Proposal's focus on public release of the underlying data used in studies  -  particularly "dose response data" and studies, evinces a clear lack of understanding of the way science is conducted and verified. The commenter provides that, while replicating the results of a study with the same data set certainly verifies it, as the National Research Council said in 2002, "[f]or large epidemiological studies, repeating a study is seldom either possible or desirable." According to the commenter, a more effective approach is some combination of new studies testing the same question in different places, and/or an independent analysis with the same data. The commenter asserts that formal peer review, whether or not prior to publication, also validates the outcome of scientific studies.
Commenter (6924) states that, because the proposed rule argues that data and methods should be made available in sufficient detail that one could redo the analysis, the proposed rule appears to be primarily concerned with reanalysis rather than replicating experiments even though the citations for this section of the rule are focused on issues with replication of experimental findings, not reanalysis. The commenter provides that a re-analysis of a study's data alone cannot verify or invalidate scientific findings without the requisite attention to evaluating the research design, methods, uncertainties, biases, and consistency with existing publications. 
Commenter (6925) states that our scientific understanding of how air pollution affects health outcomes is based on evaluating a range of high-quality studies conducted in different populations, using a variety of appropriate methods, and evaluating a variety of air pollution mixtures. The commenter provides that the concept of "repeatability" in the evaluation of whether an exposure causes a particular health outcome refers to the notion that many different studies on different populations show consistent findings, and that such studies offer complementary data to draw this conclusion. The commenter contends that this type of repeatability provides a stronger evidence base from which to conclude that there is a link between the exposure (e.g. air pollution) and the health outcome. The commenter asserts that reevaluating the same study using the same data and methods simply helps to confirm that specific analysis but does not help to expand the evidence base and our understanding of the science of how pollution impacts human health.
Commenter (8771-PH56) states that most often, reanalysis only reveals that different levels of precaution are consistent with the available scientific information; the choice among them is not a job for science, but for policy. The commenter provides that if EPA wants to advocate for a policy of taking more risks with public health in order to impose fewer costs on industry, it should do so forthrightly, rather than hiding the policy change under the banner of "better science." The commenter asks what if someone did reanalyze a health study and got a very different answer, and one that suggests the first study had exaggerated the harm? According to the commenter, in such a case, the "second" study would be "right" and the first one "wrong" only if both of the following conditions held true: 
      (1) The difference in the results was not already acknowledged (or contained but unacknowledged) within the uncertainties inherent in each answer! After all, if the first investigators claimed that banning a chemical would save somewhere between 500 and 1500 lives across the country, and EPA chose to estimate its benefits as the average of 1000 lives saved, a second study estimating only 600 lives saved would not be "different" at all; and 
      (2) The first study was not just different, but wrong. Anyone can take the same data and botch the risk analysis of it, making it seem like they have a better answer. Just as there are potential problems with an analysis that doesn't control for some variable, it can be a mistake to control for a variable that shouldn't be included -- in other words, more pseudo-sophistication doesn't necessarily mean better. 
Commenter (9227) states that reanalyzing a study using publicly available data is not necessary to ensure valid science nor sufficient to ensure against invalid results. The commenter reports that, to ensure the validity of scientific research, the scientific community relies most heavily upon peer review. The commenter reports that studies used by EPA are often evaluated by one of EPA's scientific advisory boards, such as the Clean Air Science Advisory Committee (CASAC) or the Science Advisory Board. (SAB) The commenter provides that these types of reviews do not depend on a study's data being made publicly available. 
Commenter (9227) provides that reanalyzing a study is just one of many ways the scientific community ensures integrity, and it is not, in fact a widely used mechanism. The commenter contends that reproducing study results using a different population or method is generally considered a stronger validation than simply reanalyzing the results using the same data, as it shows that the results hold across a different population. 
Response: Although the EPA agrees that meaningful insights can be obtained through similar studies in different media or context, the EPA continues to find that independent validation of the study findings and conclusions driving the EPA's dose-response assessments would provide important information. As detailed in the preamble to the final rule, there is scientific support for the usefulness of reanalyzing data, and the EPA finds this to be especially true for data that drive the quantitative requirements or analyses of EPA significant regulatory actions or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and, thus, increase the opportunity for independent subject matter experts to validate pivotal science, and as the dose-response data are better understood through independent reanalysis the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
The Agency recognizes that the 2018 NPRM used terms that were not fully defined or not defined at all. The 2020 SNPRM addressed a number of these comments. To address the concerns expressed by commenters, particularly concerning "reproducible" vs. "replicable" vs. "re-analysis", in this final rule, in Part 30.2, the EPA is defining the following terms: "data," "dose-response data," "independent validation," "influential scientific information," "pivotal science," "publicly available," "reanalyze," "science that serves as the basis for informing a significant regulatory action," and "significant regulatory actions." The EPA is also clarifying that the intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." Therefore, in the final rule the terms "pivotal regulatory science", "replication", "reproducible", and "research data" are not used or defined. 
3.1.4 Whether the Ability to Reanalyze, Replicate and Reproduce a Study Should Determine the Quality of the Study 
Comment: Commenters (6111, 6127, 6144, 6921, 9227) generally argue that, while many environmental studies simply cannot or should not be replicated or reproduced does not determine the quality of the study. 
Commenter (6111) states that it is not necessary to reproduce a study to validate it. The commenter provides that many of the studies that cannot be reproduced are valid but would be excluded by the proposed rule. 
Commenter (6127) states that, although this rule aims to propose a solution to a replication crisis, there is no mention in the rule, or its citations, or of a study that was shown to be flawed due to a lack of raw data. 
Commenter (6144) states that the ability of a study to be reproduced or replicated should not be held as the gold standard for checking the credibility of a study. The commenter asserts that, while it is impossible or simply unethical to recreate certain studies, this lack of recreation does not invalidate the findings in any way. The commenter recommends that EPA should have the flexibility to use its own criteria to judge the rigor and validity of the science informing rules as applicable. The commenter contends EPA already does this well and the rule does not point to any evidence to the contrary. 
Commenter (6144) reports that, throughout the proposed rule, EPA argues that part of the reason for the policy is to allow regulators to better determine that key findings are "valid and credible." According to the commenter, the benchmark upon which validation and credibility are measured for the purposes of this rulemaking are through the relative reproducibility and replication of studies, even though there are certainly other criteria that agency scientists currently use to evaluate a study's credibility. The commenter asserts that the EPA's argument that requiring public disclosure is the way to achieve validity and credibility is inconsistent with provisions in OMB's Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies that indicate an agency can verify the independence and credibility of a study by evaluating the research design, methods, data summaries, and how closely aligned the study is with similar literature in the field. The commenter also notes that the lack of consideration for sensitive information in the proposed rule is inconsistent with provisions in the Guidelines document that note that, while transparency of data is important, it does not override other compelling interests such as privacy, trade secrets, intellectual property, and other confidentiality protections.
Commenter (9227) provides that the Harvard Six Cities Study has been reanalyzed and reproduced -- the underlying data is not publicly available because of patient confidentiality protections bound by individual contractual agreements between the scientists and the research participants and by the HIPPA. The commenter asserts that the unavailability of the underlying data is unrelated to the validity, integrity or quality of the Harvard Six Cities Study. 
Commenter (9227) states that the results of the Health Effects Institute (HEI) reanalysis of the Harvard Six Cities Study suggests that routine assessment of quality indicators such as methodology, confounding and bias routinely evaluated in the peer review process are generally enough to confirm a study's validity. 
Similarly, commenter (8771-PH56) states that the history of calls for reanalysis of major environmental health studies such as the Harvard Six Cities Study led to an expensive and independent total reanalysis convened by the HEI which found that the Harvard researchers had only slightly underestimated the relative risk of a unit of fine particle exposure. 
Response: In the final rule the terms replicate and reproduce (or other forms of the word) are no longer used. In the final rule the term replicate is not used. The EPA is clarifying that the intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze." In addition, the EPA is modifying the definition of "independent validation" in the final rule by replacing "capable of being substantially reproduced" with "produced." The EPA will not finalize the proposed 40 CFR 30.2 definition of "capable of being substantially reproduced" because the term is not used in the final rule's definition of "independent validation" or elsewhere in 40 CFR 30.
The EPA is a leader in the evaluation of human and ecological impacts of environmental pollution and methods to cost-effectively mitigate these impacts. The selection and use of quality, peer-reviewed literature is a critical part of this work. However, the strength of the review conducted as part of the study's acceptance and publication in a scientific journal has received increased attention in the scientific community due to the lack of transparency and consistency, potential for bias, and the difficulty that independent researchers have had in reproducing published results.,,,,, Jamieson et al. (2019) state, "Journals play a key role in ascertaining the quality of the evidence in work submitted for publication through peer review. However, that practice is neither uniform nor free of exploitation. While it can select for trustable results, its existence alone is not sufficient to verify the quality of the evidence used to back claims." Because of the potential impact of significant regulatory actions, independent validation of studies that are pivotal to the EPA's significant regulatory actions and influential scientific information may increase public confidence in the findings and conclusions presented in those studies. As Jamieson et al. (2019) states, publication alone (including the typical peer review necessary for publication) is insufficient to verify a study's quality, and additional confidence could be gained from third-party reanalyses that independently validate the study results.
The EPA agrees that data transparency is vital for individuals who have not contributed to the study to be able to verify the quality and strength of published studies and agrees with commenters that the opportunity to independently validate the pivotal science that the EPA relies upon is important in furthering scientific understanding and the Agency's mission. A presenter in a 2016 NAS workshop on Principles and Obstacles for Sharing Data from Environmental Health Research stated more directly that "for environmental policy making to be legitimate, the scientific reasoning behind a given decision -- including the data supporting it -- must be transparent" (NAS Workshop Report, Ref. 26). When data are widely available, researchers can validate research results and help identify and correct unintended errors, as well as reanalyze the data for new and different purposes, examine novel questions, provide new scientific insights, and improve model development.
This final rule is not intended to address a specific instance but many commenters have provided examples of Agency actions that could have benefited from the provisions of this final rule.
Comment: Commenters (1364, 1973, 6153, 6351, 6874, 8272) assert that replicating findings across similar studies is more informative than reproducing a study using the same data.
According to commenter (1364), support for the findings of studies, especially pivotal regulatory science, is best done if more than one group is studying the same thing, replicating the study simultaneously. The commenter provides that those studies could then be averaged, compared and further analyzed. The commenter inquires that, to zoom the principle of sample size out to the bigger picture, would it not benefit the EPA to have a bigger sample size of studies to amass data from? 
Commenter (1973) states that the replication process of science provides protections against inappropriate methods because dozens of studies can be done, in different ways, by different teams of investigators. The commenter provides an example where an analysis includes a wide variability in statistical approaches and concludes that, along with the transparency in methods and with the expert peer review of the CASAC for air and other SAB committees, any reasonable goals of assurance that the standard is set based on valid science can be met with the current process.
Commenter (6351) states that, while many studies cannot be specifically repeated, especially those that examine the impacts of historic events, subsequent, similar studies from around the world have echoed their findings on health impacts. 
Commenter (9227) asserts that the ability to replicate a study with very poor scientific quality does not strengthen the scientific belief that any toxicity is present. According to the commenter, studies that attempt to reproduce the same findings must have their quality clearly established before comparisons can be made across the multiple studies. The commenter states that "reproducing" studies is generally viewed as a more informative and resource efficient approach to validation of research.
As an example, commenters (6153, 6874, 8272, 9227) assert that the findings of the Harvard Six Cities and American Cancer Society (ACS) cohorts have been replicated and supported by follow-up research. Commenters (6153, 6874, 8272) state that, since publication of these two landmark studies, dozens of additional peer-reviewed studies, as well as follow-up research involving the Harvard Six Cities and ACS cohorts, have replicated and expanded upon their findings and provided insights into specific effects of particulate pollution on certain groups. Commenter (9227) states that, through reproduction and reanalysis, the original findings of the Harvard Six Cities Study have been validated many times over.
Commenter (6117) states that the goal of repeating a study should be to increase the total amount of evidence. 
Response: In the final rule the terms replicate and reproduce (or other forms of the word) are no longer used. In the final rule the term replicate is not used. The EPA is clarifying that the intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze." In addition, the EPA is modifying the definition of "independent validation" in the final rule by replacing "capable of being substantially reproduced" with "produced." The EPA will not finalize the proposed 40 CFR 30.2 definition of "capable of being substantially reproduced" because the term is not used in the final rule's definition of "independent validation" or elsewhere in 40 CFR 30.
As Jamieson et al. (2019) states, publication alone (including the typical peer review necessary for publication) is insufficient to verify a study's quality, and additional confidence could be gained from third-party reanalyses that independently validate the study results.
Comment: Commenters (6162, 6874, 6875, 6921) agree that any data, methodology, or models produced by EPA should be transparent and available to the public in a reproducible manner. Commenters (6162, 6874, 6921) contend that replication and validation of results (whether the results are reliable and consistent when using different assessment methods) are key to determining whether the science is accurate and of high quality. 
Commenter (6921) recommends that, in setting standards, EPA should include consideration of whether the underlying study data is available for review and analysis. The commenter states that current concerns over low replication rates should make EPA hesitate to identify studies as representing the "best available science" if the underlying data and test methods are not released to allow review and replication using the same data or alternative data sets. The commenter asserts that concealing information that would allow the replication and validation of key research findings not only undermines the validity of the results, it also undermines public support for the policies. The commenter concludes that EPA has an obligation to consider transparency in making any judgments on what is the best available science for making policy decisions.
Commenter (6162) states that studies that are reproducible allow independent researchers to evaluate the myriad of choices and assumptions the researchers have made regarding the data and statistical models and the potential introduction of any sources of bias. The commenter reports that bias in research studies can arise from a variety of sources, including publication bias, model uncertainty, model selection issues, lack of adequate control for confounding variables such as other pollutants and weather, and exposure misclassification arising out of the poor correlation between ambient monitors and personal exposure. According to the commenter, privacy concerns with making accessible the data of individuals can be fully addressed using techniques already employed by the U.S. Census Bureau and other agencies, as EPA noted in its proposal. 
Response: The Agency agrees that greater transparency is necessary. The scientific significance of reanalyzing and validating study results is not a recent development. McNutt (2014) noted, "Reproducibility, rigor, transparency, and independent verification are cornerstones of the scientific method." The National Academies of Sciences workshop on Reproducibility and Replicability in Science also noted that "certainly, reproducibility and replicability play an important role in achieving rigor and transparency." 
Comment: Commenter (9227) provides that the proposed concepts of replication, reproduction and reanalysis do not include a discussion of the scientific quality of a study. According to the commenter, the proposal also does not address other key aspects of scientific quality such as generalizability and bias; and how these characteristics of any scientific study are assessed by the EPA and directly relate to the transparency of any decisions they might make.
Response: The EPA disagrees with the commenter. Data transparency is vital for individuals who have not contributed to the study to be able to verify the quality and strength of published studies and agrees with commenters that the opportunity to independently validate the pivotal science that the EPA relies upon is important in furthering scientific understanding and the Agency's mission. A presenter in a 2016 NAS workshop on Principles and Obstacles for Sharing Data from Environmental Health Research stated more directly that "for environmental policy making to be legitimate, the scientific reasoning behind a given decision -- including the data supporting it -- must be transparent" (NAS Workshop Report). When data are widely available, researchers can validate research results and help identify and correct unintended errors, as well as reanalyze the data for new and different purposes, examine novel questions, provide new scientific insights, and improve model development.
3.1.5 Replication-Specific Comments
3.1.5.1 Replication Is Primarily Limited to Laboratory Studies
Comment: Commenter (9227) states that "replicating findings" is essentially limited to laboratory animal and randomized controlled trials and does not capture the vast majority of human epidemiological studies. The commenter reports that even in laboratory experiments, replication can be difficult due to uncontrolled factors like genetic drift in cell lines and animal strains. According to the commenter, if you do have replicate studies and one has a positive finding and another has a negative finding, there would have to be additional criteria used to determine which study was correct; thus, a failure to replicate should not immediately lead to the conclusion that there is no effect. The commenter asserts that rather than replicating a study, it is far better to develop a better study that replicates the results while providing greater insight into the basis underlying any toxicity.
Response: In the final rule the term replicate is not used. The EPA is clarifying that the intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze."
3.1.5.2 Replicability Is Not Always Possible
Comment: Commenters (0563, 0671, 1335, 1346, 1364, 1385, 1409, 1481, 1486, 1973, 1999, 2004, 4845, 4848, 4933, 5164, 6046, 6105, 6111, 6114, 6116, 6125, 6127, 6130, 6131, 6132, 6133, 6144, 6153, 6158, 6170, 6196, 6351, 6450, 6932, 6939, 6864, 6874, 6876, 6880, 6895, 6909, 6915, 6916, 6924, 8272, 8801, 9227) that oppose the proposal and report that the replication of studies is not always possible.
3.1.5.2.1 One-Time Events (Accidents, Disasters)
Comment: Commenters (0559, 0563, 1335, 1364, 1385, 1409, 1999, 4840, 5164, 6111, 6114, 6116, 6153, 6158, 6450, 6880, 6924, 6932) provide that it would be unrealistic and unethical to replicate exposure conditions of one-time events (accidents, disasters) that many public health studies have been based on. Most of these commenters also state that not being able to replicate a study because of does not invalidate the scientific research.
Commenter (0563) states that a common objection is that "many public health studies cannot be replicated, as doing so would require intentionally and unethically exposing people and the environment to harmful contaminants or recreating one-time events (such as the Deepwater Horizon oil spill)." According to the commenter, what needs to be replicable in studies of such events are not the events themselves, but the procedures used to collect and analyze data and make inferences from them. The commenter provides that, for a study that used tree-rings as proxy temperature measurements, no one needed to use a time machine to return to the 11th through 20th centuries and regrow trees to recognize that the authors had committed confirmation fallacy by excluding certain data and misused a statistical procedure, resulting in false results. The commenter states that all anyone needed was access to the raw data and the computer code used to analyze it. 
Commenter (1409) states that the requirement for replicability could proscribe the use of studies of real-time science during environmental disasters, such as toxic spills, since it would be both unethical and infeasible to replicate such studies. 
Commenter (6111) asserts that in some circumstances it would be inhumane, immoral, or physically impossible to replicate studies since some studies involve natural disasters, or one-time events and cannot be reproduced or replicated. 
Commenter (6450) provides that many real-world field studies cannot be realistically or ethically replicated because doing so would require recreating unique events or exposing populations to hazardous situations and materials. 
Commenters (0563, 1335, 1385, 1364, 1999, 4840, 5164, 6111, 6114, 6116, 6153, 6158, 6450, 6880, 6924, 6932) provide the following examples of one-time events (accidents, disasters) that would be unrealistic and unethical to replicate:
 Studies of Hiroshima and Nagasaki survivors that underlies the SDWA radionuclides regulation. (6111)
 Studies based on the massive oil leak at Deepwater Horizon. (0563, 1385, 5164, 6111, 6158, 6450, 6924)
 Wildfire in the Great Smoky Mountains National Park in 2016. (1364)
 Studies of Minamata residents impacted by contaminated wastewater discharged into Minamata Bay. (6153)
 September 11, 2001 World Trade Center disaster. (6120, 6915, 8913)
 The 1979 Three Mile Island accident. (1335, 6450)
 Hurricanes. (6120, 6915, 8913)
 Studies look at health impacts on people exposed to high levels of smoke from wildfires. (6114)
 The Flint water crisis. (6450)
Response: In the final rule the term replicate is not used. The EPA is clarifying that the intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze."
3.1.5.2.2 Exposure Conditions No Longer Exist and Would Be Unethical to Replicate
Comment: Commenters (6111, 6133, 6153, 6170, 6939, 6450, 6153, 6874, 6932, 6939, 8272) contend that in some cases, the original studies cannot be replicated because the exposure conditions no longer exist and/or would be unethical to replicate. Commenter (6111) states that in some circumstances it would be inhumane, immoral, or impossible to replicate studies since some studies involve exposures and conditions that no longer exist and cannot be reproduced or replicated. According to the commenter, such incidents could not be replicated from practical, safety, or ethical points of view, yet the information they provide regarding real time exposure and dose-response interactions is invaluable. Commenter (2004) states that, while many environmental studies cannot be morally or ethically repeated due to further harm to people, these studies are peer reviewed to ensure that the results are both objective and accurate.
Commenters (5178, 6874, 6939, 8272) state that knowledge about impacts of neurotoxicants has often begun with findings of severe impacts in groups exposed to abnormally high levels of a substance  -  such as developmental delays in babies born to mothers who ate seafood from mercury-contaminated Minamata Bay17  -  and then progressed over future decades as researchers have identified subtler impacts at lower exposure levels.18 The commenters add that studies that rely on health effect data from "natural experiments," such as those experienced by Minamata residents because of the contaminated wastewater discharged into Minamata Bay, cannot ethically be replicated and may not have publicly available data, but should not be disregarded given that they can serve as an early signal of the need for regulation.
Commenters (6153, 6939) state that studies that rely on health effect data from "natural experiments," cannot ethically be replicated and may not have publicly available data but should not be disregarded given that they can serve as an early signal of the need for regulation. Commenters state that knowledge about impacts of neurotoxicants has often begun with findings of severe impacts in groups exposed to abnormally high levels of a substance.
Commenter (6170) provides that the exposures that environmental and occupational epidemiologists investigate are, by definition, considered potentially toxic to humans and thus can and should not be tested experimentally on humans. 
Commenters (6111, 6450, 6153, 6932) provide the following examples of studies/exposure conditions of studies that would be unethical to replicate:
          Studies of the effects of lead from 1970s, when blood lead levels were higher than they are now. (6111)
          Studies of worker exposure to polychlorinated biphenyls (PCBs) before PCBs were banned; these studies formed the basis of water quality standards for PCBs under the Clean Water Act (CWA). (6111)
          Long-term cohort studies of benzene exposure in workers which formed the basis of EPA's 2007 CAA regulation for emissions of hazardous air pollutants from mobile sources. (6111)
          Studies that analyze the health impacts resulting from asbestos exposure, exposure to Dichloro-diphenyl-trichloroethane (DDT), and birth defects caused by thalidomide. (6450)
          Studies of developmental delays in babies born to mothers who ate seafood from mercury-contaminated Minamata Bay. (6153)
          The chlorpyrifos risk assessment relied on an epidemiological study that was done while chlorpyrifos was still allowed for household use; this study could not be repeated today, as indoor use of chlorpyrifos is no longer allowed in the U.S. (6133, 6932)
Commenter (1335) states that knowingly allowing similar conditions to continue in spite of the broad consensus of scientists - or promoting rules that would permit the return of similar situations - would be downright murderous.
Commenter (6133) states that the mercury EPA Reference Dose (RfD) is based on recommendations of the National Research Council (NRC) of the NAS, that conducted an extensive analysis and calculations derived from three longitudinal epidemiologic studies: the Seychelles Islands, the Faroe Islands, and the New Zealand studies. The commenter states that the use of these studies to set EPA exposure limits was the result of a years-long transparent process of expert scrutiny, public engagement, inter-agency cooperation, and publication in scientific journals. The commenter notes, however, the studies can no longer be reproduced, particularly the Faroe Islands study in which the exposure to the community was a result of eating whales, a practice that has since declined due to public alerts about the hazards of eating the mercury-tainted meat particularly for children and pregnant and breastfeeding women. The commenter adds that it would take decades to repeat the studies, which took decades to conduct in the first place. According to the commenter, EPA estimates that more than 75,000 newborns each year may have increased risk of learning disabilities associated with in-utero exposure to methylmercury, based on maternal blood levels exceeding the EPA RfD.
Response: In the final rule the term replicate is not used. The EPA is clarifying that the intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze."
3.1.5.2.3 Long Term Epidemiological Studies
Comment: Commenters (0559, 6170, 6880, 6895, 6932, 8801, 9227) assert that long term epidemiological studies cannot be replicated because the temporal exposure conditions of the studies and other variables (e.g., population exposed) would be impossible to replicate. Commenters generally state that epidemiological studies are scientifically rigorous/valid studies.
          Commenter (6880) provides that these long-term studies involve a specific mix of individuals with their own medical histories and chemical exposures (e.g., National Institute of Health's (NIH's) Agricultural Health Study) that is not replicable.
          Commenter (6932) states that restricting studies that are not "replicable" is shortsighted because longitudinal epidemiological studies are a primary example of scientific studies that can be scientifically rigorous but are not replicable.
          Commenter (6170) states that by disqualifying high quality longitudinal epidemiological and clinical studies - often the most direct and relevant evidence of chemical impacts on humans - the proposed rule would diminish not strengthen the science underlying regulations. 
          Commenter (6170) states that observational epidemiology studies are the only possible setting to gain knowledge about the potential of many agents in the most relevant population: in humans. 
          Commenter (8801) states that on moral, ethical, and legal grounds, it is impossible to intentionally expose humans to known or suspected harmful substances; however, it would be absurd for EPA to argue that these types of epidemiological studies should not be considered when developing policies that affect the health of millions of Americans. 
          Commenter (9227) notes that in environmental epidemiology, randomized control trials are not feasible or ethical, and replication of observational studies is virtually impossible since it is not possible to create the same conditions as seen in the original study. 
Commenter (6895) generally supports the need for replication but states that in terms of epidemiology, replication does not require that every piece of data be available. Rather, based on the principles of scientific research, replicability is achieved when links between environmental pollutants and differing populations are seen in multiple studies by the same or different investigators.
Response: In the final rule the term replicate is not used. The EPA is clarifying that the intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze."
3.1.5.2.4 Need to Protect Individual Medical Records/Information and Confidential Business Information
Comment: Commenters (4845, 6116, 6130, 6170, 6450) provide that studies may not be able to be replicated due to ethical and legal reasons protecting individual medical records/information and confidential business information (CBI): 
          Privacy protections will often block release of individual medical records, especially for epidemiology and other studies of human cohorts. (4845)
          Industry-conducted studies may contain CBI required to be withheld by law. (4845)
          Studies that contain confidential business information, such as product formulations. (6116)
          The original raw data must be withheld to comply with ethical and legal requirements of epidemiological research (e.g., requirements of an Institutional Review Board (IRB) and/ or the Health Insurance Portability and Accountability Act of 1996 (HIPAA)). (6130, 6450)
          IRBs do not allow for such data to be made publicly available. (6170)
Commenter (6116) states that many studies performed using data from industrial sources would be excluded because they contain CBI. The commenter provides that, under federal law, there are criminal penalties for disclosing CBI  -  a serious risk of liability for EPA that the agency does not address except to say generically that all such laws will be followed, without demonstrating how compliance will be achieved in tandem with making research data publicly available.
Response: In the final rule the term replicate is not used. The EPA is clarifying that the intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze." The final rule does not require EPA to obtain, store or disclose dose-response data underlying pivotal science used by the Agency.
3.1.5.2.5 Old Data May No Longer Be Available
Comment: Commenters (6130, 6450) provide that studies may not be replicable because the data may no longer be accessible.
Commenter (4845) reports that, for older studies predating digital technology, retrieving full study records may be difficult or impossible, making it impossible to replicate a study. 
Response: In the final rule the term replicate is not used. The EPA is clarifying that the intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze."
3.1.5.3 Replication of Studies Would be Burdensome
Comment: Commenters (1481, 4848, 4933, 6116, 6127, 6153, 6874, 8272) express concern that replication of studies would be prohibitively expensive and time consuming. 
Commenter (1481) states that the required replication of data and longitudinal studies, while worth pursuing and funding, would take decades to confirm results which currently point out significant and likely threats to public health, resulting in preventable danger to and increased health costs to the public. The commenter asserts that seeking to increase replication and longitudinal studies is an excellent goal which should be accomplished through earmarked funding rather than standards which cannot efficiently serve Americans. 
Commenter (4848) asserts that EPA is not and has never been in the regular business of replicating studies. The commenter reports that the timing and the cuts in EPA funding makes replicating studies (as a condition of promulgating regulations) an impossibility.
Commenter (4933) reports that the very nature of longitudinal public health studies where health and toxins intersect are by design, large, expensive and require years or decades before results are found. The commenter provides that sample sizes can often number in the tens of thousands to millions of data points and may need to be collected over many years before a statistically significant finding is identified. According to the commenter, because these studies are so big, they are often too expensive to repeat. The commenter asserts that this transparency rule would destroy the confidential nature of research and make the burden of conducting research more difficult and expensive. 
Commenters (6153, 6874, 6916, 8272) cite (in part or in total) that in a look back at the evolution of research on the health effects of particulate pollution, lead Harvard Six Cities investigator Douglas W. Dockery commented:
      True replication requires not reanalyzing existing work but independent investigators producing independent data. The HEI process made important contributions in evaluating, advancing, and applying new methods which led to significant additional analyses. But the HEI work reanalyzing an existing body of evidence was time consuming and expensive. It would not be a good use of resources to require this approach for all studies contributing to the regulatory actions, or even to a few key studies.
Response: In the final rule the term replicate is not used. The EPA is clarifying that the intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze."
3.1.5.4 Other Comments Regarding Replicability
3.1.5.4.1 Public Health/Precautionary Principle Relies on More than Data that can be Replicated
Comment: Commenter (8276) states that the protection of public health relies on more than data that can be replicated. The commenter reports that local health departments are obliged to employ the precautionary principle, which asserts that when health or safety is threatened, preventive steps must be taken even in the face of scientific uncertainty. The commenter asserts that the proposed rule will only perpetuate barriers to the protection of public health. The commenter describes three environmental issues which occurred in Los Angeles County where the Department of Public Health (DPH) demonstrated the need to exercise the precautionary principle:
      1. In response to a natural gas leak, DPH ordered the temporary relocation of over 11,000 residents that were experiencing symptoms, largely related to the sulfur odorants. Available health studies on human exposures to complex mixtures of oil and gas were lacking and insufficient to guide public health interventions. In a major oil or gas release, where toxicity information may be vital yet lacking, action based on the precautionary principle may be the only option to protect the health of nearby residents. 
      2. The Exide Battery Recycling Plant released harmful toxins, including lead and arsenic, into the surrounding communities for decades. These emissions contaminated the soil of homes across Los Angeles and increased the risk of cancer, respiratory diseases, and learning disabilities in affected families. DPH immediately stepped into action and could not wait for the State's large-scale, time-consuming approach to soil sampling to be complete. This experience demonstrated the continued need for DPH to make critical program decisions in the absence of complete environmental data.
      3. In recent years, DPH has responded to an increasing number of community complaints of odors and noise surrounding oil and gas drilling facilities near homes and schools. Peer-reviewed literature remains largely inconclusive regarding health risks of living close to oil and gas operations; however, DPH has documented symptoms, such as headaches, nausea, and respiratory irritation, among residents near oil drilling sites. In two cases, DPH acted to reduce exposures and associated impacts in the adjoining community. 
Response: In the final rule the term replicate is not used. The EPA is clarifying that the intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze."
3.1.5.4.2 Issue of Replicability Not Resolved by Making Data Publicly Available
Comment: Commenter (6046) argues that, to assume that just by making data publicly available issues of replicability will be solved is simplistic and a false assumption. The commenter reports that it may often take researchers with expertise in certain statistical methods and having the appropriate statistical software to do so. According to the commenter, epi analyses may need to evaluate complex inter-relationships between variables using time-series, regression, Markov Chain Monte Carlo, Bayesian analysis, and other techniques. The commenter reports that, in contrast, analysis of non-epi datasets typically is completed using summary statistics, and perhaps basic methods to analyze associations between two variables in which case confidential data is more easily removed without affecting the analysis. 
Response: In the final rule the term replicate is not used. The EPA is clarifying that the intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze."
3.1.5.4.3 Use of Complex Tools and Materials in Science Can Make Replicability of Studies Infeasible 
Comment: Commenter (8913) states that many avenues of inquiry which advance modern science use complex tools and materials (like human cell culture models and specially engineered animals) which are not feasibly replicable.
Response: In the final rule the term replicate is not used. The EPA is clarifying that the intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze."
3.1.6 Reproducibility  -  Specific Comments
3.1.6.1 Reproducibility Is Not Always Possible
3.1.6.1.1 Reproducibility as a Standard is Difficult (and Sometimes Impossible)
Comment: Commenter (6144) states that reproducibility as a standard is difficult, and sometimes impossible, to uphold for many types of studies that are done in the environmental health field and that requiring reproducibility is inconsistent with provisions in the OMB Guidelines or Information Quality Guidelines.
Commenter (6915) states that, to the extent the proposal seeks to enable third parties to "re-run" an analysis using the same supporting data and the same models, this may not be possible where proprietary models, methods, designs, and/or data were used in the study. The commenter reports that, as EPA points out, based on the Information Quality Guidelines, in cases where the Agency relies on proprietary models that cannot be made publicly available, the model applications are subject to EPA's peer review policy and other validation checks. The commenter provides that the Information Quality Guidelines indicate that "[t]hese steps, along with transparency about the sources of data used, various assumptions employed, analytic methods applied, and statistical procedures employed should assure that analytic results are `capable of being substantially reproduced.'" 
Commenter (6362) states that it is unclear what constitutes a reproducible versus a non-reproducible finding. The commenter asserts that it is important to consider that there are different types of reproducibility, such as methods reproducibility, results reproducibility, and reproducibility of conclusions. The commenter reports, for example, OMB's Information Quality Guidelines define "capable of being substantially reproduced" as "independent reanalysis of the original or supporting data using the same methods would generate similar analytical results, subject to an acceptable degree of imprecision." According to the commenter, however, the inability to reproduce research studies can be related to issues of study design, variability or differences in biological test systems, data integrity, data analyses, and in some cases, scientific misconduct. The commenter asserts that new or novel findings that purport to indicate effects that have little or no biological basis, based on the weight of the evidence coupled to first principles of relevant scientific disciplines, should be subjected to suitable reproducibility requirements, which could include causal analytics.
Response: The Agency recognizes that the 2018 NPRM used terms that were not fully defined or not defined at all. The 2020 SNPRM addressed a number of these comments. To address the concerns expressed by commenters, in this final rule, in Part 30.2, the EPA is defining the following terms: "data," "dose-response data," "independent validation," "influential scientific information," "pivotal science," "publicly available," "reanalyze," "science that serves as the basis for informing a significant regulatory action," and "significant regulatory actions." The terms "pivotal regulatory science", "replication", "reproducible", and "research data" are not used in this final rule. 
This final rule establishes how the Environmental Protection Agency (EPA) will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information. When promulgating significant regulatory actions or developing influential scientific information, the EPA will give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. The final rule does not require the EPA reproduce or evaluate the reproducibility of a dose-response data.
3.1.6.1.2 Examples Where Reproducibility Is Not Always Possible
Comment: Similar to concerns expressed by many commenters regarding replicability, several commenters (1346, 3131, 4840, 4844, 5170, 6109, 6120, 6144, 6133, 6196, 6868, 6915, 8913) oppose the proposal and assert that the reproduction of studies is not always possible. These commenters generally argue that, while many environmental studies simply cannot or should not be reproduced, the studies provide key insights that should be taken into consideration. 
One-Time Events
Commenter (5170) expresses concern that, while emergency events are essentially not reproducible, the information that is gained from studying these events is incredibly valuable to understanding the health impacts of environmental emergencies as well as non-emergency emissions and discharges. 
Commenters (0563, 4840, 4845, 5170, 6111, 6114, 6116, 6133, 6144, 6196, 6868, 6915, 6924, 8913) provide the following examples of studies and studied events that are not reproducible or would be unethical to reproduce:
          Unique and catastrophic events, such as the Chernobyl accident. (4840, 6144)
          A steel mill shut down. (6868)
          Natural disasters, power plant closures, changes in traffic patterns during events like the Olympic games. (6868)
          Deepwater Horizon. (0563, 4840, 6111, 6924)
          Studies attempting to capture the impacts of one-time events like spills or plant explosions. (4845)
          Emergency events. (5170)
          Oil spill or volcanic eruptions. (5170)
          High levels of smoke from wildfires or high levels of chemicals resulting from industrial accidents or exposures. (6114, 6116)
          Katrina pollutant disaster, the Hurricane Harvey-related toxic chemical discharges, and the Church Rock uranium mill spill. (8913)
Exposure Conditions No Longer Exist and would be Unethical to Reproduce 
Commenters (3131, 6133, 6874, 6932, 6939, 8272) provide the following examples of studies and exposures that would be unethical and immoral to reproduce:
          Public health studies that would subject participants to risk cannot and should not be reproduced. (3131)
          Poor air quality conditions experienced in the past. (6133)
          Epidemiologic studies of chlorpyrifos can no longer be reproduced. (6133, 6932)
          The Mercury EPA Reference Dose (RfD) is based on recommendations of the National Research Council (NRC) of the NAS, that conducted an extensive analysis and calculations derived from three longitudinal epidemiologic studies: the Seychelles Islands, the Faroe Islands, and the New Zealand studies. The commenter states that the use of these studies to set EPA exposure limits was the result of a years-long transparent process of expert scrutiny, public engagement, inter-agency cooperation, and publication in scientific journals. The commenter notes, however, the studies can no longer be reproduced, particularly the Faroe Islands study in which the exposure to the community was a result of eating whales, a practice that has since declined due to public alerts about the hazards of eating the mercury-tainted meat particularly for children and pregnant and breastfeeding women. The commenter adds that it would take decades to repeat the studies, which took decades to conduct in the first place. According to the commenter, EPA estimates that more than 75,000 newborns each year may have increased risk of learning disabilities associated with in-utero exposure to methylmercury, based on maternal blood levels exceeding the EPA RfD. (6133)
          Knowledge about impacts of neurotoxicants has often begun with findings of severe impacts in groups exposed to abnormally high levels of a substance  -  such as developmental delays in babies born to mothers who ate seafood from mercury-contaminated Minamata Bay17  -  and then progressed over future decades as researchers have identified subtler impacts at lower exposure levels.18 Studies that rely on health effect data from "natural experiments," such as those experienced by Minamata residents because of the contaminated wastewater discharged into Minamata Bay, cannot ethically be replicated and may not have publicly available data, but should not be disregarded given that they can serve as an early signal of the need for regulation. (6874, 6939, 8272)
Observational Research
Commenter (6117) states that "methods reproducibility" may be realistic for laboratory-based studies, where it is possible to implement the exact experimental and computation procedures with the same (or very similar) tools to obtain the same results. However, according to the commenter, new clinical or environmental studies may use procedures that are closely matched, but not identical, to previous evaluations. The commenter reports that when it comes to observational research, it is well known that some weaknesses are unavoidable. The commenter explains that it is often difficult to eliminate all potential sources of confounding, or to adjust for the same confounders, across studies from different time periods. The commenter asserts that therefore, it would be unfair to argue that all aspects of a study, including the design and analyses characteristics, need to be reproduced exactly.
Need to Protect Individual Medical Records/Information
          Exposing the identities of participants in the interest of full transparency would be unethical (3131).
          Researchers cannot ethically randomize people to harmful exposures in order to tackle confounding, nor violate informed consent agreements that prohibit open sharing of private data from past studies (6133).
          Commenter (3131) states that failing to utilize data gathered in responsible studies of health records of anonymous participants would be unethical. 
Old Data May No Longer Be Available
Commenters (6196) provide that in some cases data is unavailable because the data is old and predates digital technology where retrieving full study records may be difficult or impossible.
Response: The Agency recognizes that the 2018 NPRM used terms that were not fully defined or not defined at all. The 2020 SNPRM addressed a number of these comments. To address the concerns expressed by commenters, in this final rule, in Part 30.2, the EPA is defining the following terms: "data," "dose-response data," "independent validation," "influential scientific information," "pivotal science," "publicly available," "reanalyze," "science that serves as the basis for informing a significant regulatory action," and "significant regulatory actions." The terms "pivotal regulatory science", "replication", "reproducible", and "research data" are not used in this final rule. 
This final rule establishes how the EPA will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information. When promulgating significant regulatory actions or developing influential scientific information, the EPA will give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. The final rule does not require the EPA reproduce or evaluate the reproducibility of a dose-response data.
3.1.6.2 Comments Supporting Reproducibility as Described in the Proposal
3.1.6.2.1 Recent EPA Action(s) Support the Need for the Proposal
Comment: Commenter (6366) states that recent EPA action on the retrospective review of the Lead Renovation, Repair and Painting Program further reinforces the need for these actions. The commenter reports that, in finalizing the review, EPA claimed a new economic analysis had been completed, yet the data provided cannot be reproduced and is insufficient to fully evaluate the decisions EPA made. The commenter states that because all of the data used to conduct EPA's analysis has not been transparently presented in the associated docket, stakeholders have been unable to fully evaluate the report's underlying economic claims or their resulting impact on the review's conclusion. According to the commenter, this creates continuing challenges for stakeholders as they seek to work with EPA to ensure the program is effective and robust and that all impacts on small entities have accurately been taken into consideration. 
Response: The Agency recognizes that the 2018 NPRM used terms that were not fully defined or not defined at all. The 2020 SNPRM addressed a number of these comments. To address the concerns expressed by commenters, in this final rule, in Part 30.2, the EPA is defining the following terms: "data," "dose-response data," "independent validation," "influential scientific information," "pivotal science," "publicly available," "reanalyze," "science that serves as the basis for informing a significant regulatory action," and "significant regulatory actions." The terms "pivotal regulatory science", "replication", "reproducible", and "research data" are not used in this final rule. 
This final rule establishes how the EPA will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information. When promulgating significant regulatory actions or developing influential scientific information, the EPA will give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. The final rule does not require the EPA reproduce or evaluate the reproducibility of a dose-response data.
3.1.6.2.2 Inability to Reproduce 2010 IRIS Assessment of Formaldehyde
Comment: Commenter (6369) states that transparency, reproducibility and application of current scientific knowledge are paramount to providing the foundation required for sound regulation. The commenter reports that, using the data obtained from the National Cancer Institute (NCI), in combination with data provided in the draft 2010 Integrated Risk Information System (IRIS) assessment for formaldehyde, further detailed analyses were conducted to validate pivotal scientific findings used by EPA's IRIS program to draw conclusions. The commenter provides that the results have been found to disagree with the original study findings and/or the findings cannot be replicated due to methodological issues. 
Commenter (6369) states that, using the data obtained from NCI, in combination with data provided in the draft 2010 IRIS assessment for formaldehyde, further detailed analyses were conducted to validate pivotal scientific findings used by EPA's IRIS program to draw conclusions. The commenter notes that the results have been found to disagree with the original study findings and/or the findings cannot be replicated due to methodological issues. The commenter presents the following examples:
 Mundt et al. conducted additional and refined analysis on the underlying data from the Zhang et al. (2010) study provided by NCI and relied upon in the IRIS assessment to identify formaldehyde as a leukemogen. The Mundt et al. analysis found no association between individual formaldehyde exposure estimates and frequency of aneuploidy, which the original study authors suggested were indicators of myeloid leukemia risk.
 Checkoway et al. sought to replicate the findings reported by Beane Freemen et al. (2009) using the underlying data provided from NCI via the technology transfer agreement. The findings from this re-analysis failed to support the original study hypothesis that formaldehyde causes acute myeloid leukemia. Specifically, the results indicated that acute myeloid leukemia was unrelated to "peak" or any other formaldehyde metric including the conventional cumulative exposure.
 Van Landingham et al. conducted an independent analysis of the dose-response models used by the IRIS program in its chemical assessment of formaldehyde, relying upon the documentation provided in the IRIS assessment. The authors specifically noted that the documentation of the methods applied by EPA lacked sufficient detail for duplication of the risk estimates. This lack of transparency and detail could potentially result in different estimates of risks.
Response: The Agency recognizes that the 2018 NPRM used terms that were not fully defined or not defined at all. The 2020 SNPRM addressed a number of these comments. To address the concerns expressed by commenters, in this final rule, in Part 30.2, the EPA is defining the following terms: "data," "dose-response data," "independent validation," "influential scientific information," "pivotal science," "publicly available," "reanalyze," "science that serves as the basis for informing a significant regulatory action," and "significant regulatory actions." The terms "pivotal regulatory science", "replication", "reproducible", and "research data" are not used in this final rule. 
This final rule establishes how the EPA will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information. When promulgating significant regulatory actions or developing influential scientific information, the EPA will give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. The final rule does not require the EPA reproduce or evaluate the reproducibility of a dose-response data.
3.1.6.3 Comments Opposing Reproducibility as Described in the Proposal
3.1.6.3.1 Reproducing Effects in Multiple Studies versus Reproducing a Particular Study 
Comment: Commenter (9227) asserts that reproduction does not enhance transparency. The commenter reports that the majority of research on the health effects of environmental hazards fall into this category. According to the commenter, a series of studies that address the same hypothesis and give the same basic result does indeed strengthen findings of toxicity. The commenter states that an example of how a study is reproduced is the Harvard Six Cities and ACS CPSII studies on mortality that were published in 1993 and 1995, respectively., The commenter provides that the investigators of the Harvard Six Cities study, along with others, reproduced their finding in a separate assessment of the association between fine particle levels and mortality. 
Commenter (9227) states that reproducing effects in multiple studies that are not identical is the basis for almost all scientific decisions on environmental issues and should be the focus of the EPA's approach to regulatory science. 
Response: The Agency recognizes that the 2018 NPRM used terms that were not fully defined or not defined at all. The 2020 SNPRM addressed a number of these comments. To address the concerns expressed by commenters, in this final rule, in Part 30.2, the EPA is defining the following terms: "data," "dose-response data," "independent validation," "influential scientific information," "pivotal science," "publicly available," "reanalyze," "science that serves as the basis for informing a significant regulatory action," and "significant regulatory actions." The terms "pivotal regulatory science", "replication", "reproducible", and "research data" are not used in this final rule. 
This final rule establishes how the EPA will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information. When promulgating significant regulatory actions or developing influential scientific information, the EPA will give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. The final rule does not require the EPA reproduce or evaluate the reproducibility of a dose-response data.
3.1.6.3.2 Articles Cited by the Proposal as Supporting Reproducibility Do Not Support Proposal
Comment: Commenter (6916) states that the articles the Agency cites on the reproducibility of studies do not support its Proposal, and EPA does not explain why it thinks they do. The commenter reports that at least one of the cited articles, by Steven Goodman, is entirely irrelevant - it does not demonstrate that there is a problem with the reliability of science in support of Agency decision making, nor does it attempt to; rather, it discusses the confusion in the lexicon about reliability, and attempts to more properly define the term "reproducibility." 
According to the commenter (6916), the industry-funded Mercatus Center paper cited by the Agency does not support the proposition that it is necessary to limit or forbit EPA's use of scientific studies where the data are not publicly available. The commenter reports that Mercatus recommends an avenue for the Agency to follow where there is a need to reproduce or validate study results and the researchers cannot release the underlying data, but the "[A]gency still believes that relying on the results of the study is warranted." Commenter states that, in that situation, the paper recommends that the Agency would provide a more extensive explanation of why it has "sufficient confidence to use the study." 
Response: The Agency recognizes that the 2018 NPRM used terms that were not fully defined or not defined at all. The 2020 SNPRM addressed a number of these comments. To address the concerns expressed by commenters, in this final rule, in Part 30.2, the EPA is defining the following terms: "data," "dose-response data," "independent validation," "influential scientific information," "pivotal science," "publicly available," "reanalyze," "science that serves as the basis for informing a significant regulatory action," and "significant regulatory actions." The terms "pivotal regulatory science", "replication", "reproducible", and "research data" are not used in this final rule. 
This final rule establishes how the EPA will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information. When promulgating significant regulatory actions or developing influential scientific information, the EPA will give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. The final rule does not require the EPA reproduce or evaluate the reproducibility of a dose-response data.
3.1.6.4 Reproduction of Studies Would be Burdensome
Comment: Commenter (1346) states that some studies, especially in public heath, are so large and long-term that they could not realistically be repeated and, even if they could be repeated, doing so would delay whatever decisions hinge on them by decades or more, potentially risking the health and lives of millions of people.
Commenter (4840) states that, while in theory most studies could be reproduced, they rarely are because it is a waste of resources. The commenter provides that the scientific enterprise involves approaching the same question in different ways to determine if the results support each other. 
Response: The Agency recognizes that the 2018 NPRM used terms that were not fully defined or not defined at all. The 2020 SNPRM addressed a number of these comments. To address the concerns expressed by commenters, in this final rule, in Part 30.2, the EPA is defining the following terms: "data," "dose-response data," "independent validation," "influential scientific information," "pivotal science," "publicly available," "reanalyze," "science that serves as the basis for informing a significant regulatory action," and "significant regulatory actions." The terms "pivotal regulatory science", "replication", "reproducible", and "research data" are not used in this final rule. 
This final rule establishes how the EPA will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information. When promulgating significant regulatory actions or developing influential scientific information, the EPA will give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. The final rule does not require the EPA to reproduce or evaluate the reproducibility of a dose-response data.
3.1.6.5 Other Comments Regarding Reproducibility
3.1.6.5.1 Standards for Research Reproducibility
Comment: Commenters (5015, 6117) request and suggest recommendations for study reproducibility standards/framework be established.
Commenter (6117) states that, although the proposed regulation outlines the importance of "reproducibility", it does not provide a framework for how the reproducibility of an individual study can or should be assessed. 
Commenter (6117) asserts that it is important to have a clear understanding of research reproducibility, so that different standards (i.e., "case-by-case" decisions) are not applied to individual studies. The commenter suggests there are three key areas of reproducibility: 
 Methods reproducibility 
 This is when a study provides enough information about the experimental and/or computational procedures so that future authors can repeat the study using the same data to obtain the same results.
 Results reproducibility 
 This is when a new study produces corroborating results using experiment methods that are closely matched to a previous study.
 Inferential reproducibility 
 This is when a new study draws qualitatively similar conclusions from either an independent replication or re-analyses.
Commenter (5015) states that reproducibility is a critical challenge for science: can the results of an important study be reproduced? The commenter suggests that the most effective way to test the reproducibility and validity of scientific results is not necessarily to simply reproduce the same results in the same data sets --because that also reproduces all the weaknesses and limitations of the original study. According to the commenter, it is most important to answer the following questions: 

   Are the results consistent when tested in other independent studies...
 That use new and different data not affiliated with the original studies?
 Have different investigators applying the same and/or alternative statistical techniques?
 And test the sensitivity of the results against a wide range of possible other explanations, e.g. smoking behavior, socioeconomic status, access to medical care, and more.
Response: The Agency recognizes that the 2018 NPRM used terms that were not fully defined or not defined at all. The 2020 SNPRM addressed a number of these comments. To address the concerns expressed by commenters, in this final rule, in Part 30.2, the EPA is defining the following terms: "data," "dose-response data," "independent validation," "influential scientific information," "pivotal science," "publicly available," "reanalyze," "science that serves as the basis for informing a significant regulatory action," and "significant regulatory actions." The terms "pivotal regulatory science", "replication", "reproducible", and "research data" are not used in this final rule. 
This final rule establishes how the EPA will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information. When promulgating significant regulatory actions or developing influential scientific information, the EPA will give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. The final rule does not require the EPA to reproduce or evaluate the reproducibility of a dose-response data.
3.1.6.5.2 "Reproduced" Epidemiological Study Results
Comment: Commenter (6915) states that, in contrast to the controlled conditions of animal studies, there is more variability among humans than the strains of lab animals and the exact underlying conditions of human studies can rarely be exactly replicated even when the same protocols are followed. The commenter concludes that even a contradictory result in a "reproduced" epidemiological study would not necessarily invalidate an observation from an earlier study, provided that the first study followed valid methods and conducted appropriate statistical analyses.
Response: The Agency recognizes that the 2018 NPRM used terms that were not fully defined or not defined at all. The 2020 SNPRM addressed a number of these comments. To address the concerns expressed by commenters, in this final rule, in Part 30.2, the EPA is defining the following terms: "data," "dose-response data," "independent validation," "influential scientific information," "pivotal science," "publicly available," "reanalyze," "science that serves as the basis for informing a significant regulatory action," and "significant regulatory actions." The terms "pivotal regulatory science", "replication", "reproducible", and "research data" are not used in this final rule. 
This final rule establishes how the EPA will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information. When promulgating significant regulatory actions or developing influential scientific information, the EPA will give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. The final rule does not require the EPA to reproduce or evaluate the reproducibility of a dose-response data.
3.1.6.5.3 "Reproduced" Associations/Findings
Comment: Commenter (9227) states that a large body of literature shows that the association of fine particle pollution and mortality has been reproduced in different populations across the globe, over different periods of time, contexts and using different methods. The commenter states that it is this accumulation of evidence of reproducible effects in multiple studies that is critical in determination of causality and validation of an effect and is already an integral part of the EPA process of supporting causality. 
Response: The Agency recognizes that the 2018 NPRM used terms that were not fully defined or not defined at all. The 2020 SNPRM addressed a number of these comments. To address the concerns expressed by commenters, in this final rule, in Part 30.2, the EPA is defining the following terms: "data," "dose-response data," "independent validation," "influential scientific information," "pivotal science," "publicly available," "reanalyze," "science that serves as the basis for informing a significant regulatory action," and "significant regulatory actions." The terms "pivotal regulatory science", "replication", "reproducible", and "research data" are not used in this final rule. 
This final rule establishes how the EPA will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information. When promulgating significant regulatory actions or developing influential scientific information, the EPA will give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. The final rule does not require the EPA to reproduce or evaluate the reproducibility of a dose-response data.
Comment: Commenter (1346) states that, while not all studies are fully reproducible, we can replicate the results using those data anyway or can use other methods to analyze those data and compare the results.
Response: The Agency recognizes that the 2018 NPRM used terms that were not fully defined or not defined at all. The 2020 SNPRM addressed a number of these comments. To address the concerns expressed by commenters, in this final rule, in Part 30.2, the EPA is defining the following terms: "data," "dose-response data," "independent validation," "influential scientific information," "pivotal science," "publicly available," "reanalyze," "science that serves as the basis for informing a significant regulatory action," and "significant regulatory actions." The terms "pivotal regulatory science", "replication", "reproducible", and "research data" are not used in this final rule. 
This final rule establishes how the EPA will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information. When promulgating significant regulatory actions or developing influential scientific information, the EPA will give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. The final rule does not require the EPA to reproduce or evaluate the reproducibility of a dose-response data.
3.1.7 Replication (or Reproduction) Crisis
3.1.7.1 Replication (or Reproduction) Crisis
Comment: Commenters (0039, 0563, 0844, 1486, 2850, 4541, 6867, 6921) refer to a replication (or reproduction) crisis as evidence that supports the need for the proposed rule. 
Commenter (0563) states that the rule is important to the integrity of EPA regulations because of a widely recognized crisis: scientific studies, including those subjected to peer review, frequently are found to be irreproducible or to have false findings because they fail to meet high standards for data archiving, valid statistical analysis, experiment procedures, and other elements of valid research. 
Commenter (0039) supports the EPA's prioritization of the application of reproducibility reforms in the area of dose-response regulation. The commenter states that many supposedly scientific results cannot be reproduced in subsequent investigations. The commenter expresses support for the EPA choosing to address concerns about the irreproducibility crisis of modern science. The commenter refers to a report they have written on how the improper use of statistics, arbitrary research techniques, lack of accountability, political groupthink, and a scientific culture biased toward producing positive results together have produced a reproducibility crisis that afflicts a wide range of scientific and social-scientific disciplines, from epidemiology to social psychology. The commenter recommends extensive changes to scientific procedures and to the way government judges the science it uses to make policy--including measures such as the proposed rule, to require that government make policy only based on scientific research whose data and procedures are available for other scientists to reproduce. According to the commenter, the larger concerns about reproducibility suggest that this measure should be applied generally within the EPA and across the Federal Government.
Commenter (0844) states that science should be reproducible to be reliable. The commenter cites a February 22, 2017 British Broadcasting Corporation (BBC) News report [no other citations], "Most scientists can't replicate studies by their peers," that recounted that, "more than two-thirds of researchers have tried and failed to reproduce another scientist's experiments." The commenter reports that, according to the article, Dr. Tim Errington, who runs the Reproducibility Project at the University of Virginia, attempted to repeat the findings in five landmark cancer studies but was only able to confirm two of the original studies' findings. The commenter provides that Dr. Errington said, "it's worrying because replication is supposed to be a hallmark of scientific integrity." The commenter states that Dr. Errington's comment does not just apply to cancer research, it also applies to all research, including environmental and climate studies. The commenter further states that, although the Union of Concerned Scientists (UCS) letter argued that some of the studies could not be replicated because it would require "intentionally and unethically exposing people and the environment to harmful contaminants or recreating one-time events," it is conceivable that most, if not all studies could be replicated in the lab with research animals, since this type of research is conducted all the time, or could be duplicated using other scientific models.
Commenter (1486) states that there is a wealth of publicly available information that documents the pervasiveness of irreproducible studies across the spectrum of scientific disciplines. However, the commenter does not provide any examples but suggests the literature be reviewed. The commenter provides that the public health sciences (particularly epidemiology and toxicology) are a notable segment of that problem but are not the only offenders. The commenter states that recent history shows that some decisions regarding national health policy were made based upon questionable science, have not resulted in the desired outcomes, and have eroded the public's trust in the wisdom of its government.
Commenter (2850) states that, in the last 10 years, a terrible crisis in science has come to light that many published research findings are false. The commenter refers to work conducted by John P.A. Ioannidis. The commenter asserts that it is of the utmost importance that the science used by the government to create regulations is transparent. According to the commenter, the time that politicians and scientists were trusted to have the highest integrity so that the population didn't feel the need to check them is long gone. 
Commenter (4541) states that EPA's admitted reliance on "secret science" (science that cannot be properly vetted and evaluated for reliability) occurs at a time when Nature, PLOS, Science, The Economist, and others report half or more of published research on public health issues cannot be replicated. The commenter states that this "replication crisis" is genuine, and even more broad and critical than the sources cited by EPA for this proposed Rule are willing to admit. The commenter argues that a scientific publishing industry has been created by lavish government funding of politically directed research. The commenter provides examples as including studies finding a human impact on climate, or an association between ambient small particle and ozone levels in the air and public health. The commenter states that it may take generations before the effects of this scientific misconduct can be overcome.
Commenter (4541) believes that the root cause of EPA's science malfunction has been corruption of EPA's peer review process and intentional adoption of scientific methods that exaggerate public health and environmental risks. According to the commenter, peer review for EPA has become "pal review," with insiders funded by EPA grants reviewing the work of fellow EPA-funded scientists. The commenter believes that both sides of this "crony" arrangement, individuals as well as their institutions, are beneficiaries of millions of dollars in government funding. The commenter states that EPA-sponsored reviewers censure and exclude scientists who disagree with the agency's reigning political agenda. According to the commenter, that perverts the whole point of peer review, turning it into a tool used to shut out anyone who disagrees instead of a process forcing scientists to defend their work against critics. The commenter provides that the widespread replication crisis is a strong argument for the proposed Rule, since unreliable science has affected most of the world's leading science journals and professional journals as well as national academies of science. The commenter asserts that it is naïve to think such scientific malfeasance does not impact politically hot subject areas that are also hot spots for generous government. 
Commenter (6867) states that in the light of overwhelming evidence that a significant portion of studies published in prominent peer reviewed journals are not reproducible, it is dubious to claim that any research is valid and relevant unless -- at a minimum -- other researchers and the public have access to the underlying data, methodology, and computational code. The commenter asserts that pre-publication peer review is not an adequate substitute for public availability of data, methodology, and computational code. The commenter reports that, in response to the Proposed Rule, some commenters have suggested the Proposed Rule is an attempt to remove valid and relevant scientific evidence from the rule-making process. According to the commenter, that empirical evidence indicates that peer reviewers routinely fail to identify even major errors. The commenter asserts that peer review in less prominent journals may often occur in name only. Commenter argues that the best available metascience -- science about science -- indicates that pre-publication peer review is not adequate to ensure the validity of published scientific claims. 
Commenter (6867) provides that there is strong evidence of widespread, outcome-altering errors in the computational code underlying many scientific studies. The commenter states that outcome-altering errors in computational code have been suggested as a significant contributor to the replication crisis. The commenter reports that high profile retractions, technical comments, and corrections because of coding errors include papers in prominent journals such as Science, PNAS, the Journal of Molecular Biology, Ecology Letters, Journal of Mammalogy, Journal of the American College of Cardiology, Hypertension, and American Economic Review. The commenter expresses concern with the quality of the following published science:
       2005, when a study of 45 highly-cited articles in New England Journal of Medicine, JAMA, and Lancet concluded that at least 7 articles (16%) were contradicted by subsequent research and another 7 articles (16%) claimed stronger effects than were supported by subsequent research.
       A widely-read essay published later that year suggested that most published research findings are false.
       Consistent with that suggestion, Bayer Healthcare disclosed in 2011 that 43 (65%) of the company's attempts to reproduce 67 published studies resulted in inconsistent data.
       In 2012, Amgen, Inc., similarly disclosed that it was only able to reproduce 6 (11%) of 53 studies that the company attempted to confirm. 
       Consistent with the Bayer and Amgen disclosures, recent estimates for irreproducibility in preclinical and biomedical research range as high as 90% of that research, even for articles published in high-quality journals.,,
       A recent survey of 804 ecologists and evolutionary biologists found that questionable research practices were widespread, with 64% of surveyed researchers reporting they had at least once failed to report results because they were not statistically significant ("cherry picking"); 42% reporting they had collected more data after inspecting whether results were statistically significant (a form of "p-hacking"); and 51% acknowledging they had reported an unexpected finding as though it was hypothesized from the start.
       A 2016 survey by the journal Nature found that lack of reproducibility is a widespread concern among scientists.,
Commenter (6921) states that the Proposed Rule appropriately recognizes the importance of reproducibility, especially given concerns over the "reproducibility crisis" affecting many scientific studies. The commenter believes that EPA's rulemaking and focus on increasing transparency is critical given the significance of EPA's reliance on published research and the magnitude of the potential replication problem. The commenter reports that many researchers believe that the reproducibility of published experiments is the foundation of science. According to the commenter, Federal policies that exhort transparency but that do not materially affect ongoing practices may be more harmful by giving the appearance of increased transparency without the result. 
Commenter (6921) reports that the inability to access underlying research data or models has become a source of increasing concern in the wake of recent reports of high replication failures rates. Commenter states that in 2011, Bayer HealthCare in Germany published a report reporting that only about 25 percent of 67 published preclinical studies could be validated to the point at which the projects would continue. Commenter states that one year later, a second pharmaceutical firm, Amgen, found the reported scientific findings could only be confirmed in 6 of 53 studies. Commenter states that in 2016, Nature published the results of a survey that found that more than 70 percent of researchers have tried and failed to reproduce another scientist's experiments, and more than half have failed to reproduce their own experiments. Commenter states that, while some level of replication failure can be expected simply based on statistical probabilities, the higher than expected failure rates are of concern especially when considering the use of research studies to support policy decisions.
 Commenter (2169) suggests that reproducibility should be assessed alongside its parallel  -  the "producibility crisis." The commenter provides that rather than review and consider studies available to it, the EPA has often swept them aside, failing to consider the legitimate concerns raised by these scientific inquiries. The commenter provides information regarding particulate matter (PM) in six attachments and states that every attempt to get EPA to focus on their concerns regarding PM has been met by resistance or have been simply disregarded by EPA, who continues to rely on the now outdated 2005 PM study. The commenter also reports that a request for a copy of the Agency for Toxic Substances and Disease Registry (ATSDR) assessment of a class of toxic chemicals that has contaminated water supplies near military bases, chemical plants and other sites from New York to Michigan to West Virginia was recently denied.
Response: Numerous public commenters on this rulemaking and some members of the EPA's SAB noted concern about a "crisis" in science due to such low reported success rates and noted support for the rulemaking as a potential solution. The EPA agrees with commenters that the low success rates published in reanalysis studies are potentially worrisome and also agrees with the National Academies of Sciences and others that the paucity of full data sharing makes it difficult to even systematically evaluate the extent of the problem within or across scientific disciplines. One barrier is having access to enough of the underlying data, models, and methods to be able to conduct the independent validation. Encouraging greater transparency and data and model sharing, then, is vital for individuals outside of the publication of the study to be able to verify the quality and strength of published studies and to identify and correct suboptimal research and publication practices, such as positive bias, p-hacking, and post hoc reasoning that may influence the results presented in a publication.,,,,,,, 
The EPA disagrees with commenters who believe the rule is unnecessary because there is little evidence of a widespread reanalysis issue in science. Collectively, evidence from published reanalysis studies, retractions, surveys of scientists, and public commenters provides sufficient evidence that additional policies that remove barriers to data availability and reanalysis and create greater transparency are needed. The EPA's SAB, several public commenters, and authors of peer-reviewed literature have noted that data sharing, like that envisioned under this rulemaking, would strengthen science. The scientific benefits of openness in research are well-recognized. Open sharing of data and models reduces duplication of effort from different researchers attempting to collect the same datasets and allows for the use and reuse of data and models among the research community. Scientific openness also improves the ability for the scientific community to validate research results, helping to prevent errors and reducing the risk of fraud in science and research. Also, when data and models are made widely available, researchers can re-analyze the data for new and different purposes, examining novel questions and providing new scientific insights. Some studies indicate that data sharing can actually increase the number of citations by third-party researchers, which can indicate higher influence of the original research.,, Collaboration and communication, both within the scientific community and between researchers and other organizations or individuals, are also enhanced by sharing of data and models.
3.1.7.2 No Evidence to Support the Existence of a Replication (or Reproduction) Crisis
Comment: Commenters (1589, 6127, 6135, 6137, 6158, 6450, 6924, 6925, 6955, 8771-PH56) report that there is no demonstrated evidence or proposed rule record support indicating the existence of a replication (or reproduction) crisis. 
Commenters (1589, 6135, 6137, 6158, 6955, 8771-PH56) state there is no demonstrated "replication crisis" in environmental health. 
Commenter (6158) states that, while there is some concern, broadly speaking, that scientific findings are often difficult to replicate, the writers of the proposed rule have made no credible case that this issue is particularly severe within the public health or environmental studies used by the EPA. 
Commenter (6135) states that the proposed rule fails to provide any evidence of a replication crisis among the types of studies that have traditionally been used as the basis for air quality regulations. 
Commenter (6955) points out that the Agency proposal ties its rationale for the access rule to replication problems identified in some scientific fields but has provided no record of a problem in the fields of environmental studies, chemistry, or biochemistry. 
Commenter (6137) states that, while there are discussions within certain scientific fields, such as clinical psychology, about inconsistent results that have been obtained when experiments conducted in those fields are repeated, EPA has not explained why these discussions should cast doubt on the dose-response data or models used by the Agency. 
Commenter (8771-PH56) argues that, just because the results of some psychological experiments have proven to be accidental or artifactual does not mean EPA faces any kind of "replication crisis"
Commenter (6924) states that there is no documentation to support that there is a replication crisis related to reanalysis of data.
Commenter (6955) states that this rule would make a radical, broad-brush change in agency policy easily challenged in court, particularly when recommendations for improving reproducibility in the beleaguered fields focus, not on public access of data, but on pre-registration of studies, better use of statisticians, an end to sole reliance on statistical significance and p-values as an indicator of merit, and increased incentives to publish null results  -  measures that the EPA proposal does not address. The commenter notes that the identified problems will not be solved by post-hoc analysis of a study's data. In fact, the commenter contends that unless careful concern for post- hoc statistics is made part of the analysis, post-hoc studies in regulatory proceedings will make the chances of reaching a good decision worse. 
Response: Numerous public commenters on this rulemaking and some members of the EPA's SAB noted concern about a "crisis" in science due to such low reported success rates and noted support for the rulemaking as a potential solution. The EPA agrees with commenters that the low success rates published in reanalysis studies are potentially worrisome and also agrees with the National Academies of Sciences and others that the paucity of full data sharing makes it difficult to even systematically evaluate the extent of the problem within or across scientific disciplines. One barrier is having access to enough of the underlying data, models, and methods to be able to conduct the independent validation. Encouraging greater transparency and data and model sharing, then, is vital for individuals outside of the publication of the study to be able to verify the quality and strength of published studies and to identify and correct suboptimal research and publication practices, such as positive bias, p-hacking, and post hoc reasoning that may influence the results presented in a publication.,,,,,,, 

The EPA disagrees with commenters who believe the rule is unnecessary because there is little evidence of a widespread reanalysis issue in science. Collectively, evidence from published reanalysis studies, retractions, surveys of scientists, and public commenters provides sufficient evidence that additional policies that remove barriers to data availability and reanalysis and create greater transparency are needed. The EPA's SAB, several public commenters, and authors of peer-reviewed literature have noted that data sharing, like that envisioned under this rulemaking, would strengthen science. The scientific benefits of openness in research are well-recognized. Open sharing of data and models reduces duplication of effort from different researchers attempting to collect the same datasets and allows for the use and reuse of data and models among the research community. Scientific openness also improves the ability for the scientific community to validate research results, helping to prevent errors and reducing the risk of fraud in science and research. Also, when data and models are made widely available, researchers can re-analyze the data for new and different purposes, examining novel questions and providing new scientific insights. Some studies indicate that data sharing can actually increase the number of citations by third-party researchers, which can indicate higher influence of the original research.,, Collaboration and communication, both within the scientific community and between researchers and other organizations or individuals, are also enhanced by sharing of data and models.
Comment: Commenters (1589, 6127, 6137, 6450, 6925, 8771-PH56) argue that the citations in the proposed rule provided as support that there is a replication crisis do not support the proposal. 
Comment (1589) states that the proposed rule incorrectly and falsely uses data from several primary sources to support its suppositions and conclusions, which does not hold up to the scientific process or any logical scrutiny. The commenter reports that the proposed rule uses several citations concerning the so-called "replication crisis" in science to argue for its requirement. According to the commenter, upon reading those citations, one would find that the citations are to studies that indicate that the critiques are already being addressed by the scientific community (https://www.nature.com/articles/s41562-016-0021 and http://science.sciencemag.org/content/343/6168/229.long), and that published articles respond to articles calling into question the validity of methods and conclusions (http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124), to opinion pieces from popular magazines rather than from qualified primary sources involved in the type of scientific work being described (https://www.economist.com/news/leaders/21588069-scientific-research-has-changed-world-now-it-needs-change-itself-how-science-goes-wrong), and to articles simply about improving the nomenclature of data reproducibility to improve understanding of uncertainty (http://stm.sciencemag.org/content/8/341/341ps12.full). 
Commenter (6137) states that the Proposed Rule claims  -  with no support  -  that the policies of major scientific journals that purportedly informed the development of the Proposed Rule were "spurred in some part by the `replication crisis.'" The commenter reports that the term "replication crisis" does not occur in any of the sources cited by EPA in support of this clause. The commenter contends that the word "crisis" only appears in one of the sources, which refers to a "reproducibility crisis" and they notes that it is "debatable" whether this term is appropriate. According to the commenter, although EPA provides web addresses to three commentaries authored by Dr. John Ioannidis of Stanford University, an editorial in the journal Science, and an editorial by The Economist newspaper, EPA does not explain why any of the observations made in these commentaries or editorials are relevant to dose-response data and models. The commenter reports that the authors cited by EPA have sharply criticized the Proposed Rule. The commenter provides that:
 Dr. Ioannidis wrote, "If the proposed rule is approved, science will be practically eliminated from all decision-making processes. Regulation would then depend uniquely on opinion and whim." 
 Science editorialized, "Here, a push for transparency appears actually to be a mechanism for suppressing important scientific evidence in policy-making, thereby threatening the public's well-being." 
 The Economist has described the proposal as part of "a campaign to stifle science at the EPA." As they put it, "[a]ir-quality rules and pesticide limits rely on analyses of confidential medical records -- which Mr. Pruitt may now label suspect and try to undo."  
Commenter (6450) explains that, while EPA claims that the proposal's "policies are informed by the policies recently adopted by some major scientific journals, spurred in some part by the `replication crisis,'" those same science journals  -  Science, Nature, and the Proceedings of the National Academy of Sciences  -  have issued a strong statement in opposition to the proposed rule: 
      It does not strengthen policies based on scientific evidence to limit the scientific evidence that can inform them; rather, it is paramount that the full suite of relevant science vetted through peer review, which includes ever more rigorous features, inform the landscape of decision making.
Commenter (6450) states that, while the proposal also cites a 2013 editorial in The Economist to argue that there is a "replication crisis" plaguing public health studies, ironically, the author is anonymous. According to the commenter, misinterpreting the data policies of peer-reviewed scientific journals and citing an anonymous editorial from five years ago is not sufficient evidence to justify such a sweeping proposal.
Commenter (6925) reports there is no commonly recognized "replication crisis" in science, and EPA's casual mention of it in this proposed rulemaking appears a blatant attempt to cast doubt on the current state of peer-reviewed science. The commenter states that the study cited in support of this assertion is a 13-year-old paper that made the sweeping claim that "most current published research findings are false" without much further backing of this claim. The commenter contends that more troubling though is EPA's stretching the truth regarding its characterization of support of this notion from the recommendations of several third-party organizations advocating for open data. The commenter reports that for at least two of these organizations, the sole author of one of the studies and co-author of the other stated that, referring to EPA's proposal, "[t]hey don't adopt any of our recommendations, and they go in a direction that's completely opposite, completely different." The commenter asserts that, in citing these sources, EPA attempts to cast doubt on the scientific method and build support for solving a fabricated "crisis" when there is in fact none.
Commenter (8771-PH56) states that, of the five URLs EPA provides in its Footnote 12 to document its claim that there is a "replication crisis," two are broken links and the other three discuss psychology studies and clinical trials. The commenter states that the endpoints in epidemiology, toxicology, and exposure studies are simply not as subjective as psychology experiments are. The commenter further states that some researchers have found replication problems with clinical trials of drugs and therapies, but (1) these typically have smaller sample sizes than epidemiologic studies; and (2) unmeasured interindividual variability is likely much more important with respect to whether a drug will work than whether a pollutant is harmful. According to the commenter, they would not be surprised if a complicated molecule for cancer immunotherapy would cure 10% of patients and be ineffective in 90% of others -- which means that different study groups could easily show widely differing efficacy rates -- but adds (for example) that methylene chloride causes carboxyhemoglobin formation in everybody, and we understand how and why and the reasons we would not expect different groups to have different risks if we kept replicating these exposures. The commenter provides that, most importantly, EPA has cited no studies offering even a guesstimate of what percentage of environmental science studies might need replication or reanalysis.
Response: The EPA disagrees with commenters who believe the rule is unnecessary because there is little evidence of a widespread reanalysis issue in science. Collectively, evidence from published reanalysis studies, retractions, surveys of scientists, and public commenters provides sufficient evidence that additional policies that remove barriers to data availability and reanalysis and create greater transparency are needed. The citations discussed in the comments above are no longer referenced in the final rule.
3.1.7.3 Other Comments on the Replication (or Reproduction) Crisis
Comment: Commenter (6135) states that, while the background of the proposed rule does mention a general concern regarding the reproducibility of scientific studies, it fails to acknowledge the extensive efforts that have already been enacted by funding agencies, scientific journals, and scientific societies to ensure the transparency and reproducibility of biomedical research. According to the commenter, given the procedures already in place to ensure scientific integrity it would be redundant and inefficient for the EPA in enact the proposed rule or to act as a clearinghouse for data that are already generally available.
Commenter (2906) states that barriers to reproducibility are not new, and despite them, the pace of discovery has been accelerating decade after decade. The commenter contends that recent increased attention to the challenges in repeating and extending results of individual published papers is giving rise to new tools and practices to further strengthen the scientific endeavor. The commenter provides that over time, given the self-correcting nature of research, science does uncover truth. The commenter states that, while any single result may be difficult to reproduce, with evidence and consensus, novel discoveries are made, and theories are confirmed.
Response: The EPA disagrees with commenters who believe the rule is unnecessary or redundant. Collectively, evidence from published reanalysis studies, retractions, surveys of scientists, and public commenters provides sufficient evidence that additional policies that remove barriers to data availability and reanalysis and create greater transparency are needed.
The EPA is a leader in the evaluation of human and ecological impacts of environmental pollution and methods to cost-effectively mitigate these impacts. The selection and use of quality, peer-reviewed literature is a critical part of this work. However, the strength of the review conducted as part of the study's acceptance and publication in a scientific journal has received increased attention in the scientific community due to the lack of transparency and consistency, potential for bias, and the difficulty that independent researchers have had in reproducing published results.,,,,, Jamieson et al. (2019) state, "Journals play a key role in ascertaining the quality of the evidence in work submitted for publication through peer review. However, that practice is neither uniform nor free of exploitation. While it can select for trustable results, its existence alone is not sufficient to verify the quality of the evidence used to back claims." Because of the potential impact of significant regulatory actions, independent validation of studies that are pivotal to the EPA's significant regulatory actions and influential scientific information may increase public confidence in the findings and conclusions presented in those studies. As Jamieson et al. (2019) states, publication alone (including the typical peer review necessary for publication) is insufficient to verify a study's quality, and additional confidence could be gained from third-party reanalyses that independently validate the study results.
Many scientific publications do require authors to make a data availability or data access statement, which discloses where and under what conditions the underlying study data are available. Yet the EPA cannot solely rely on data availability statements made in published research because initiatives toward greater data sharing and transparency amongst scientific journals and international organizations are still being implemented, are inconsistently enforced, and the true accessibility of data in a public repository is still limited. For example, Christensen et al. (2019) evaluated 1,072 peer-reviewed articles and "found that rates of data availability for empirical articles published after journals adopted data-sharing policies differ widely between journals, from 0 percent to 83 percent, with a mean of 35 percent."
Comment: Commenters (6158, 6448) recommend that EPA better articulate how the proposed rule addresses replication concerns beyond existing practices in support of science-based decision-making. 
Commenter (6448) states that, while it is appreciated that EPA is concerned about the "replication crisis," it is unclear how these concerns are, or even if they should be, incorporated into the proposed rule. The commenter provides that utilization of the "best available science" and expert peer review are processes that EPA has long integrated to minimize the impact of irreproducible results in regulatory decisions. 
 Response: The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. The EPA will then identify and consider "pivotal science in accordance with the provisions of this rule," unless the implementation of the rule conflicts with statutory requirements and associated implementing regulations.
3.1.8 Reanalysis  -  Specific Comments
3.1.8.1 Reanalysis Does Not Require Disclosing Participant Information/Confidential Data
Comment: Commenters (1973, 2336, 5015, 6153, 6874, 6916, 8272, 9227) refer to the HEI 2000 report which conducted an independent reanalysis of the data from two landmark studies that have informed the NAAQS for fine particulate pollution: the Harvard Six Cities Study and the American Cancer Society (ACS) Study. The commenters assert that the HEI report was a successful reanalysis study which retained the participants' information confidential. 
Commenter (1973), referring to the HEI study as well as Needleman's 1979 study of lead and IQ, states that reanalysis of some key studies is something that has already been done, without the damaging requirements for public access that EPA has proposed. 
Commenter (6117) states that rigorous reanalysis and replications are possible without mandating public availability of all data. 
Commenter (9227) states that, to the extent that specific circumstances justify replicating a study, EPA fails to explain why it is necessary to make a study's underlying data broadly available to the public rather than employing a more secure approach that protects personal privacy, such as the approach taken in the HEI reanalysis study.
Commenter (2336) states that when the EPA introduced the new NAAQS standard for PM2.5 in 1997, some in Congress, industry and the scientific community debated the advantages and disadvantages of requiring public access to data much like the one that surrounds this proposed rule. The commenter reports that, to address these concerns, Harvard and the American Cancer Society asked HEI to organize an independent reanalysis of the data from these studies, on condition that HEI and members of the reanalysis team would agree to keep the study participants' information confidential. Commenters (6153, 6874, 8272) state that, instead of making data public, investigators agreed to supply it to the HEI, which used a competitive process to select a team to reanalyze the data. 
Commenters (2336, 5015, 6153, 6874, 8272) state that, in 2000, HEI reported that the reanalysis "assured the quality of the original data, replicated the original results, and tested those results against alternative risk models and analytic approaches without substantively altering the original findings of an association between indicators of particulate matter air pollution and mortality." 
Commenter (9227) contends that the provisions in the proposal appear to preclude EPA from using the Harvard Six Cities Study because the underlying data is not publicly available due to patient confidentiality protections bound by individual contractual agreements between the scientists and the research participants and by the HIPPA and that such an exclusion is inconsistent with provisions in the OMB's data quality guidelines that use the Harvard Six Cities Study as an example of how data may be validated or corroborated without public release of the underlying raw data.
Commenter (9227) reports that the HEI reanalysis example demonstrates that it is possible to undertake a reanalysis without making underlying data broadly available to the entire public. The commenter expresses concern that EPA's proposed rule apparently would bar regulators from relying on these high quality and extensively vetted studies because the underlying data was never made publicly available. The commenter states that EPA does not -- and cannot -- explain how a rule that would prohibit the agency from considering these seminal, high quality scientific studies comports with its goal of strengthening the agency's use of science in regulatory actions. 
Response: The EPA agrees that it is important not to categorically exclude any study because the data underlying a study at the stage required by this rulemaking may not be available for independent validation. Therefore, the EPA is not finalizing the primary proposal in the 2020 SNPRM that would have categorically required that for studies to be considered pivotal science the underlying data would need to be available for independent validation. However, given that transparency is an important aspect of EPA's regulatory actions and assessments, it should be an important consideration in how the Agency considers pivotal science. As described in 40 CFR 30.5 of the final rule, the EPA will rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies --  based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements or quantitative analyses of an EPA significant regulatory action or influential scientific information will be identified as pivotal science.
 Further, the EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (2336) suggests that the rule, in its final form, should provide an avenue for reanalysis of studies employing confidential data by objective third parties who access confidential input data under appropriate protective conditions. The commenter suggests that, only when that approach is taken, and reanalysis uncovers material defects in the data or its modelling, should a study be disqualified for use. The commenter contends that, if the rule takes this form, it might further the overriding objective of applying the best available science to regulatory decision-making.
Response: As described in the final rule, the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. The Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
3.1.8.2 Reanalysis of Studies Would be Burdensome
Comment: Commenters (4542, 5015, 6132, 9227) express concern regarding the cost and time-consuming nature of reanalysis. 
Commenter (4542) states that experience with reanalysis shows that simply making data available is insufficient; the participation of the original researchers is often required because of their unique expertise in the design, collection and analysis, especially for large or complex datasets. The commenter expresses concern that such participation could be a substantial time and resource burden on researchers. 
Commenter (5015) reports that, in a limited number of cases, where there are not comparable studies in other datasets, it may be useful to gain access to the original study data and analytic codes to allow for independent evaluation. The commenter contends that this is a highly cost-intensive and time-consuming endeavor which should only be applied in cases where there are one or just a few studies in a given area.
Commenter (9227) states that it would be infeasible to undertake expensive and time-consuming reanalysis for the vast majority of studies.
Commenters (6153, 6874, 8272) state that in a look back at the evolution of research on the health effects of particulate pollution, lead Harvard Six Cities investigator Douglas W. Dockery commented that "the HEI work reanalyzing an existing body of evidence was time consuming and expensive and it would not be a good use of resources to require this approach for all studies contributing to the regulatory actions, or even to a few key studies."
Response: In the final rule, Part 30.2, the EPA, Reanalyze means to analyze exactly the same dose-response data to see if a similar result emerges from the analysis by using the same methods, statistical software, models, or statistical methodologies that were used to analyze the dose-response data, as well as to assess potential analytical errors and variability in the underlying assumptions of the original analysis and Independent validation means the reanalysis of study dose-response data by subject matter experts who have not contributed to the development of the study to evaluate whether results similar to those reported in the study are produced.
The EPA would like to emphasize that this final rule does not impose requirements on any entity outside of the EPA and thus does not require a third-party to expend resources to reanalyze the dose-response data underlying pivotal science.
EPA continues to find that independent validation of the study findings and conclusions driving the EPA's dose-response assessments would provide important information. As detailed in Section III.A.1 of the preamble to the final rule, there is scientific support for the usefulness of reanalyzing data, and the EPA finds this to be especially true for data that drive the quantitative requirements or analyses of EPA significant regulatory actions or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and, thus, increase the opportunity for independent subject matter experts to validate pivotal science, and as the dose-response data are better understood through independent reanalysis the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
3.1.8.3 Limitations of Reanalysis
Comment: Commenters (1973, 4840, 6132, 9227) contend that reanalysis of the same data does not strengthen the findings or validity of a study.
Commenter (1973) states that, while it is sometimes argued that the failure to replicate studies is a "crises in science" that necessitates the EPA proposal, the EPA proposal will only facilitate reanalysis. 
Commenter (4840) asserts that reanalyzing the same study over and over is little different from checking on a surprising newspaper article by buying additional copies of the same paper to see if it says the same thing. 
Commenter (6132) suggests that, if the EPA really wants to evaluate the validity of studies, it should fund more such studies to independently test the study design elsewhere to see if the results are confirmatory (as has already been repeatedly shown for particulate matter air pollution health effects), not by repeatedly reanalyzing the statistics from past studies over and over again.
Commenter (9227) states that, if all credible methods of reanalysis yield effectively the same results as the original analysis, they do not strengthen the original findings. The commenter suggests that the use of differing statistical models be assessed with care and there be a demonstration that the assumptions supporting a new method of analysis is significantly more credible than the original analysis. The commenter provides that it is easy to develop methods of analysis that can demonstrate a different finding but are created solely for that purpose and these should not be given greater weight in evaluating a study. 
Response: The EPA disagrees with these commenters. Reanalysis of data can be valuable. Jamieson et al. (2019) state, "Journals play a key role in ascertaining the quality of the evidence in work submitted for publication through peer review. However, that practice is neither uniform nor free of exploitation. While it can select for trustable results, its existence alone is not sufficient to verify the quality of the evidence used to back claims." Because of the potential impact of significant regulatory actions, independent validation of studies that are pivotal to the EPA's significant regulatory actions and influential scientific information may increase public confidence in the findings and conclusions presented in those studies. As Jamieson et al. (2019) states, publication alone (including the typical peer review necessary for publication) is insufficient to verify a study's quality, and additional confidence could be gained from third-party reanalyses that independently validate the study results.
Other commenters support the requirement for independent validation by reanalysis and the EPA agrees. Reanalysis is more than just checking for mathematical errors. For example, reanalyzing the underlying dose-response data, independent researchers can evaluate the myriad of choices and assumptions the original researchers have made regarding the data and statistical models and the potential introduction of any sources of bias. 
3.1.8.4 Other Comments Regarding Reanalysis
Comment: Commenter (8771-PH56) asserts that EPA presents no evidence that anyone is hindering anyone from reanalyzing. The commenter reports that any bioassay that EPA would use would have individual tumor data and exposures and could be reanalyzed with any model anyone wanted. The commenter states that, for epidemiology studies, important confounders can almost always be tested if need be. According to the commenter, any failure to reanalyze is not the fault of the original investigators, so they and the public should not be punished for the purported lack of reanalysis. 
Response: The EPA disagrees with this commenter. Reanalysis cannot be conducted by just "anyone" if the underlying data is not available. The EPA would like to emphasize that this final rule does not impose requirements on any entity outside of the EPA and thus does not require a third-party to reanalyze the dose-response data underlying pivotal science.
3.1.9 Recommends Further Consultation to Develop New Rule Addressing Reproducibility and Replicability Concerns
Comment: Commenters (1742, 4894) recommend that the rule be revised to address reproducibility and replicability concerns in consultation with the National Academies of Sciences and scientific Journals.
Commenter (1742) states that this rule should be revised and re-evaluated by scientists from the National Academy of Sciences, Nature, Science, and Cell if EPA wants to help reproducibility. 
Commenter (4894) recommends the rule be withdrawn and suggests that a new rule addressing concerns about reproducibility and replicability be developed in public, with participation by the scientific community, the environmental community, and industry. The commenter suggests that the rule developers avail themselves of the results of the ongoing Reproducibility and Replicability study being conducted by the National Academies of Sciences. The commenter provides that the report for this study is expected to be completed in December 2018.
Response: In development of this final rule, EPA considered the public comments on the 2018 NPRM, the 2020 SNPRM, and the advice of the SAB. Numerous scientific organizations provided comment on the NPRM and SNPRM during the public comment periods.
3.2 SNPRM Capable of Being Substantially Reproduced; Replication and Reproduction
3.2.1 Capable of Being Substantially Reproduced; Replication and Reproduction
3.2.1.1 Proposed Definition for Capable of Being Substantially Reproduced
3.2.1.1.1 General Support for Definition
Comment: Commenters (9587, 11401, 12406, 12471) support the use of the term as defined in the SNPRM. Recommendations for modification are provided:
Commenter (12471) proposes the definition to be modified to read:
      Capable of being substantially reproduced means that independent analysis of the original and/or supporting data using identical or other validated methods would generate similar analytic results, subject to an acceptable degree of imprecision or error.
The commenter states the "and/or" to indicate the instance of both original and supporting data being available and that the suggested addition of "or other validated" opens the possibility that there are equally valid methods other than the original researcher used, and reproduced results using other validated methods should also yield substantially similar results.
Response: The EPA found that comments concerning inconsistencies and a lack of clarity with the definition of "capable of being substantially reproduced" have merit and is modifying the definition of "independent validation" in the final rule by replacing "capable of being substantially reproduced" with "produced." The EPA will not finalize the proposed 40 CFR 30.2 definition of "capable of being substantially reproduced" because the term is not used in the final rule's definition of "independent validation" or elsewhere in 40 CFR 30. As a result, "substantially" will not need to be defined or described in the final rule. The EPA is also modifying the definition of "reanalyze" to specify the use of the same methods because as proposed it specified the use of the "same or different" methods. This change was made so that the definition would be consistent with the final rule's definition of "independent validation." 
3.2.1.1.2 Issues With the Provided Definition
Generally Confusing
Comment: Commenter (11328) contends that the definition remains confusing due to the term reproducible being used in a manner that directly contradicts the proper definition on the very same page of the Federal Register notice. Due to this, the commenter states that it is clear that the supplemental proposal lacks input from the scientific community.
Response: The EPA found that comments concerning inconsistencies and a lack of clarity with the definition of "capable of being substantially reproduced" have merit and is modifying the definition of "independent validation" in the final rule by replacing "capable of being substantially reproduced" with "produced." The EPA will not finalize the proposed 40 CFR 30.2 definition of "capable of being substantially reproduced" because the term is not used in the final rule's definition of "independent validation" or elsewhere in 40 CFR 30. As a result, "substantially" will not need to be defined or described in the final rule. The EPA is also modifying the definition of "reanalyze" to specify the use of the same methods because as proposed it specified the use of the "same or different" methods. This change was made so that the definition would be consistent with the final rule's definition of "independent validation." 
Comment: Commenter (11479) states that EPA's draft rule, as modified in the supplemental notice, reveals that the Agency wants to use original data and methods to reanalyze rules and prove that they are reproducible before using them to support a rule. The commenter claims that the supplemental notice still has not clarified what it means by "capable of being substantially reproduced," and it contradicts itself. The commenter provides that the provided definition is in conflict with the National Academy of Sciences (NAS) workshop report citation that defines reproducibility as "producing something that is very similar to that research, but is in a different medium or context. In other words, a researcher who is reproducing an experiment addresses the same research question from a different angle than the original researcher did." The commenter believes that EPA is disguising its quest to cherry-pick evidence with unclear definitions of reanalysis, independent validation, and "capable of being substantially reproduced."
Response: The EPA found that comments concerning inconsistencies and a lack of clarity with the definition of "capable of being substantially reproduced" have merit and is modifying the definition of "independent validation" in the final rule by replacing "capable of being substantially reproduced" with "produced." The EPA will not finalize the proposed 40 CFR 30.2 definition of "capable of being substantially reproduced" because the term is not used in the final rule's definition of "independent validation" or elsewhere in 40 CFR 30. As a result, "substantially" will not need to be defined or described in the final rule. The EPA is also modifying the definition of "reanalyze" to specify the use of the same methods because as proposed it specified the use of the "same or different" methods. This change was made so that the definition would be consistent with the final rule's definition of "independent validation." 
The rule does not require the Agency to reanalyze, reproduce, or independently validate pivotal science before using the pivotal science to support a significant regulatory action or influential scientific information. 
Provides Insufficient Guidance
Comment: Commenters (11176, 11576, 12443, 12715, 12720) note that aspects of the definition, and related definitions, do not provide for several key aspects that would be required for uniform application. Specifically:
Commenter (11176) states that, to the EPA's credit, the 2002 OMB definition for "capable of being substantially reproduced" is used. However, the commenter notes that the definition contains the words "similar" and "acceptable," which need a standard for meaning or who will make the determination (e.g. scientific staff with oversight of an EPA scientific advisory panel). The commenter similarly raises this concern for the word "substantially" in the term "capable of being substantially reproduced." Additionally, the commenter provides that guidance for these determinations would need to take into account how unlikely it is for two studies to exactly replicate. The commenter provides that in 2015 Ralph Cicerone, then president of the National Academy of Sciences (NAS) addressed this point in a speech, noting that a NAS panel had grappled with the very question: "Because variability across studies is to be expected, how can we assess the acceptable degree of variability and when should we be concerned about reproducibility?" According to the commenter, there should also be guidance to require explanation of whether research has or has not met the sufficient-for-independent-validation criterion.
Commenter (11576) provides that the term is ambiguous and relies on personal judgement. As such, the commenter notes that EPA should specify what an "acceptable degree of imprecision or error" means for specific types of data and models. According to the commenter, without greater specificity, the degree of imprecision or error found to be acceptable will vary between decision-makers. The commenter contends that "capable of being substantially reproduced" could be defined, for instance, relative to the impact of reproducing data on directional outcomes or by providing specific quantitative thresholds for acceptable degrees of imprecision or error.
Commenter (12443) asserts that for any proposal with broad implications, the proposal needs to define what are the constraints under which a study is deemed capable of being reproduced.
Commenter (12715) takes pause with the inclusion of the words "substantially," "similar," and "acceptable degree" as they reflect upon the term's subjectivity. The commenter states that the supplemental proposal does not include any criteria for what would constitute as these, nor does it indicate where in the regulatory process those determinations would be made, who would make them, and whether those determinations would be explained in the public notice process.
Commenter (12720) states that EPA does not define how a "contribution" to a particular study is interpreted, and whether that contribution may be in the form of financial resources or not, nor does EPA define a threshold that would qualify as "substantial" reproduction of study findings, and does not detail how "capable of being substantially reproduced" is interpreted. The commenter adds that it is not clear from EPA's definition of "independent validation" to what degree study findings in all of their forms (e.g., general results, point estimates, confidence intervals, sensitivity analyses, statistical significance) must be "reproduced."
Response: The EPA found that comments concerning inconsistencies and a lack of clarity with the definition of "capable of being substantially reproduced" have merit and is modifying the definition of "independent validation" in the final rule by replacing "capable of being substantially reproduced" with "produced." The EPA will not finalize the proposed 40 CFR 30.2 definition of "capable of being substantially reproduced" because the term is not used in the final rule's definition of "independent validation" or elsewhere in 40 CFR 30. As a result, "substantially" will not need to be defined or described in the final rule. The EPA is also modifying the definition of "reanalyze" to specify the use of the same methods because as proposed it specified the use of the "same or different" methods. This change was made so that the definition would be consistent with the final rule's definition of "independent validation." The EPA did not define "similar" in the regulatory text as this would be overly prescriptive. The EPA intends to issue implementation guidelines and statute-specific rulemakings that will provide further clarity.
Focus on Reanalysis Rather Than Reproduction
Comment: Commenter (11479) asserts that the inclusion of the definition focuses on reanalysis, which clarifies the necessity of underlying data being made available to interested stakeholders to confirm reproducibility. As such, the commenter contends that the goal of the rule is to introduce an avenue for stakeholders with enough time and resources to recalculate study data to better meet deregulatory outcomes rather than to increase transparency.
Response: As described in the preamble to the final rule, the purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information, not to encourage any particular stakeholder outcome. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Having a better understanding of the underlying data will enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information.
The EPA found that comments concerning inconsistencies and a lack of clarity with the definition of "capable of being substantially reproduced" have merit and is modifying the definition of "independent validation" in the final rule by replacing "capable of being substantially reproduced" with "produced." The EPA will not finalize the proposed 40 CFR 30.2 definition of "capable of being substantially reproduced" because the term is not used in the final rule's definition of "independent validation" or elsewhere in 40 CFR 30. As a result, "substantially" will not need to be defined or described in the final rule. The EPA is also modifying the definition of "reanalyze" to specify the use of the same methods because as proposed it specified the use of the "same or different" methods. This change was made so that the definition would be consistent with the final rule's definition of "independent validation." 
Comment: Commenter (12727) states that the proposed definition of "capable of being substantially reproduced" attempts to bring the word reproduced into the realm of reanalyzed. According to the commenter, reproducing an experiment addresses the same research question, but approaches the question from a different angle than the original experiment, while a reanalysis is when a person analyzes exactly the same data to see if the same result emerges using the same program and statistical methodologies, or perhaps alternative methodologies. Due to this, the commenter contends that EPA's definition is inappropriate and would mislead the public into believing that some new scientific insights have been applied to an original set of data such that the original research findings are made stronger. The commenter provides that instead, the definition highlights a misconception that independent validation is accomplished through reanalysis of study data by other experts; the strength of scientific findings are not determined by endless reanalysis of a single data set, but rather gained over time through the conduct of separate investigations utilizing different methodologies that yield corroborating conclusions around a hypothesis being explored.
Response: The EPA found that comments concerning inconsistencies and a lack of clarity with the definition of "capable of being substantially reproduced" have merit and is modifying the definition of "independent validation" in the final rule by replacing "capable of being substantially reproduced" with "produced." The EPA will not finalize the proposed 40 CFR 30.2 definition of "capable of being substantially reproduced" because the term is not used in the final rule's definition of "independent validation" or elsewhere in 40 CFR 30. As a result, "substantially" will not need to be defined or described in the final rule. The EPA is also modifying the definition of "reanalyze" to specify the use of the same methods because as proposed it specified the use of the "same or different" methods. This change was made so that the definition would be consistent with the final rule's definition of "independent validation."
Reproducibility as Opposed to Replicability
Comment: Commenter (11405) contends that the terms reproducibility and replicability are used as though they were synonymous throughout the supplemental proposal. The commenter asserts that in actuality, reproducibility is a quantitative measure that determines if the outputs of the original analysis and a reproducibility check are quantitatively identical, while replicability is only achieved if the design and findings of the original study and replication attempts are qualitatively similar. According to the commenter, the qualitative nature of replicability can be highly controversial due to the substantive assumptions about which similarities and differences. 
Response: The Agency recognizes that the 2018 NPRM used terms that were not fully defined or not defined at all. The 2020 SNPRM addressed a number of these comments. To address the concerns expressed by commenters, in this final rule, in Part 30.2, the EPA is defining the following terms: "data," "dose-response data," "independent validation," "influential scientific information," "pivotal science," "publicly available," "reanalyze," "science that serves as the basis for informing a significant regulatory action," and "significant regulatory actions." The terms "pivotal regulatory science," "replication," "reproducible," and "research data" are not used in this final rule.
What Open Science is Able to Support
Comment: Commenter (11405) states that open science primarily supports reproducibility, by enabling or encouraging researchers to make their data and analysis code available for reproducibility checks. The commenter also provides that because reproducibility and replicability are independent, supporting reproducibility does not necessarily support replicability. Consequently, the commenter asserts that open science does not necessarily address concerns about replicability.
Response: The Agency recognizes that the 2018 NPRM used terms that were not fully defined or not defined at all. The 2020 SNPRM addressed a number of these comments. To address the concerns expressed by commenters, in this final rule, in Part 30.2, the EPA is defining the following terms: "data," "dose-response data," "independent validation," "influential scientific information," "pivotal science," "publicly available," "reanalyze," "science that serves as the basis for informing a significant regulatory action," and "significant regulatory actions." The terms "pivotal regulatory science," "replication," "reproducible," and "research data" are not used in this final rule.
3.2.1.1.3 Relation to Office of Management and Budget (OMB) Guidance
Comment: Commenter (11479) asserts that the Agency has not shown that the current review process based on the 2001 OMB guidance is not sufficient to catch simple calculation errors. The commenter provides that EPA has not provided evidence that there is an epidemic of calculation mistakes plaguing Agency science that would require an impossible requirement of public access to raw data that steps beyond the provisions of current EPA policies and practices and beyond the needs of the Agency.
Response: The purpose of the rule is not to fix prior mistakes. As detailed in the preamble to the final rule, the purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Comment: Commenter (11911) contends that the supplementary proposal is inconsistent with the replication standards of the OMB.
Response: The EPA maintains that this rule designed to build upon OMB guidelines. Rather, the rule will complement and augment them and other transparency efforts. The EPA would like to clarify that it does not derive authority for this rule from the OMB Guidelines. The EPA is relying exclusively on its housekeeping authority.
3.2.1.2 Application to EPA Practices
3.2.1.2.1 Expansion of Proposal to Include Reproducibility of Key EPA Decisions
Comment: Commenter (10973) contends that EPA should expand its proposal to include the reproducibility how regulatory and non-regulatory limits are set across many environmental statutes, which is considerably more valuable for stakeholders to engage with the Agency than individual data points.
Response: The EPA does not agree with the recommended expansion of the proposal. As discussed in the preamble to the final rule, the Agency is pursuing an incremental approach to maximizing transparency in the science that it relies upon, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
3.2.1.2.2 No Requirement to Ensure Analytic Results are Reproducible
Comment: Commenter (11381) provides that the Information Quality Guidelines (IQG) transparency requirements apply to derivative information produced by EPA as well as derivative information produced by third-parties that the Agency wants to disseminate. According to the commenter, failing to require EPA program offices to comply with this IQG requirement goes against the concept of transparency. As such, the commenter recommends EPA require its offices to demonstrate adherence to the IQG requirement by reproducing the information it intends to disseminate.
Response: The EPA disagrees with the recommended expansion to this rule. This final rule establishes how the EPA will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information. When promulgating significant regulatory actions or developing influential scientific information, the EPA will give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. The final rule does not require the EPA to reproduce or evaluate the reproducibility of dose-response data, but instead increases the opportunity for subject matter experts to independently validate pivotal science, if they so choose.
3.2.1.3 Reproducibility and Good Science
3.2.1.3.1 Reproducibility as a Determinant of Good Science
Reproducibility Is Not a Standalone Factor
Comment: Commenter (11103) states that there is no doubt that reproducible studies instill credibility in scientific knowledge, but reproducibility is an inadequate measure to solely judge the rigor of scientific research. The commenter notes that successfully reproducing a result does not prove that the original scientific result is correct, nor does failing to reproduce it definitively negate the original claims. In fact, the commenter provides that the less variation there is in the conditions of a study, the fewer contexts in which the results are true. The commenter references a 2009 study on the effect of environmental variations on reproducibility in animal experiments which found that reducing variation led to results that were only true in the specified conditions, with little external validity. The commenter asserts that when the extent to which the results of a study do not apply in other contexts or populations that differ from the original one, the results cannot be determined to be robust enough, no matter how reproducible they are. According to the commenter, the EPA should not be focusing solely on reproducibility to ensure data is valid, but instead should concentrate on generalizability as well as the effect and size of data. The commenter recommends that EPA revoke its proposal to prioritize scientific studies on the nonscientific basis of public availability of underlying data. Furthermore, the commenter notes this will allow EPA to assess the entire body of research on a subject and gain confidence in the state of scientific knowledge by more accurate benchmarks of quality; namely generalizability, not reproducibility.
Response: The Agency recognizes that the 2018 NPRM used terms that were not fully defined or not defined at all. The 2020 SNPRM addressed a number of these comments. To address the concerns expressed by commenters, in this final rule, in Part 30.2, the EPA is defining the following terms: "data," "dose-response data," "independent validation," "influential scientific information," "pivotal science," "publicly available," "reanalyze," "science that serves as the basis for informing a significant regulatory action," and "significant regulatory actions." The terms "pivotal regulatory science," "replication," "reproducible," and "research data" are not used in this final rule. 
The EPA agrees with the commenter in part and provided additional clarifications in the final rule. As explained in 40 CFR 30.5, the EPA will use existing factors (including soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review) to evaluate study quality. Subsequently, the EPA would identify the highest quality, most relevant studies that would inform a dose-response assessment and evaluate the availability of the dose-response data underlying pivotal science. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
Comment: Commenter (11479) contends that the importance of reproducibility in the proposed rule is inconsistent with provisions in the 2002 OMB guidelines that detail the nuance between the reproducibility of a study and other criteria that Agency scientists currently use to evaluate a study's credibility, that "the objectivity standard does not override other compelling interests such as privacy, trade secrets, intellectual property, and other confidentiality protections," and that agencies can apply robustness checks in situations where public access to data and methods will not occur due to other compelling interests." According to the commenter, the Agency can verify the independence and credibility of a study in other ways, such as by evaluating its research design, methods, data summaries, and degree of alignment with similar studies in the field. The commenter asserts that EPA scientists should have the flexibility to use scientific criteria to judge the rigor and validity of the evidence. Additionally, the commenter states that EPA already does this well, and the rule does not point to any evidence to the contrary.
Response: The EPA agrees with the commenter in part and provided additional clarifications in the final rule. As explained in 40 CFR 30.5, the EPA will use existing factors (including soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review) to evaluate study quality. Subsequently, the EPA would identify the highest quality, most relevant studies that would inform a dose-response assessment and evaluate the availability of the dose-response data underlying pivotal science. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
Reproducibility as a Necessity
Study Validity's Dependence on Reproducibility
Comment: Commenter (12427) notes that in Dr. Allison's testimony to the House of Representatives' Committee on Science, Space, and Technology, it is stated that "reproducibility is neither a necessary nor a sufficient condition for a scientific study or data set to be valid or useful," as, "a study can be reproducible and transparent and yet completely invalid". The commenter further agrees with Dr. Allison's comment that "it would not be appropriate to determine the rigor or the regulatory applicability of a study based solely on its reproducibility as reproducibility is defined in the National Academies' report for the reasons I have stated above. In short, reproducibility is neither a necessary nor a sufficient condition to determine the validity of a study for in turn determining the truth of a proposition."
Conversely, commenter (12712) states that science proceeds by replication  -  and what the rule calls reanalysis  -  and without data availability, this process cannot happen.
Response: In the final rule the terms replicate and reproduce (or other forms of the word) are no longer used. In the final rule the term replicate is not used. The EPA is clarifying that the intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze." In addition, the EPA is modifying the definition of "independent validation" in the final rule by replacing "capable of being substantially reproduced" with "produced." The EPA will not finalize the proposed 40 CFR 30.2 definition of "capable of being substantially reproduced" because the term is not used in the final rule's definition of "independent validation" or elsewhere in 40 CFR 30.
The EPA continues to believe that data transparency is vital for individuals who have not contributed to the study to be able to verify the quality and strength of published studies and agrees with commenters that the opportunity to independently validate the pivotal science that the EPA relies upon is important in furthering scientific understanding and the Agency's mission. A presenter in a 2016 NAS workshop on Principles and Obstacles for Sharing Data from Environmental Health Research stated more directly that "for environmental policy making to be legitimate, the scientific reasoning behind a given decision -- including the data supporting it -- must be transparent" (NAS Workshop Report, Ref. 26). When data are widely available, researchers can validate research results and help identify and correct unintended errors, as well as reanalyze the data for new and different purposes, examine novel questions, provide new scientific insights, and improve model development.
While the availability of dose-response data underlying a study in a manner sufficient for independent validation is an important component of determining the level of consideration to afford a study, the EPA agrees that availability by itself is not sufficient to determine study quality and made subsequent modifications to the final rule. As explained in 40 CFR 30.5, the EPA will use existing factors (including soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review) to evaluate study quality. Subsequently, the EPA would identify the highest quality, most relevant studies that would inform a dose-response assessment and evaluate the availability of the dose-response data underlying pivotal science. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7.
Consideration of Information by EPA
Comment: Commenter (11899) contends that all materials must be replicated, reproduced or confirmed by EPA and/or outside independent experts before they can be used or considered by EPA in formulating policies or regulations.
Response: Although the Agency agrees that greater transparency is necessary, the EPA disagrees with the recommended expansion of the rule. The final rule does not require the EPA to reproduce or evaluate the reproducibility of dose-response data, but instead increases the opportunity for subject matter experts to independently validate pivotal science, if they so choose. Future statute-specific rulemakings may be more expansive as the EPA continues to make incremental progress toward maximizing transparency. 
Coding Errors and Replicability
Comment: Commenter (11405) notes that in some cases, coding errors might lead to conclusions that are qualitatively different from the results of otherwise similar studies and may lead to replicability problems. The commenter provides that they know of no empirical study that provides evidence that these kinds of coding errors are a widespread problem in epidemiology or any related field, nor does the supplemental proposal appear to cite any such sources.
Response: The EPA disagrees with the assertion that the rule is unnecessary because there is little evidence of a widespread reanalysis issue in science. Collectively, evidence from published reanalysis studies, retractions, surveys of scientists, and public commenters provides sufficient evidence that additional policies that remove barriers to data availability and reanalysis and create greater transparency are needed. The EPA's SAB, several public commenters, and authors of peer-reviewed literature have noted that data sharing, like that envisioned under this rulemaking, would strengthen science. The scientific benefits of openness in research are well-recognized. Open sharing of data and models reduces duplication of effort from different researchers attempting to collect the same datasets and allows for the use and reuse of data and models among the research community. Scientific openness also improves the ability for the scientific community to validate research results, helping to prevent errors and reducing the risk of fraud in science and research. Also, when underlying data are made widely available, researchers can re-analyze the data for new and different purposes, examining novel questions and providing new scientific insights. Some studies indicate that data sharing can actually increase the number of citations by third-party researchers, which can indicate higher influence of the original research.[,][,] Collaboration and communication, both within the scientific community and between researchers and other organizations or individuals, are also enhanced by sharing of data and models.
Replication Is a Matter of Internal and External Validity
Comment: Commenter (11405) asserts that replicability is a matter of generalizability and extrapolation, from the original study's sample to another sample. The commenter states that applying different statistical techniques to the original study's sample will not tell whether and to what extent findings can be generalized. While robustness checks may give some indications of the internal validity of a study, the commenter notes that they tell nothing about how broadly its conclusions can be generalized. Thus, according to the commenter, while open data does promote robustness, this does not do anything to promote replicability. As such, the commenter contends that given a study, the best way to determine whether it is replicable is to attempt to replicate it, not to re-analyze the same data.
Response: Although the EPA agrees that meaningful insights can be obtained through similar studies in different media or context, the EPA continues to find that independent validation of the study findings and conclusions driving the EPA's dose-response assessments would provide important information. As detailed in the preamble to the final rule, there is scientific support for the usefulness of reanalyzing data, and the EPA finds this to be especially true for data that drive the quantitative requirements or analyses of EPA significant regulatory actions or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and, thus, increase the opportunity for independent subject matter experts to validate pivotal science, and as the dose-response data are better understood through independent reanalysis the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
3.2.1.3.2 Alternative Pathways to Attaining Confidence in Science
Comment: Commenter (11360) states that reproducibility and replicability are elements for determining quality science, but are only a part of the overall quality assurance landscape and that there are limits to these metrics as described in the supplemental proposal.
Commenter (11182) contends that the supplemental proposal's assumption that confidence in science can only be attained by reanalysis of the data is mistaken. The commenter notes that the National Academies of Science notes that "replicability and reproducibility are crucial pathways to attaining confidence in scientific knowledge, although not the only ones." Additionally, the commenter states that NAS describes other widely accepted methods for improving scientific rigor, including research synthesis, systematic reviews, and meta-analyses and specifically notes that "some research involves nonpublic information that cannot legally be shared, such as patient records or human subject data."
Commenter (12464) provides that it is through the combined operation of peer review, replication, and reproduction that the scientific community typically determines whether the results of a particular study are reliable. The commenter notes that peer review has been a widely-accepted method for ensuring high-quality results for the past three and a half centuries, and EPA has integrated the scientific community's peer-review standards into its policies since at least 1993., In addition, the commenter states that reproduction, especially when it involves the integration of results from multiple lines of inquiry, is particularly important; as NAS recently explained, "[t]he robustness of science is less well represented by the replications between two individual studies than by a more holistic web of knowledge reinforced through multiple lines of examination and inquiry." According to the commenter, all information necessary for these are typically made available by researchers, and does not entail the data privacy and confidentiality issues raised by the supplemental proposal. Furthermore, the commenter holds the supplemental proposal to be inconsistent with these best practices, especially as EPA has traditionally relied on these best practices under its Framework for Incorporating Human Epidemiologic & Incident Data in Risk Assessments for Pesticides. The commenter states that this framework allowed for the basis of EPA's 2016 Revised Human Health Risk Assessment for chlorpyrifos on the fact that studies from "different investigators, locations, points in time, exposure assessment procedures, and outcome measurements" all reached similar results.
Response: In the final rule the terms replicate and reproduce (or other forms of the word) are no longer used. In the final rule the term replicate is not used. The EPA is clarifying that the intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze." In addition, the EPA is modifying the definition of "independent validation" in the final rule by replacing "capable of being substantially reproduced" with "produced." The EPA will not finalize the proposed 40 CFR 30.2 definition of "capable of being substantially reproduced" because the term is not used in the final rule's definition of "independent validation" or elsewhere in 40 CFR 30.
While the independent validation is an important component of determining the level of consideration to afford a study, the EPA agrees that availability by itself is not sufficient to determine study quality and made subsequent modifications to the final rule. As explained in 40 CFR 30.5, the EPA will use existing factors (including soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review) to evaluate study quality. Subsequently, the EPA would identify the highest quality, most relevant studies that would inform a dose-response assessment and evaluate the availability of the dose-response data underlying pivotal science. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7.
Further, although the EPA agrees that meaningful insights can be obtained through similar studies in different media or context, the EPA continues to find that independent validation of the study findings and conclusions driving the EPA's dose-response assessments would provide important information. The benefits of greater data transparency and the significance of reanalyzing and validating study results are well-documented in scientific literature. McNutt (2014) noted, "reproducibility, rigor, transparency, and independent verification are cornerstones of the scientific method." The National Academies of Sciences, Engineering, and Medicine (NAS) workshop on Reproducibility and Replicability in Science also noted that "certainly, reproducibility and replicability play an important role in achieving rigor and transparency." Munafò et al. (2017) state, "the credibility of scientific claims is rooted in the evidence supporting them, which includes the methodology applied, the data acquired, and the process of methodology implementation, data analysis and outcome interpretation. Claims become credible by the community reviewing, critiquing, extending and reproducing the supporting evidence. However, without transparency, claims only achieve credibility based on trust in the confidence or authority of the originator. Transparency is superior to trust." The 2019 NAS workshop on Reproducibility and Replicability in Science also concluded, "the scientific enterprise depends on the ability of the scientific community to scrutinize scientific claims and to gain confidence over time in results and inferences that have stood up to repeated testing."  The EPA agrees that data transparency is vital for individuals who have not contributed to the study to be able to verify the quality and strength of published studies and finds that the benefits of reanalysis are especially important for data that drive the quantitative requirements or analyses of EPA significant regulatory actions or influential scientific information. Implementation of this rule will increase transparency and, thus, the opportunity for independent subject matter experts to validate pivotal science, and as the dose-response data are better understood the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
3.2.1.3.3 Reproducibility Crisis
The Need to Address the Crisis
Comment: Commenters (11401, 11899) highlight the need for reproducibility given reports of the inability to reproduce many scientific studies today. Specific concerns include:
Commenter (11401) supports EPA's efforts to give greater weight to transparency and reproducibility, two basic components of sound science. The commenter states that peer review is not always an indication that a scientific report has been properly critiqued, and rarely includes independent verification of reproducibility. According to the commenter, the Federal Government's own Information Quality Act guidelines recognize that "there is a significant scholarly literature documenting quality problems with articles published in peer-reviewed research." The commenter also asserts that this runs the risk that regulations are based on faulty science. Additionally, the commenter contends that ensuring EPA holds itself to the highest standards of transparency and reproducibility help to mitigate that risk, and this rulemaking is important in that regard. The commenter references the fact EPA has received comments from some who are invested in a system that enables them to receive grant money and publish research without having to subject their research to the rigors of reproducibility and asserts that EPA should understand the interests behind those comments when evaluating them. According to the commenter, science, like every human endeavor, is subject to human foibles. The commenter states that they have seen clearly biased scientists publish studies that are fatally flawed, funded by activists, purposefully misleading, even containing falsified results. The commenter notes that even when debunked through the peer review process, erroneous studies persist in the public domain and influence rulemaking at every level of government.
Commenter (11899) asserts that the problem of questionable scientific research and government Agency reliance on it has become systemic and systematic, stating the vast majority of studies in numerous scientific arenas cannot be replicated, even those published in "reputable" peer-reviewed scientific and medical journals that the vast majority. The commenter contends that incompetence and outright fraud are primary causes for irreproducibility. In the case of fraud, the commenter states that fraud can likewise arise from politically motivated regulatory actions and biased, secretive, highly selective studies and analyses conducted to support and justify predetermined Agency actions.
Response: The EPA agrees with commenters on the need and importance of greater transparency in science. Numerous public commenters on this rulemaking and some members of the EPA's SAB noted concern about a "crisis" in science due to low reported success rates and noted support for the rulemaking as a potential solution. The EPA agrees that the low success rates published in reanalysis studies are potentially worrisome and also agrees with the National Academies of Sciences and others that the paucity of full data sharing makes it difficult to even systematically evaluate the extent of the problem within or across scientific disciplines. One barrier is having access to enough of the underlying data, models, and methods to be able to conduct the independent validation. Encouraging greater transparency and data and model sharing, then, is vital for individuals outside of the publication of the study to be able to verify the quality and strength of published studies and to identify and correct suboptimal research and publication practices, such as positive bias, p-hacking, and post hoc reasoning that may influence the results presented in a publication.,,,,,,,  Open sharing of data and models reduces duplication of effort from different researchers attempting to collect the same datasets and allows for the use and reuse of data and models among the research community. Scientific openness also improves the ability for the scientific community to validate research results, helping to prevent errors and reducing the risk of fraud in science and research. Also, when data and models are made widely available, researchers can re-analyze the data for new and different purposes, examining novel questions and providing new scientific insights. 
Scientific Community Initiatives
Comment: Commenter (10974) contends that the supplemental proposal does not integrate, nor does it provide solutions for or improvements to, the current work on data reproducibility within the scientific community. The commenter provides that, similar to how the supplemental proposal does not address an extant need for rulemaking regarding data transparency, the supplemental proposal (and 2018 proposal) do not support nor adequately address the existing efforts by all types of institutions to improve data transparency and availability issues.
Response: As discussed in the preamble to the final rule, the purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information. The requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
Validation as the Next Step to Addressing the Crisis
Comment: Commenter (11381) notes that the reproducibility/replication crisis convincingly shows that the IQG reproducibility requirement was justified in 2002 and is only more so today. In addition, the commenter asserts that reproducing an Agency work product or a third-party study, as that that term is understood in the IQG, is not sufficient, and that validation is clearly the next step. However, the commenter provides that the supplemental proposal does not provide a credible basis for such a requirement. According to the commenter, the supplemental proposal does not define the term, explain what Agency program offices are expected to do, describe how Agency peer review would be modified, or inform the public concerning how to test Agency claims. The commenter contends that new Part 30 needs a rigorous treatment of validation, and this almost certainly requires a second supplemental proposal. The commenter asserts that therefore, the best path forward is to promulgate the IQG in rule form now and commit to propose additional provisions on validation. 
Response: The EPA agrees with the commenter on the necessity and importance of independent validation. The EPA would like to clarify that in the final rule the terms replicate and reproduce (or other forms of the word) are no longer used. In the final rule the term replicate is not used. The intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze." In addition, the EPA is modifying the definition of "independent validation" in the final rule by replacing "capable of being substantially reproduced" with "produced." The EPA will not finalize the proposed 40 CFR 30.2 definition of "capable of being substantially reproduced" because the term is not used in the final rule's definition of "independent validation" or elsewhere in 40 CFR 30. 
In consideration of these and other comments, the EPA also modified the final rule to provide additional detail and clarity, including text related to process of evaluating studies, identifying pivotal science, determining the consideration to afford pivotal science, and peer review. The EPA believes that the final rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings. 
3.2.1.3.4 Process of Determining if Results Can Be Replicated
Comment: Commenter (11403) asserts that because scientists conduct their own rigorous review of others' research, scientists do not need raw data to determine if a study was conducted correctly but rather are trained to do so by examining data collection and analysis methods to determine whether the results can be replicated. The commenter provides that when doctors prescribe a medicine, they have deemed effective, it is not because they have reviewed and analyzed the raw data of patients in the clinical trials themselves. Instead, according to the commenter states that they have reviewed the published papers on the medication and have seen both that the trials were done correctly and that the results were significant enough to prescribe the treatment. The commenter contends that this is no different with research scientists.
Response: The EPA disagrees with this commenter. Peer review does not typically include reanalysis of the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information) and, thus, peer review is not considered a replacement for the data availability requirements of this rule. Jamieson et al. (2019) state, "Journals play a key role in ascertaining the quality of the evidence in work submitted for publication through peer review. However, that practice is neither uniform nor free of exploitation. While it can select for trustable results, its existence alone is not sufficient to verify the quality of the evidence used to back claims." The authors go on to say that additional confidence could be gained from third-party reanalyses that independently validate the study results. In addition, as detailed in OMB's Final Information Quality Bulletin for Peer Review, "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." The scientific community has also embraced greater data transparency, as evidenced by data sharing and availability requirements for many high-impact journals and the emergence of organizations, such as the Center for Open Science, and international initiatives like Findable, Accessible, Interoperable, and Reusable (FAIR) data principles; Facilitate Open Science Training in European Research (FOSTER); and Guidelines for Transparency and Openness Promotion (TOP) in Journal Policies and Practices that incentivize greater transparency in research.[,] The EPA supports these efforts and believes that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety.	
3.2.1.4 Feasibility of Reproducing Certain Studies
3.2.1.4.1 Capability to Obtain Scientific Information
Comment: Commenter (11381) provides that EPA often relies on scientific studies published in peer-reviewed journals. According to the commenter, obtaining and disclosing all information necessary to assure independent reproducibility is required to fulfil the IQG.[,] Without a provision to obtain this information, the commenter asserts that EPA program offices will be expected to rely on the limited information disclosed in peer-reviewed journals, which would prevent the achievement of the goal of transparency.
Response: The Agency recognizes that the 2018 NPRM used terms that were not fully defined or not defined at all. The 2020 SNPRM addressed a number of these comments. To address the concerns expressed by commenters, in this final rule, in Part 30.2, the EPA is defining the following terms: "data," "dose-response data," "independent validation," "influential scientific information," "pivotal science," "publicly available," "reanalyze," "science that serves as the basis for informing a significant regulatory action," and "significant regulatory actions." The terms "pivotal regulatory science," "replication," "reproducible," and "research data" are not used in this final rule. 
As further described in the preamble to the final rule, based on public comments and other considerations, the EPA is concentrating its efforts in the final rule to increase transparency of dose-response data, as the dose-response data are discrete and the dose-response assessment is a well-defined and impactful step in the quantitative assessment of risk. The EPA is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings. The final rule does not require the EPA to reproduce or evaluate the reproducibility of dose-response data. This rule is apart from and not intended to address potential issues with IQG. Future statute-specific rulemakings may be more expansive as the EPA continues to make incremental progress toward maximizing transparency.
Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7.
3.2.1.4.2 Ethical Issues
Comment: Commenters (11360, 11403, 11479, 11900, 12422, 12546, 12720) note that some public health studies are not reproducible due to ethical concerns. Specific concerns include:
Commenter (12422) queries whether studies on disasters like the Deepwater Horizon or Chernobyl would be discarded as they are not reproducible.
Commenter (11360) states that certain situations are either impossible to recreate or should not be due to the severity of health outcomes or the circumstances surrounding the exposures. The commenter provides that emergency responses to the 2010 Deepwater Horizon explosion and oil spill or a decade-long prospective cohort study on lead exposure in drinking water and adverse effects on childhood IQ would be difficult and unethical to replicate or reproduce.
Similarly, commenter (12546) provides that situations such as the 2010 Deepwater Horizon explosion and oil spill, health impacts from exposures during hurricanes such as Katrina and Rita, or a decade-long prospective cohort study on lead exposure in drinking water and adverse effects on childhood IQ would be impossible and unethical to replicate. According to the commenter, considering the severity of the health outcomes, dismissing or downgrading this critical research due to the Agency's proposed rigid transparency rule may result in regulatory delays in EPA's charge to protect human and environmental health.
Commenter (11403) provides that many public health studies -- including those focusing on acute environmental impacts -- are not reproducible because reproducing them would require intentional, unethical exposure of human beings, other living beings, or the environment to toxins. The commenter notes that this is not a limitation, nor does it discount the importance and reliability of those studies. According to the commenter, ignoring or giving low priority to peer-reviewed studies because authors are unable to publish raw data or because it is not reproducible is arbitrary and capricious.
Commenter (11479) notes that in addition to privacy, trade secret, intellectual property, and other confidentiality concerns with the underlying data, certain studies, such as observational studies, must be used and it is often not possible or ethical to recreate the conditions under which people were exposed to a contaminant. The commenter contends that the provisions in the proposed rule that require reproducibility are inconsistent with provisions in OMB guidance that state the reproducibility standard cannot be applied to all science.
Commenter (12720) states that the reference dose for mercury is based on an extensive analysis and calculations derived from three longitudinal epidemiologic studies: the Seychelles Islands, the Faroe Islands, and the New Zealand studies. The commenter notes that the studies can no longer be reproduced, particularly the Faroe Islands study in which the exposure to the community was a result of eating whales, a practice that has since declined due to public alerts about the hazards of eating the mercury-tainted meat particularly for children and pregnant and breastfeeding women. In addition, the commenter provides that it would take decades to repeat the studies, which took decades to conduct in the first place. According to the commenter, if EPA may no longer rely on these studies under the supplemental proposal, the Agency will not have access to the information it needs to ensure its regulations are sufficiently protective of public health.
Commenter (11900) contends that many foundational human environmental public health studies cannot be replicated because people are no longer exposed to the hazards at high levels -- often thanks to public health action taken in response to these same studies. The commenter provides that replicating the high levels of exposure observed in these studies is unethical and impossible, because it would involve intentionally exposing people to a harmful chemical. Even if these studies could be repeated, the commenter states that doing so would be a tremendous waste of time and funds. As such the commenter asserts that research initiated prior to the effective date of this rule be exempt from the requirement of the rule.
Response: The Agency recognizes that the 2018 NPRM used terms that were not fully defined or not defined at all. The 2020 SNPRM addressed a number of these comments. To address the concerns expressed by commenters, in this final rule, in Part 30.2, the EPA is defining the following terms: "data," "dose-response data," "independent validation," "influential scientific information," "pivotal science," "publicly available," "reanalyze," "science that serves as the basis for informing a significant regulatory action," and "significant regulatory actions." The terms "pivotal regulatory science," "replication," "reproducible," and "research data" are not used in this final rule. 
This final rule establishes how the EPA will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information. When promulgating significant regulatory actions or developing influential scientific information for which the conclusions are driven by the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect, the EPA will give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. The final rule does not require the EPA to reproduce or evaluate the reproducibility of dose-response data.
3.2.1.4.3 Availability of Resources
Information from Previous Studies is Unavailable
Comment: Commenter (11576) notes that there are studies for which the availability of information is limited and cannot be replicated.
Response: In the final rule the term replicate is not used. The EPA is clarifying that the intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze."
Further, although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7.
Certain Studies May Not be Reproducible with Standard Resources
Comment: Commenter (11389) provides that some data, such as the Tropospheric Monitoring Instrument (TROPOMI) methane concentration dataset, are not easily manipulated without significant computing resources unavailable on a typical personal computer. The commenter states that, as a result, studies conducted using these data are not necessarily reproducible by a person without these resources.
Response: The Agency recognizes that the 2018 NPRM used terms that were not fully defined or not defined at all. The 2020 SNPRM addressed a number of these comments. To address the concerns expressed by commenters, in this final rule, in Part 30.2, the EPA is defining the following terms: "data," "dose-response data," "independent validation," "influential scientific information," "pivotal science," "publicly available," "reanalyze," "science that serves as the basis for informing a significant regulatory action," and "significant regulatory actions." The terms "pivotal regulatory science," "replication," "reproducible," and "research data" are not used in this final rule. 
This final rule establishes how the EPA will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information. When promulgating significant regulatory actions or developing influential scientific information for which the conclusions are driven by the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect, the EPA will give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. The final rule does not require the EPA to reproduce or evaluate the reproducibility of dose-response data. Further, the EPA includes the term "subject matter expert" in its definition of independent validation to acknowledge that this work requires some level of expertise.
3.2.1.4.4 Depersonalized Data Does Not Allow for Reproduction
Comment: Commenter (11921) asserts that it is not possible to conduct a high-quality air pollution and health study without knowing the locations of those being studied, i.e. where they live, the sources and levels of their air pollution exposure, as well as their specific covariates, and specific health outcomes. The commenter states that, since the goal should be to find ways to share data that enables full replication and sensitivity analysis of original studies, it is valuable to consider several aspects of large population air pollution studies that have moved them towards utilizing data at smaller spatial scales, including: 
          the utilization of land use regression and other methods that can account for the distance of a subject's home from roadways, industrial facilities, and other sources of air pollution and finer-grained community-level covariates to address earlier air pollution study reliance on central air quality monitoring data, 
          as well as address two potentially significant sources of uncertainty in results: the so-called "ecological confounding"29 and "spatial autocorrelation."30, by using data at smaller scales. 
The commenter notes that while these have generally enhanced the quality and sensitivity of the studies, these characteristics increase the difficulty of providing a fully "de-identified" data set while also enabling a different investigator to conduct a full replication and sensitivity analysis of the original study results. As such, the commenter asserts that analyses using more limited data sets may well therefore result in incomplete and misleading results.
Response: The Agency recognizes that the 2018 NPRM used terms that were not fully defined or not defined at all. The 2020 SNPRM addressed a number of these comments. To address the concerns expressed by commenters, in this final rule, in Part 30.2, the EPA is defining the following terms: "data," "dose-response data," "independent validation," "influential scientific information," "pivotal science," "publicly available," "reanalyze," "science that serves as the basis for informing a significant regulatory action," and "significant regulatory actions." The terms "pivotal regulatory science," "replication," "reproducible," and "research data" are not used in this final rule. 
This final rule establishes how the EPA will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information. When promulgating significant regulatory actions or developing influential scientific information for which the conclusions are driven by the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect, the EPA will give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation. The final rule does not require the EPA to reproduce or evaluate the reproducibility of dose-response data. Further, the EPA includes the term "subject matter expert" in its definition of independent validation to acknowledge that this work requires some level of expertise.
3.2.2 Reanalyze
3.2.2.1 EPA's Position on Reanalysis
Comment: Commenter (12645) states that on a particularly important point, the definition of the science subject to EPA's new requirement, EPA has a lengthy paragraph in the Federal Register notice that is almost word-for-word from the National Academies of Science (NAS) Workshop report's account of what was said by Dr Lynn Goldman who chaired the workshop. The commenter notes that no citation is given to the speaker. The commenter provides below the wording in the workshop report to which the commenter has added track changes to demonstrate how EPA has edited these remarks to remove any language that would make it seem other than an NAS consensus report. The commenter provides EPA's language from the Federal Register following the statements from the NAS workshop.
      [The commenter provides Dr. Goldman's statement with tracked changes from a NAS Workshop on Principles and Obstacles for Sharing Data from Environmental Health Research. The tracked changes are shown here with strikethrough. The Federal Register language is provided afterwards and is underlined for comparison.]
       There are various approaches to testing and validating previous scientific work, she said." A reanalysis is when you conduct a further analysis of data." A person doing a reanalysis of data may use the same programs and statistical methodologies that were originally used to analyze the data or may use alternative methodologies, but the point is to analyze exactly the same data and see if the same result emerges from the analysis.
      "Replication means that you actually repeat a scientific experiment or a trial to obtain a consistent result," she continued,. The second experiment uses exactly the same protocols and statistical programs but with data from a different population. The goal is to see if the same results hold with data from a different population.
      "And then, finally, When you reproduce, you are producing something that is very similar to that research, but it is in a different medium or context," she said. In other words, a researcher who is reproducing an experiment addresses the same research question but from a different angle than the original researcher did. "Most of us, when we are doing systematic reviews, are more convinced that something is going on when we see reproducibility as well as replicability
      [Federal Register language]
      A reanalysis is when you conduct a further analysis of data. A person doing a reanalysis of data may use the same programs and statistical methodologies that were originally used to analyze the data or may use alternative methodologies, but the point is to analyze exactly the same data and see if the same result emerges from the analysis.
      Replication means that you actually repeat a scientific experiment or a trial to obtain a consistent result,. The second experiment uses exactly the same protocols and statistical programs but with data from a different population. The goal is to see if the same results hold with data from a different population.
      When you reproduce, you are producing something that is very similar to that research, but it is in a different medium or context. In other words, a researcher who is reproducing an experiment addresses the same research question but from a different angle than the original researcher did.
Commenter (12645) states further that by removing the quotation marks and "she said" language from the Workshop statement, EPA hides the fact that this was the opinion of an individual speaker rather than the consensus of an NAS Committee. The commenter suggests that EPA's omission of the last sentence from the workshop paragraph which clearly is at odds with EPA's goals was also highly misleading. The commenter notes that stated in that summary sentence is that environmental health science inherently moves forward by "reproducibility as well as replicability" which is contrary to EPA's focus on reanalysis of existing data by industry consultants. The commenter notes that had it been known to the readers of the Federal Register announcement that it was Dr Goldman who was being directly quoted they could have looked up her views on EPA's transparency proposal where they would have found that she is in fact strongly opposed to this action. The commenter states that the standard way of deciding what influences a decision in both science and law is to look at the references cited which clearly have influence, or they would not have been cited. The commenter suggests that another reasonable inference is that, "pivotal" findings should certainly be among these references, and would likely be among the most highly referenced. The commenter adds that in both science and law, the references chosen to be cited are crucial determinants of the probity and validity of the argument advanced, so it is highly ironic that EPA has misled the public in this disgraceful way.
Commenter (12645) asserts that EPA's heavy reliance on the NAS is also ironic in that the current EPA leadership has steadfastly refused to consult the Academies on any of the issues related to its attempts to change the scientific processes at EPA. The commenter states that, similarly, in violation of congressional mandates, EPA has not involved its external scientific committees in considering the rationale for any of its major proposed changes in EPA's scientific processes. 
Commenter (12645) states that it is unclear as to whether this deception is unintentional or intentional. The commenter asserts that if unintentional, it is one more piece of evidence as to how out of touch EPA's policy leadership is with its own scientists, let alone the scientific community. The commenter argues that presence of appropriate quotation marks in a footnote in the Federal Register notice describing an OMB directive supports the likelihood that the intention was to deceive.
Response: First, EPA disagrees with the commenter's implication that EPA was being misleading. EPA explicitly explained the source of the information it was using for the definitions in the preamble. Indeed, EPA clearly identified it as an NAS workshop report. Further, the document is easily obtainable, so that any reader of the Federal Register notice for the supplemental proposal could easily obtain the document. Second, the commenter is correct that anyone could easily retrieve Dr. Goldman's opinions as well as her extensive body of work in this area. In summary, EPA disagrees that the Agency was being deceptive. 
As to the commenter's comment about Dr. Goldman's opinions, her participation on the 2016 NAS Workshop predates 2018 proposed rule. Thus, her opinions on the 2018 proposed rule are not relevant to her participation in the 2016 NAS workshop. 
The EPA consulted with the Science Advisory Board (SAB) on the 2018 proposed rule in August 2019. In addition to the consultation, the SAB elected to review the scientific and technical basis of the 2018 proposed rule and provided a report to the EPA in April 2020. In the report, the SAB recognized the importance of the purpose of this rule and provided recommendations to improve its technical feasibility. Comments related to EPA's consultation with NAS or SAB on issues unrelated to this rule are outside the scope of this rulemaking.
Finally as to the comment that summary sentence is that environmental health science inherently moves forward by "reproducibility as well as replicability," EPA agrees. However, the focus of this rule is transparency which does not conflict with this statement from the 2016 NAS workshop report. 
Comment: Commenter (11381) suggests that if the Information Quality Guideline (IQG) framework were used as the baseline, reanalyze/reanalysis has no value-added. The commenter states that under the IQGs, it does not matter whether information is original or derivative; it is all covered if it is disseminated by an Agency. The commenter adds that under the IQGs, a "high degree of transparency is required if the information is influential," again, independent of whether the information or original or derivative.
Response: While this rule builds on the IQG, it is not equivalent. The rule applies to a different stage of data than that covered by the IQG. This rule applies to an earlier stage of data than discussed in the IQG. In particular, see the 2020 SNPRM Section III.B for a discussion of stage of data and Section III.C of the preamble to the final rule. Reanalysis of the earlier stage of data provides for a greater degree of transparency.
Commenter (12699) states that it is unclear why EPA only mentions "data and models" at the start of § 30.5, as this is just a subset of information that underlies pivotal regulatory science and pivotal science. The commenter suggests that EPA appears to recognize this in both versions of § 30.5, when it states, "information is considered `available in a manner sufficient for independent validation' when it includes the information necessary to understand, assess, and reanalyze findings." The commenter recommends that EPA should be focusing § 30.5 on all information necessary to understand, assess, and reanalyze findings (basically, in plain English, as a general rule, the public should have access to whatever information is necessary to evaluate a study). The commenter states that the list of examples in § 30.5 includes some of the additional information that is necessary, including computer codes and "associated protocols necessary to understand, assess, and extend conclusions."
Response: This rule applies to dose-response data not data and models. EPA has modified 40 CFR 30.5 to incorporate many of the suggestions by the commenter. This includes by adding the following l language to 40 CFR 30.5(b): "The EPA will use the following factors to assess the quality of studies identified in the systematic review: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review". The information needed to assess these factors are generally included in a study when published and available to the public.
Comment: Commenter (11892) suggests that the proposed rule should consider alternative sites, or methods, rather than the same data for a reanalysis, and these can also be corroborative findings or show limitations or expansion of using results. The commenter notes that this is particularly for field studies in wetlands and waters, where regional site characteristics may provide differing results and that reasons for the difference can be further explored, but some results may be more applicable for certain uses or areas than others and should be considered. 
Response: The commenter describes essentially reproducing results as described in the 2016 NAS Workshop report: "When you reproduce, you are producing something that is very similar to that research, but it is in a different medium or context." While reproducing a study is helpful, it does not result in increasing the transparency of a study that EPA uses as pivotal science which is the purpose of the rule.
Commenter (12712) does not support the modified rule to the extent that it suggests that significant EPA regulations may give "consideration" to studies based on data that is neither publicly available nor available through tiered access. The commenter states that this is too vague. The commenter believes that no such study, by definition unavailable for reanalysis by independent researchers, should even be a factor in an EPA decision to regulate. The commenter believes that the requirement in many of the statutes that EPA administers to the effect that regulations must be based on the "best available science" fully supports, and indeed may be argued to require the transparency rule. The commenter asserts that science proceeds by replication -- and what the rule calls reanalysis -- and without data availability, this process cannot happen.
Response: EPA disagrees that its approach is too vague. EPA has listed factors at 40 CFR 30.5 that it shall consider when determining the consideration to afford pivotal science for which the dose-response data are not available for independent validation. These are:
 the quality of the study relative to other studies for which the dose-response data are available; 
 the extent to which there are other studies for which the dose-response data are available;
 the sensitivity of the conclusions in the significant regulatory action or influential scientific information based on the use of the study;
 the extent to which the study is fit for the purpose or intended use relative to other pivotal science for which the dose-response data are available;
 the use of exposures or doses in a range and duration that is relevant for the intended use and that minimizes the need for extrapolations;
 the extent to which the study is supported by other scientific evidence;
 the extent to which the study accounted for unique scientific considerations;
 the extent to which the study minimizes the use of defaults and assumptions, uses appropriate and strong statistical methods, and includes a robust representation of uncertainty and confidence intervals; and
 the study's consideration of a broad range of parametric dose-response or concentration-response models, a robust set of potential confounding variables, nonparametric models that incorporate fewer assumptions, various threshold models across the dose or exposure range, and models that investigate factors that might account for spatial heterogeneity.

Further, as EPA has described in Section III.E of the preamble, only using studies in which the underlying dose-response data are available for independent validation would categorically exclude many valid studies. How EPA and the courts have interpreted the requirement to use best available science under the environmental statutes EPA administers is not consistent with a categorical exclusion of studies that include dose-response data that cannot be made available for independent validation. 
Comment: Commenter (12715) provides that in some cases, the data and models used in a reanalysis would not be available to the public. Therefore, the commenter states that the EPA could use a reanalysis, its own or that of another party, which differs from the analysis provided in the original publication of the study, and the data and models underlying this reanalysis would not be available to the public. According to the commenter, this conflicts with EPA's stated goal of increasing transparency.
Response: Upon consideration of the comments, the EPA agrees that it is important not to categorically exclude any study because the data underlying a study at the stage required by this rulemaking may not be available for independent validation. EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Further, the rule does not require any entity, including EPA, to conduct independent validation.
3.2.2.2 Definition of "Reanalyze" and Related Terms
Comment: Commenters (9587, 11401, 12406) support the clarity achieved by defining "Capable of being substantially reproduced", "Data", "Independent validation", "Influential scientific information", "Model", "Pivotal science", ``Publicly available" and "Reanalyze". 
Response: EPA notes the commenters support. However, as described in the preamble to the rule, EPA is not finalizing the definitions of "capable of being substantially reproduced" or "model."
Comment: Commenter (9587) notes that EPA would be well-served by a systematic reform to provide clear definitions of all terms relevant to policymaking. The commenter suggests that EPA, following government best practices in the Department of Defense, institute a formal ontology to establish terminological exactitude in all policymaking. 
Response: This comment is outside of the scope of this rulemaking.
Comment: Commenter (12454) is concerned the proposed language could be interpreted as excluding subsequent reanalysis of the data used to develop a rule during the period between the proposed rule and the final rule. The commenter requests that EPA include language explicitly addressing data reanalysis in its final rule.
Response: The rule does not require that a reanalysis be done at any time or even at all. The rule requires that EPA give greater consideration to pivotal science where the underlying dose-response data are publicly available in a manner sufficient for independent validation.
Comment: Commenter (12727) suggests that EPA's proposed definition for "capable of being substantially reproduced" attempts to make "reanalysis" synonymous with "reproduce," which is wholly inappropriate. The commenter states that to use the term "reproduced" in this context is to mislead the public into believing that some new scientific insights have been applied to an original set of data such that the original research findings are made stronger. The commenter asserts that this definition highlights a fundamental misconception underpinning the premise of the proposal -- that somehow independent validation is accomplished through reanalysis of study data by other experts. The commenter argues that the strength of scientific findings are not determined by endless reanalysis of a single data set, but rather gained over time through the conduct of separate investigations utilizing different methodologies that yield corroborating conclusions around a hypothesis being explored.
Response: EPA has not finalized "capable of being substantially reproduced". In his rule independent validation of pivotal science occurs through the reanalysis of the underlying dose-response data. This rule does not define independent validation as reproducing results of a study in another context or with another population.
Comment: Commenter (12720) states that in the definition for "reanalyze", EPA does not restrict reanalysis to "subject matter experts" as it did for "independent validation." The commenter notes that EPA does not describe any assumptions underlying analyses that may lead a person to fail to arrive at the "same result" despite analyzing "exactly the same data" and does not clarify what the "same result" is to be interpreted as. The commenter states that this definition focuses on using the "same" data but allows for "the same or different statistical software, models, and statistical methodologies," yet expects "the same result" to emerge, which is problematic. The commenter asserts that this definition clashes with the reality that, even using the same underlying data, changes in software, modeling approach, and statistical techniques would reasonably be expected to deliver results that are not exactly "the same" as results produced in an original analyses. The commenter concludes that EPA's definition is unreasonable, illogical, and flawed.
Response: EPA has finalized the definition of "reanalyze" at 40 CFR 30.2 to specify the use of "the same methods, statistical software, models, or statistical methodologies that were used to analyze the dose-response data." 
There is no need to restrict "reanalysis" to "subject matter expert" because "reanalysis" is done for purposes of "independent validation." It is sufficient that the latter definition restricts it.
Comment: Commenter (11381) states that the supplemental proposal definition for "reanalyze" (and its cognate noun, reanalysis) is narrow; to be covered, a reanalysis must use "exactly the same data". The commenter suggests that this definition is incompatible with one of the Agency's stated purposes  -  "to assess potential analytical errors and variability in the underlying assumptions of the original analysis." The commenter states that such assessments typically require additional data, different data, or alternative formulations of the same data, none of which can be exactly the same. The commenter asks: What would happen under new Part 30 to information that is neither data nor models, or even if within those terms, not exactly the same? The commenter believes such information would be exempt from the provisions applicable to that which is included within reanalyze/reanalysis.
Response: Information that is not dose-response data would be outside the scope of the final rule. 
Comment: Commenter (12715) is concerned that the supplemental proposal's new definition of "reanalyze" could ultimately result in scientific decisions being made based on reanalysis of data with different models and assumptions than those used by the original researcher(s), thereby altering the scientific conclusions presented in the originally published and peer-reviewed scientific study. 
Response: The rule does not require that EPA or any other party reanalyze dose-response data before EPA can consider it in its decision making.
Comment: Commenter (11575) states that EPA's definition of reanalysis should not include the use of different statistical software, models, and methodologies, as it should limit the definition to only the use of the same methods. The commenter notes that the use of different statistical tests to assess the same data is a new analysis rather than a reanalysis. The commenter asserts that EPA has failed to specify the stage of data required for reanalysis and has not considered the implications of this definition on all types of data and models expected to be impacted by the expanded scope of this rulemaking. The commenter states that in the absence of specifying a consistent framework for the types of statistical analyses required for specific types of data to ensure consistency across reanalyses, the ambiguity in the definition of reanalysis invites opportunities for manufacturing uncertainty through conflicting results from alternative statistical tests. The commenter asserts that this approach inherently violates standard scientific process as attempts for reanalysis will disproportionately be targeted towards findings some would like to undermine. Commenter (11495) states that EPA has clarified in the supplemental proposal that the proposed actions are no longer premised on "replication" but rather on "reanalysis" of published scientific studies. The commenter adds that the existing discussion of reanalyzing data is problematic, with potentially severe consequences to quality and pragmatism. The commenter suggests that using the same data, program and statistical methodologies should yield similar results to published studies and is a viable definition of what it could mean to perform a reanalysis of a published study. The commenter notes that using different statistical software while using the same data and methodologies could also be a potentially useful form of reanalysis to ensure that results are not driven by quirks in the specific software used by study authors but this rule goes far beyond that definition.
Commenter (11495) states that in terms of the provided definition in the supplemental proposal, reanalysis using "alternative methodologies" should not necessarily be expected to deliver the same results as the original study. The commenter states that even in cases when alternative methodologies do provide the same results, these results would need to be interpreted differently given that an alternative methodology was employed. The commenter believes this is a far cry from what is traditionally referred to as a reanalysis in which new information or variables are added to a previous analysis to test new research questions. The commenter states that reanalysis studies of this type can be informative, but it has never been the expectation that someone interested in conducting such a study is entitled to having someone else do the majority of the work in preparing all of the data for continued study.
Response: EPA has finalized the definition of "reanalyze" at 40 CFR 30.2 to specify the use of "the same methods, statistical software, models, or statistical methodologies that were used to analyze the dose-response data."
EPA has explicitly explained the stage of data that should be considered. In particular, see Section III.B. of the 2020 SNPRM which discusses different stages of data. Also, see Section III.C of the preamble to the final rule. 
EPA disagrees that reanalysis of data is equivalent to having someone else do the study for them. The study will be conducted and published by the first author(s). The person doing the reanalysis did not conduct the study, generate the data or publish the study. The person doing the reanalysis will use the data from the first study, reanalyze the data and determine if the same results were achieved. 
Comment: Commenter (11479) states that EPA's draft rule, as modified in the supplemental notice, reveals that the Agency wants to use original data and methods to reanalyze rules and prove that they are reproducible before using them to support a rule. The commenter claims that the supplemental notice still has not clarified what it means by "capable of being substantially reproduced," and it contradicts itself. The commenter states that in the notice, EPA defines "capable of being substantially reproduced" as "independent analysis of the original or supporting data using identical methods would generate similar analytic results, subject to an acceptable degree of imprecision or error" but goes on to cite a NAS workshop report that defines reproducibility as "producing something that is very similar to that research, but is in a different medium or context. In other words, a researcher who is reproducing an experiment addresses the same research question from a different angle than the original researcher did." The commenter believes that EPA is disguising its quest to cherry-pick evidence with unclear definitions of reanalysis, independent validation, and "capable of being substantially reproduced."
Response: EPA agrees with commenters that "capable of being substantially reproduced" is confusing and inconsistent with the other defined terms. EPA has not finalized a definition of "capable of being substantially reproduced." 
EPA disagrees that its definitions of "reanalyze" and "independent validation" are an attempt to cherry pick evidence. The definition of "reanalyze" is based on and indeed is very close to the definition in the 2016 NAS Workshop report cited and discussed in Section III.B of the 2020 SNPRM. The definition of "independent validation" hinges in large part on the definition of "reanalzye."
Comment: Commenter (11392) states that the proposed definition for "reanalyze" would introduce the risk of data manipulation into the Agency's regulatory process. The commenter states that by naming "different programs and statistical methodologies" as legitimate methods for the reanalysis of a body of data, the Agency would permit external entities with vested interests to deliberately deploy biased methodologies in order to reach different results from the same data. The commenter states that analyzing data with different methodologies will frequently lead to different results, but the Agency does not explain how the existence of different results would help improve the Agency's policymaking. The commenter suggests that EPA lacks any objective means to evaluate the superiority of one methodology over another as a part of the reanalysis process. The commenter asserts that the proposed definition would therefore result in confusion about how a study should be interpreted and undermine the role of scientific data in the Agency's regulatory process. The commenter recommends that EPA clarify why its proposed definition of "reanalyze" permits different methodologies to be used in reanalyzing study data, describe the process that it will utilize to assess divergent methodologies, and justify how the definition is consistent with the Agency's obligation to consider the best available science. 
Commenter (11892) recommends developing different terms for analyzing the exact same data and method vs. different method or model for analysis. The commenter notes that models can be biased toward certain outcomes and recommends measures be in place to prevent rejection or acceptance of results due to bias in analysis.
Commenter (11576) states that the EPA's definition of reanalysis should not include the use of different statistical software, models, and methodologies, as it should limit the definition to only the use of the same methods. The commenter asserts that the use of different statistical tests to assess the same data is a new analysis rather than a reanalysis. 
Response: EPA has taken into consideration the comments and has finalized the definition of "reanalyze" at 40 CFR 30.2 to specify the use of "the same methods, statistical software, models, or statistical methodologies that were used to analyze the dose-response data." 
3.2.2.3 Issues Associated with Reanalysis
3.2.2.3.1 Protocols are Needed for Reanalysis
Comment: Commenter (11495) states that the supplemental proposal only requires data be made publicly available to support reanalysis but does not discuss circumstances in which reanalysis might actually occur and how the Agency would respond to such activities. 
Commenter (12617) asserts that the supplemental proposal's expansion of research that may have to be reanalyzed is unrealistic and vulnerable to manipulation. The commenter notes that when imagining what reanalyzing even a portion of relevant studies would look like, it sounds like a plan to hopelessly bog down the regulatory process. The commenter states that the Agency gets to choose which studies must be reanalyzed -- by unknown criteria -- by "subject matter experts" and "stakeholders." The commenter adds that, according to the proposal, reanalysis means not just performing the same data processing as the study authors but also experimenting with adding additional variables to models and additional assumptions and methods in the statistical analysis. The commenter asserts that a single study could hold up crucial rulemaking for years, perhaps indefinitely, as the Agency's chosen subject matter experts and stakeholders go through these layers of reanalysis --  especially if a stakeholder has an interest in slowing down or preventing a rule that, say, limits pollution.
Commenter (12720) reiterates comments provided in the SAB's review of the supplemental proposal: 
      The SAB finds that requiring the identification of all studies and regulatory science supporting regulatory actions and making them available for reanalysis will be a complex process. As previously discussed, identifying and making "pivotal science data or studies" available could present challenges if some studies were considered in the regulatory decisions but were not used to determine the point of departure (POD) or reference dose (RfD) or other regulatory/technical level. It is not clear how much information and which studies should be included in the requirement to identify and make studies available. As further discussed in Sections 3.2 and 3.3 of this report, if the intent of the rule is to make available for reanalysis "all underlying pivotal science supporting influential scientific information and/or pivotal regulatory science" that contributed to the ultimate regulatory decision, then practical procedures must be established for an independent validation. 
Response: The rule does not require that EPA or any other party reanalyze dose-response data and consider it in decision making. Thus, this rule does not include requirements for how a reanalysis should be conducted or how it should be used.
EPA will identify the pivotal science that includes dose-response data available for independent validation at the proposed rule stage for significant regulatory action and when influential scientific information is submitted for peer review which will provide the public the opportunity to comment on EPA's characterization and choice.
Comment: Commenter (11186) states that the rule does not provide information on the logistics of reanalysis. The commenter questions who would do the reanalysis and how much would this cost. The commenter notes that to access data, there would need to be a process involving an extensive protocol, servers to store data, and trained and qualified "re-analyzers", which would result in tremendous costs and time that would be better spent in research studying the prevention and treatment of adverse health outcomes. 
Response: The rule does not require that EPA or any other party reanalyze dose-response data and consider it in decision making. 
This rule does not require that any party either access data or make data available for independent validation.
Comment: Commenter (11496) asserts that re-analysis of studies already peer-reviewed and published in the scientific literature is not justified. The commenter does not consider the reanalysis of peer-reviewed and published data a legitimate purpose since pivotal regulatory decisions are already peer-reviewed and supported by a broad weight and range of evidence. The commenter states that even if distribution of private health information for reanalysis is a valid research proposal, EPA has not articulated how reanalysis requests and plans will be vetted or peer-reviewed and how the results of these exercises will be published or shared in a transparent process.
Response: The rule does not require that EPA or any other party reanalyze dose-response data before EPA can consider it in its decision making.
Comment: Commenter (10339) states that practical details about the implementation of the rule remain unresolved, including many issues raised by the EPA's Science Advisory Board draft report on the scientific and technical basis for the rule. The commenter notes that it is not clear precisely if/how EPA will pursue tiered access to data or make a final determination on whether the level of access is sufficient to be compliant with the rule.
Response: EPA will address how it will determine whether dose-response data are available for independent validation as it develops its implementation processes for the rule. The final rule does not require EPA to obtain, store or disseminate dose-response data nor does it require EPA to conduct independent validation of pivotal science.
3.2.2.3.2 Resources Required for Reanalysis
Comment: Several commenters (11921, 11408, 11492) state that is would be costly to EPA and external parties to reanalyze all the studies given the large number of studies with data and models. 
Commenter (11495) states that the Supplemental Notice creates a new, complex and potentially resource- and time-consuming process at EPA without any discussion of parameters by which they would respond to the very activity that they are hoping to enable.
Response: The rule does not require that EPA or any other party reanalyze studies before EPA can consider it in its decision making. 
3.2.2.3.3 Conflicting Reanalysis Results
Comment: Commenter (11361) notes that a crucial piece of omitted information in the supplemental proposal and the 2018 proposal relates to what will happen in the event of conflicting reanalyses of data. The commenter states that there is a complete lack of information provided about what could happen if an attempted independent validation conflicted with original study results. The commenter states that EPA provides no metrics for evaluating the appropriateness of methods used in a reanalysis, and conspicuously, provides no information about how a conflicting reanalysis result would impact the EPA's consideration of a study. The commenter observes that this obvious omission introduces another way in which this proposed rule could be manipulated and result in the dismissal of credible scientific studies.
Response: This rule is not intended to address the results of the independent validation through reanalysis. The purpose of the rule is transparency. EPA is increasing the transparency of its significant regulatory actions and influential scientific information by giving greater consideration to more transparent studies. 
3.2.2.3.4 Potential for Bias
Comment: Commenter (11479) states that reanalyzing every study used by the EPA is unattainable and setting a goal of reanalysis may give stakeholders excessive power to weaken EPA's regulatory decisions. The commenter suggests that under the supplementary proposal, industry and others could pick apart the modeling done by the agency to assess residual risk of air pollution impacts of industrial facilities on communities and then challenge the agency's assessment of risks to communities.
Commenter (9455) states that allowing regulated industry-funded scientists to flood the regulatory process with "reanalyses", new "studies", and "independent validations" invites corruption of the regulatory process and failure to generate needed standards and policies. The commenter states that rather than introducing greater transparency and better science, this unethical program will muddy the process and introduce biased pseudoscience. 
Commenter (9455) states that EPA wants all studies that support public policies and/or standard-setting to be posted for re-analysis and alternative modeling. The commenter is concerned that this opens the regulatory process to infinite delays and infinite debate without any clear scientific advantage. The commenter notes that the regulated industries have large resources to fund analyses and critiques that "prove" there is no need for regulation or other policy-making. The commenter states that in this proposed rule, EPA has failed to develop mechanisms for preventing unnecessary delays in setting needed standards or generating needed policy, as well as mechanisms to prevent conflict-of-interest bias from entering into data assessment, re-analyses, and alternative modeling. The commenter adds that nothing is said about assessing the quality of the re-analyses, alternative modeling, or independent validations. Commenter (11575) notes that several examples exist within public-health and epidemiological studies where reanalysis funded by industry groups resulted in the emergence of less protective results related to toxicity or mortality estimates for different compounds. The commenter concludes that without consistent guidelines for when reanalysis is required and what statistical tests are utilized, the proposal risks introducing bias through "statistical fishing expeditions."
Response: EPA disagrees that the rule will lead to the weakening of EPA's regulatory decisions. The rule does not require EPA or any other party, including stakeholders, to pivotal science based on dose-response data before EPA can use those studies. Stakeholders and other interested parties can raise challenges to the science underlying EPA's significant regulatory actions and influential scientific information regardless of whether they reanalyze the studies in question. 
Comment: Commenter (11495) states that there is a lack of clarity over who the "stakeholders" involved in data reanalysis may be, and the suggestion that they may cherry pick data to force into models that support "alternative assumptions" goes directly against the scientific process and certainly looks at least superficially like exactly the sort of bias the EPA is hoping to avoid with their rule. 
Response: It is not necessary or possible for EPA to describe the universe of stakeholders that may decide to conduct independent validation of pivotal science used in significant regulatory actions and influential scientific information. Also, the rule does not require any entity, including EPA, to conduct independent validation before EPA can consider pivotal science based on dose-response data.
Comment: Commenter (12715) states that, although the supplemental proposal requires that data be available in a manner that allows reanalysis, it does not require that EPA perform a reanalysis of the data of every study that is used as the basis for decision-making, and no criteria for determining when a reanalysis would be performed are provided. The commenter states that, therefore, EPA could selectively decide whether to rely on a reanalysis based on how the results of the reanalysis compare with the original analysis performed by the researcher and EPA's preferred outcome. The commenter notes that it is not clear whether the sitting Administrator -- a political appointee -- could require independent validation of studies that he or she finds objectionable. The commenter suggests that all of this is contrary to the proposed rule's stated goal of increasing transparency. 
Response: The rule does not require that reanalysis be conducted nor does it designate that a reanalysis be given preferential treatment as pivotal science. Any reanalyses would be considered with other information provided to EPA as it develops significant regulatory actions and influential scientific information.
3.2.2.3.5 Reanalysis Requirements May Restrict Use of the Best Science
Comment: Commenter (11392) states that the proposed definition for "reanalyze" does not address how EPA would incorporate any completed reanalyses into its policymaking. The commenter adds that a number of critical questions remain undiscussed in the supplemental proposal: Would the Agency be required to consider a reanalysis alongside the original study, or would it pick and choose analyses to consider? Will the Agency publicly explain its decision to incorporate a reanalysis into a work product? How would the Agency compare a reanalysis to the original study? Will a reanalysis be required to go through peer review? If a reanalysis does not undergo peer review, would it receive equal consideration to an original peer-reviewed study, or would the Agency consider the absence of peer review as a factor in determining the value of the reanalysis? Would the Agency be compelled to delay the finalization of a rulemaking until the completion of a relevant reanalysis? The commenter states that all of these questions represent important practical considerations for the Agency. The commenter observes that if the Agency affords equal weight to any non-peer reviewed reanalysis as it does for a peer reviewed study, it calls into question whether Agency rulemakings are truly considering the best available science. The commenter notes that this also introduces questions about how EPA is satisfying the Information Quality Act, which requires that all significant regulatory information from federal agencies be peer reviewed, as well as EPA's own information quality guidelines.
Response: If a reanalysis of a study were considered pivotal science, it would need to be peer reviewed consistent with 40 CFR 30.6. Nothing in this rule requires independent validation through reanalysis, nor does the rule specify giving greater weight to the reanalysis of studies. Thus, the rule does not require that the timing of a significant regulatory action or influential scientific information be dependent on independent validation through reanalysis of pivotal science. This rule does not require that a reanalysis be considered alongside the original study.
Comment: Commenter (11395) states that reanalysis of data is only one of several mechanisms available for validating studies. The commenter notes that the scientific community uses numerous tools to validate studies, including rigorous peer review, replication of a study using the same methodology but with different data sources, or reproduction of a study's conclusions using different methodologies and data, and notes that many of these tools do not require access to all underlying study data. The commenter states that under the proposed rule, a peer reviewed study that has been replicated many times by different investigators using different data may be excluded from consideration, while one that uses an inferior database that is publicly available would be included.
Response: There is no categorical exemption for pivotal science for which the underlying dose-response data are not available. Further, as discussed in Section III.E of the preamble to the rule, in determining the consideration to give pivotal science for which the underlying dose-response data are not available, EPA will consider a range of factors in addition to the availability and the quality of the study.
3.2.2.3.6 Tiered Access and Restricted Data Issues Related to Reanalysis
Comment: Commenter (11921) states that although the proposed supplemental proposal acknowledges that personal information may not be able to be protected in all cases, and therefore "tiered access" to these data may be suitable, it is critical to recognize that while "depersonalized" data sets can be created, in many instances they will not allow for full replication and reanalysis  -  and could lead to incomplete and misleading results. The commenter notes that some have argued that it should be possible to create a "depersonalized" data set by stripping all personal identifiers such as address, date of birth, medical history, etc. and making such a data set widely available. The commenter suggests that it is not possible to conduct a high-quality air pollution and health study without knowing the locations of those being studied, i.e. where they live, the sources and levels of their air pollution exposure, as well as their specific covariates, and specific health outcomes, and unfortunately, should such information be made available at smaller spatial scale, it is possible to disclose extensive personal and medical information for individual study subjects, raising privacy and ethical concerns.
Commenter (12720) states that the claim that publicly available dose response data and models would allow for independent validation stands in direct contradiction to the legal privacy protections that apply to key data necessary for re-analysis. The commenter states that the proposed partial redaction of sensitive information poses a cascading set of problems, because the statistical models characteristic of epidemiologic investigations rely on the inclusion of potentially confounding variables (e.g., age, sex, home address, health status, diet and alcohol consumption, smoking history) in order to properly isolate the pollution-health relationship with precision. The commenter states that these studies analyze detailed health, demographic, spatial, and behavioral information from thousands of people to understand the dose-response connection and this information is extremely sensitive and collected at the individual level. The commenter notes that as such, our nation's health privacy laws and IRB protocols require researchers to keep the data secure and confidential to prevent misuse. The commenter states that collectively, these data points help researchers understand and isolate the cause-effect relationship between exposure to air pollution and risks for various health problems and notes that it would be extremely difficult if not impossible for anyone using partially redacted data sets derived from epidemiologic cohort studies to "validate" the results of the original studies, because such investigators would not be working with complete data sets. The commenter adds that as further demonstration, the 2009 Integrated Science Assessment for PM2.5 notes that "[a]ppropriate statistical adjustment for confounders requires identifying and measuring all reasonably expected confounders," and, therefore, exclusion of some potentially sensitive confounding variables from an underlying dataset likely would lead a different team of investigators to a different result. The commenter asserts that causing this wrongheaded and indefensible outcome results from the core approach and conceit in the Proposal, revealing it to be yet again, arbitrary and capricious and an abuse of EPA discretion. The commenter states that, put another way, the quantitative findings of dose-response relationships would almost certainly differ -- not as a result of any true difference in the quantitative exposure-effect relationship, but because the original work relied on complete data sets and the new analyses would not -- due to the Proposal. The commenter concludes that the resulting discrepancies in quantitative findings could serve as motivation to call the original study results into question due to faulty and incomplete re-analyses.
Response: Nothing in this rule require the disclosure of personally identifiable information (PII) or other sensitive information. Rather, this rule describes how EPA will handle studies based on whether the underlying dose-response data are available. Further, as stated elsewhere, this rule does not require that EPA or any other party conduct reanalysis of the dose-response data underlying pivotal science before EPA can use the data.
Comment: Commenter (11900) questions who will be able to access restricted data. The commenter states that the Supplemental Notice described the restricted access tier as "limited to authorized researchers and not possible for the general public." The commenter notes that EPA does not define a process for vetting applicants or criteria for granting access. The commenter states that restricted access must be limited to researchers with a legitimate scientific interest in the data, otherwise restricted access is simply `accessible to the public by application.' The commenter states that the Research Data Center (RDC) run by the National Center for Health Statistics (NCHS)/CDC -- identified as a possible model by the Supplemental Notice -- requires researchers "submit a research proposal outlining the need for restricted-use data." The commenter notes that a stated primary goal of the Supplemental Notice in making data available is to support reanalysis of study data. The commenter suggests that it is unclear on what basis applications seeking to perform reanalysis would be judged, because reanalysis is not typically considered research. The commenter recommends that EPA clearly define how applications to access restricted-access data will be evaluated.
Response: EPA does not intend to make all dose-response data underlying pivotal science available for independent validation. There may be instances where EPA does not own the dose-response data, lacks access to part or all of the dose-response data or does not have the authority to provide access to part or all of the dose-response data. Rather, this rule describes how EPA will handle studies based on whether the underlying dose-response data are available. Further, as stated elsewhere, this rule does not require that EPA or any other party conduct reanalysis of the dose-response data underlying pivotal science.
Comment: Commenter (12715) states that the supplemental proposal acknowledges that, in some cases, the data and models used in a reanalysis would not be available to the public (e.g., instances where EPA does not own the data and models, lacks access to part or all of the data and models, or does not have the authority to provide access to part or all of the data and models). The commenter suggests that this could result in EPA using a reanalysis, its own or that of another party, which differs from the analysis provided in the original publication of the study, and the data and models underlying this reanalysis would not be available to the public. The commenter states that this conflicts with EPA's stated goal of increasing transparency.
Response: In determining whether the underlying dose-response data are publicly available in a manner sufficient for independent validation, EPA will consider whether availability is such that a subject matter expert who has not contributed to the development of the study can evaluate whether results similar to those reported in the study are produced.
3.2.3 Independent Validation
3.2.3.1 Definition of "Independent Validation"
Comment: Commenter (11192) states that the definition the EPA introduces for "independent validation" narrows validation by using the same data and the same methods to see if similar results are produced. The commenter suggests that while one can deduce that independent validation is a subtype of reanalysis, that isn't detailed in the supplemental proposal, and could lead to confusion. 
Response: For the purposes of this rule, independent validation is not a subtype of reanalysis. Independent validation may be the result of reanalysis as it is defined in this rule.
Comment: Commenter (11405) states that the term "robustness" is not defined in the supplemental proposal; based on the examples it requires qualitatively similar findings using the same data but different analytical methods. The commenter states that because it involves different analytical methods and only qualitatively similar findings (not quantitatively identical outputs), robustness is not the same as reproducibility; it is closer to replicability. The commenter notes that, as defined in the supplemental proposal, the purpose of "independent validation" is to check reproducibility "using identical methods". The commenter concludes that, using these definitions strictly, an independent validation should not check for robustness by varying the methods.
Response: While robustness was used in the preambles of the 2018 proposed rule and the 2020 SNPRM, there is no requirement for robustness in this rule. Therefore, a definition of robustness is not included in 40 CFR 30.2.
Comment: Commenter (11381) states that the supplemental proposal's new term "independent validation", is unhelpful. The commenter notes that it focuses only on the adjective (independent) and ignores the noun (validation). The commenter adds that it conflates fundamentally different concepts: the "reanalysis of study data" and the independence of such a reanalysis ("by subject matter experts who have not contributed to the development of the study"). The commenter concludes that the definition permits obvious conflicts of interest to be ignored which could occur when the "subject matter experts" involved are not genuinely independent, or when they are employees of the Agency proposing to rely on study data as pivotal scientific information. The commenter states that while internal Agency reanalysis may not be sufficient, there is no rational basis for excluding it from a transparency rule.
Response: EPA disagrees that the definition does not focus on the noun "validation". The concept of validation is captured by the phrase "to evaluate whether results similar to those reported in the study are produced." The use of the term "independent" refers to being independent from the original study, that is by subject matter experts who have not contributed to the development of the study.
Comment: Commenter (12471) suggests clarifying the definition of "independent validation" as follows (underlined text is new text and struck out text is deleted text): 
      Independent validation means the reanalysis of study data by subject matter experts who have not contributed to the development of the original study to demonstrate that the same analytic results observed reported in the study are capable of being substantially reproduced.
Response: "Reported" is more appropriate than "observed" because the reanalysis would be conducted by someone other than the one who conducted the original study. The person who conducts the reanalysis can only compare her/his results to what the original researcher(s) reported, not what was observed.
Comment: Commenter (11381) states that the supplemental proposal cites some external authorities approvingly for definitions, including for example National Research Council (NRC) (2007) and NAS (2016). The commenter suggests that the 2007 report offered promiscuous advice on validation but never defined it, and the 2016 report did not address validation at all. The commenter notes that other NRC reports have addressed validation but were not cited in the supplemental proposal, including NRC (2012) and NAS (2017). The commenter states that the 2012 report is a 145-page treatise on verification, validation, and uncertainty quantification (VVUQ) that contains a definition that EPA could have adopted and the 2017 report included a definition of validation that is consistent with the information quality paradigm; it utilizes reproducibility and replicability (as configured in the Information Quality Guidelines (IQGs)) in the pursuit of reliability and accuracy. The commenter asserts that it is therefore hardly surprising that the Science Advisory Board (SAB) was spun up about the Agency's proposal to mandate validation without defining it.
Response: As discussed in the 2020 SNPRM, EPA evaluated the literature in addition to the NAS and NRC reports to aid in developing appropriate definitions. EPA also considered the many public comments received on the terminology used in the 2018 proposed rule, most notably the comment on replicate. Other than noting that there are other documents which have definitions for related terms which EPA could have adopted, this comment does not provide constructive comment on how to improve the proposed definitions. 
Comment: Commenter (12720) states that EPA's definition of "independent validation" in the supplemental proposal is inconsistent with the remainder of the proposal because it restricts the concept of "independent validation" to "subject matter experts who have not contributed to the development of the study." The commenter states that EPA restricts "independent validation" to an activity only conducted by "subject matter experts," a distinction in conflict with EPA's definition of "publicly available," which is more expansive and as such, it is unclear whether validation of study data by any member of the public is consistent with EPA's definition. The commenter notes that EPA fails to define "subject matter experts" nor explain how expertise is to be evaluated.
Response: EPA disagrees that there is a conflict. A subject matter expert is not someone who deals only with sensitive data. A subject matter expert is someone who has the education, experience and knowledge of a particular area or subject. The rule does not restrict activities of the public or subject matter experts. The sections cited by the commenter are definitions. The definition of independent validation includes subject matter expert because a subject matter expert would know for the particular type of dose-response study the appropriate methods to use to reanalyze the data to evaluate whether results similar to those reported in the study are produced. The definition of "publicly available" describes the availability of data and not the persons who might conduct independent validation.
Comment: Commenter (12720) states that the SAB's criticism underscores EPA's failure to adequately define terms in the Supplemental Proposal within the context of its scientific activities. The commenter states that, for example, the Supplemental Proposal makes no mention of internal and external validity. The commenter states that the proposal does not make any mention of the need to assess study validity, nor does it distinguish between internal validity (reduction of error and bias) and external validity (applicability of results to other populations, exposure settings, or time periods). Commenter (12720) states that it is highly problematic that EPA has not adequately defined "independent validation" within the proposal nor outlined clear criteria for assessing whether or not a study can be independently validated. The commenter adds that for meta-analyses and other studies that rely on multiple component studies, it is not clear what threshold would have to be met for the overall study to qualify as one that can be independently validated. The commenter asserts that the proposal does not address these difficulties and should be withdrawn.
Response: The determination of whether the dose-response data underlying a study will be available for independent validation will depend upon the specific study and the parameters of that study. Note that the rule does not require any entity to perform independent validation, including EPA. The threshold would be that a subject matter expert would have the dose-response data of the appropriate stage of data to be able to reanalyze the data. 
3.2.3.2 Issues Associated with Independent Validation
3.2.3.2.1 Implications for Researchers
Comment: Commenter (12727) states that EPA has not identified what information would be needed for independent validation of different areas of research. For example, the commenter suggests that the Final SAB Report raises additional points regarding the deficiencies in EPA's proposed definition for independent validation, highlighting the significant ambiguity that remains and insufficient attention to other, related considerations:
      EPA's supplemental proposal indicates that independent validation means the reanalysis of study data by subject matter experts who have not contributed to the development of the study to demonstrate that the same analytic results reported in the study are capable of being substantially reproduced. However, the specific definition of independent validation drives the feasibility of whether EPA can make data and models available for independent validation. For example, the EPA should consider the following questions: How much information is sufficient for independent validation? Will this information consist of equations where reviewers can verify the math, more detailed models where assumptions and limitations are described, or code to allow the public to evaluate and run the models if desired? Is this information simply the dose-response data for the endpoint of concern driving a regulatory limit, or is it availability of all data from a pivotal study to allow reviewers to examine the potential contributions of other variables on the primary endpoint of concern? Endpoint data are seldom evaluated in isolation so providing sufficient study information to allow an independent assessment seems important to meet the goals of the Proposed Rule. For example, an effect on pup body weights in a toxicology study should be examined with knowledge of maternal gestational body weight gains, litter size, food consumption, maternal/litter clinical signs, etc. Sample size and variability also play a key role in data interpretation.
Response: The determination of whether the dose-response data underlying a study will be available for independent validation will depend upon the specific study and the parameters of that study. The threshold would be that a subject matter expert would have the dose-response data of the appropriate stage of data to be able to reanalyze the data. 
Regarding endpoints, EPA agrees that endpoint data are seldom evaluated in isolation. This rule would not require that the dose-response data be only for the endpoint. It would be the dose-response data underlying the study.
Comment: Commenter (11921) states that a requirement that any pivotal study shall ensure that "data and models are available in a manner sufficient for independent validation" implies a complex and potentially costly and time-consuming set of steps that include making available two detailed and complex sets of information:
 The workflow that was implemented to create the analytical data set used for the analysis (e.g. harmonized in time and space) from the original and often heterogenous data sets (individual electronic health records, gridded exposure data, confounders etc.), and
 The data analysis (statistical software) that was used to analyze the analytical data set.
The commenter states that creation of both of these in a manner that enables even highly-qualified investigators to reproduce the same results is difficult and time consuming. The commenter suggests based on HEI's hands-on experience that this is likely to be costly to the researchers and their institutions  -  both in the creation of the data and methods hubs, and in the continuous requests for further explication from those seeking the data. The commenter notes that in contrast to the proposal's statement in response to Executive Order 13771: Reducing Regulations and Controlling Regulatory Costs: "This action is not expected to be an Executive Order 13771 regulatory action because it relates to ``Agency organization, management or personnel,'' the proposal would clearly impose substantial additional costs on a wide range of research institutions.
Response: The EPA disagrees that this rule would impose costs on third-party researchers. This is a rule of internal procedure that does not impose requirements on any party other than the EPA. This rule imposes no costs on researchers or their institutions, and the EPA will consider and evaluate all relevant and appropriate science in its significant regulatory actions and influential scientific information. The EPA recognizes that researchers and other third parties may voluntarily consider the EPA's position on data availability, as described in this rule, as they make their own decisions about how to conduct research and the extent to which they make data available. Researchers may choose to make more data available, but the EPA recognizes that these parties will weigh their own benefits and costs and make choices that they deem appropriate. 
Comment: Commenter (12430) asserts that the supplemental proposal would require publishing data so thoroughly redacted as to be effectively useless for validation.
Response: EPA disagrees with the commenter that the rule would require that thoroughly redacted data be published. This rule does not include requirements for publishing studies or data. 
Comment: Commenter (12422) states that some of the findings on which our public health protections are based are the result of studies done across the globe. The commenter suggests that it is unlikely that Canadian or European scientists would turn over their raw data to a U.S. Agency for "independent validation."
Response: Nothing in the rule requires scientists to provide raw data to EPA for independent validation. Further, the rule does not require any entity, including EPA, to conduct independent validation of studies
3.2.3.2.2 Conflicting Conclusions
Comment: Commenters (11192, 11361) state that a crucial piece of omitted information in the supplemental proposal and the proposed rule relates to the complete lack of information provided about what could happen if an attempted independent validation conflicted with original study results. The commenters suggest that this obvious omission introduces another way in which this proposed rule could be manipulated and result in the dismissal of credible scientific studies. The commenters state that EPA omits this pivotal information in the proposed rule and the supplemental proposal.
Response: The rule does not require that EPA or any other entity conduct independent validation before EPA can use pivotal science. Nor does it establish requirements for the use of the independent validation if it is conducted.
3.2.3.3 Current Processes Do Not Need Data to Be Publicly Available for Validation
Comment: Commenter (11479) states that EPA already independently validates the research without using underlying data by asking the following questions: 
      a) To what extent has there been independent verification or validation of the study method and results? What were the conclusions of these independent efforts, and are they consistent? b) To what extent has independent peer review been conducted of the study method and results, and how were the conclusions of this review taken into account? c) Has the procedure, method or model been used in similar, peer reviewed studies? Are the results consistent with other relevant studies? d) In the case of model-based information, to what extent has independent evaluation and testing of the model code been performed and documented? 
The commenter states that these guiding questions ensure that the Agency does not have to fully reanalyze every study and can instead focus on the weight of the evidence approach. The commenter argues that including data availability in the weighting injects an arbitrary parameter, unrelated to study quality and reliability.
Response: The rule complements the activities that EPA currently conducts by giving greater consideration to studies for which the underlying dose-response data are available for independent validation. For studies in which the dose-response data are available for independent validation, this will increase the transparency of EPA decision making in significant regulatory actions and independent scientific information.
Comment: Commenter (12715) states that independent validation and reanalysis can often be conducted with more processed form of data than the stringent stage of data EPA is requesting. Instead, the commenter suggests the EPA justify the rule's requirements in specific contexts and for specific types of data and models to avoid eliminating studies where access to less processed data would jeopardize privacy or lead to logistical challenges for data sharing.
Response: EPA has defined "data" in part to mean "the set of recorded factual material commonly accepted in the scientific community as necessary to validate research findings." Thus, the stage of data that is available for independent validation would be that stage of data necessary to accomplish the independent validation. The rule does not include a categorical exclusion of studies where the underlying data cannot be made available for independent validation.
3.2.3.4 Independent Validation Does Not Imply or Improve Validity or Accuracy
Comment: Several commenters contend that independent validation does not imply or improve validity or accuracy and therefore there is no justification for this rule.
Commenter (11176) states that independent validation as EPA defines the term in the supplement is a rather narrow method of evaluating research and assessing a scientific conclusion that could ultimately hamstring the work of EPA scientists to determine the best science to inform EPA work. The commenter suggests that the focus on independent validation puts undeserved emphasis on an individual study and that such a focus is contrary to the science community's practice of generally not being interested in a specific study but rather on its underlying hypotheses. The commenter states that no hypothesis can be conclusively addressed by an individual study; instead, a hypothesis' validity should be assessed on a body of evidence.
Response: While EPA agrees that the body of evidence is important, this rule does not conflict with the importance of transparency of the dose-response data that EPA uses in significant regulatory actions and influential scientific information. The rule does not require any entity, including EPA, to conduct independent validation of studies before EPA can use those studies in significant regulatory actions and influential scientific information.
Comment: Several commenters (11176, 11496, 11399, 12727) contend that independent validation does not imply or improve validity or accuracy and therefore there is no justification for this rule.
Commenter (11176) states that the proposed rule also seems to assume that a study that can be independently validated is of higher quality. The commenter argues that there is no evidence to support such an assertion. The commenter notes that Leek and Peng discussed in further depth the lack of a connection between research that has been reproduced and quality in a 2015 article in Proceedings of the NAS. The commenter states that EPA also acknowledged this point in the document, Plan to Increase Access to Results of EPA -Funded Scientific Research where EPA states "Whether research data are fully available to the public or available to researchers through other means does not affect the validity of the scientific conclusions from peer-reviewed research publications." Commenter (11186) states that independent validation as described is not a sound method for assessing accuracy. The commenter notes that changes in statistical methodologies for the same sample can result in a different research question answered and this practice will undermine an established practice that scientists already use: the peer-review process.
In addition, commenter (11496) states that the need for independent validation is not justified. The commenter states that the most important of its concerns is EPA's stated desire to exclude any pivotal regulatory science which cannot be "independently validated" beyond the existing and well-established peer review process. The commenter asserts that this is an inherently flawed perspective on how science, peer review and weight of evidence are used to protect public health. The commenter states that if EPA identifies peer-reviewed pivotal science that is insufficient, EPA or partner agencies like the Agency for Toxic Substances and Disease Registry or National Institute of Health should fund follow-up confirmation studies by independent researchers, not seek to disclose personal information to third parties tasked with independently reanalyzing existing datasets. The commenter requests that EPA articulate why independent validation of peer-reviewed science is even necessary for regulatory decisions; this type of redundant statistical analysis has never been a requirement of the OMB or any other federal Agency. Commenter (12727) states that the definition of "independent validation" suffers from EPA inappropriately conflating "validation" with "reanalysis." The commenter states that validation of scientific data is not accomplished by simply independently reanalyzing the data. The commenter notes that false or inappropriately obtained data will be invalid regardless of whether it can be independently reanalyzed. The commenter suggests that reproducing study results using a different population or method is generally considered a stronger approach to validation, rather than simply reanalyzing the results using the same data, as it shows that the results hold across different populations and different study designs.
Commenter (11399) states that additional "independent validation" that goes above and beyond the peer review process and EPA existing science advisory infrastructure is not necessary to assure scientific validity and accuracy. The commenter argues that this new procedural hurdle limits the universe of scientific studies that EPA may consider. The commenter asserts that creating such an impediment undermines EPA's legal obligation to use the best-available science to guide its decision-making processes and portends a weakening of the scientific underpinnings of health-based environmental regulations. Commenter (12427) states that scientific rigor depends on elements of scientific inquiry and judgments regarding the validity, value, and utility of science. The commenter states that the elements of scientific inquiry include data, methods, logic connecting data, and methods to conclusions. The commenter states that to judge validity, value, and utility of science, it is necessary to know how one got the answer. The commenter states that adequacy for "conclusion-making" occurs when scientific evidence supports a sufficient degree of certainty that rules out alternative explanations and that "Transparency" is not a determinant of scientific rigor. The commenter argues that judgments regarding validity, value, and utility of science can be and are made via peer review not through the approach taken in this rulemaking.
Response: Although the EPA agrees with commenters that meaningful insights can be obtained through similar studies in different media or context, the EPA continues to find that independent validation of the study findings and conclusions driving the EPA's dose-response assessments would provide important information. The benefits of greater data transparency and the significance of reanalyzing and validating study results are well-documented in scientific literature. McNutt (2014) noted, "reproducibility, rigor, transparency, and independent verification are cornerstones of the scientific method." The National Academies of Sciences, Engineering, and Medicine (NAS) workshop on Reproducibility and Replicability in Science also noted that "certainly, reproducibility and replicability play an important role in achieving rigor and transparency." Munafò et al. (2017) state, "the credibility of scientific claims is rooted in the evidence supporting them, which includes the methodology applied, the data acquired, and the process of methodology implementation, data analysis and outcome interpretation. Claims become credible by the community reviewing, critiquing, extending and reproducing the supporting evidence. However, without transparency, claims only achieve credibility based on trust in the confidence or authority of the originator. Transparency is superior to trust." The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety.
This rule is designed to build upon OMB M-13-13. It is also consistent with OMB M-19-15, which highlights the need to characterize the sensitivity of an agency's conclusions to analytic assumptions, as well as other federal guidance documents that require greater data transparency. The EPA's attention to data transparency is also responsive to the broader interest in greater data and model transparency observed in the numerous transparency initiatives in the scientific community and federal government, as well as the criticism the EPA has received from members of the public, scientific community, and Congress on the transparency of the scientific basis for EPA's decisions in previous influential scientific information assessments and regulatory actions.
The scientific community has also embraced greater data transparency, as evidenced by data sharing and availability requirements for many high-impact journals (Ref. 15) and the emergence of organizations, such as the Center for Open Science, and international initiatives like Findable, Accessible, Interoperable, and Reusable (FAIR) data principles; Facilitate Open Science Training in European Research (FOSTER); and Guidelines for Transparency and Openness Promotion (TOP) in Journal Policies and Practices that incentivize greater transparency in research.

.

 Chapter 4: Data and Models
Section 30.2 of the notice of proposed rulemaking (NPRM) defines dose response data and models as follows: 
      Dose response data and models means the data and models used to characterize the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and the magnitude of a predicted health or environmental impact. Such functions typically underlie pivotal regulatory science that drives the size of benefit- cost calculations, the level of a standard, and/or the points of departure from which reference values (reference doses or reference concentrations) are calculated.
Section 30.5 includes the following requirement:
      When promulgating significant regulatory actions, the Agency shall ensure that dose response data and models underlying pivotal regulatory science are publicly available in a manner sufficient for independent validation. Where the Agency is making data or models publicly available, it shall do so in a fashion that is consistent with law, protects privacy, confidentiality, confidential business information (CBI), and is sensitive to national and homeland security.
In addition, the NPRM (§30.6) states that, when available, EPA shall consider high quality studies that explore:
 Appropriateness of a linear, no-threshold dose response;
 A broad class of parametric dose-response or concentration-response models; 
 A robust set of potential confounding variables; 
 Nonparametric models that incorporate fewer assumptions; 
 Threshold models across the dose or exposure range; and 
 Models that investigate factors that might account for spatial heterogeneity.
The NPRM (§30.6) also requires EPA to describe and document any assumptions and methods used and should describe variability and uncertainty. 
In the SNPRM, the EPA proposes revising the NPRM regulatory text at 40 CFR 30.3, 30.5, 30.6, and 30.9 so that the provisions would apply to data, not only dose-response data. The SNPRM also proposes to delete the definition of "research data" and to add a definition for "data" at 40 CFR 30.2 as follows:
      Data means the set of recorded factual material commonly accepted in the scientific community as necessary to validate research findings in which obvious errors, such as keystroke or coding errors, have been removed and that is capable of being analyzed by either the original researcher or an independent party.
The SNPRM proposes a broader applicability with this definition as transparency of EPA's science should apply to other types of data that drive the requirements and/or quantitative analysis of EPA final significant regulator decisions and influential scientific information. The SNPRM specifically requests comment on whether the definition of "data" should apply to another stage of data and on whether there is another definition of "data" that should be considered.
Regarding the NPRM, several comments were received concerning dose response data and model proposal requirements, including the requirements of §30.5 and §30.6. Additionally, several comments were received regarding data and the use of data within the SNPRM. In this Chapter, these comments are organized under sections 4.1 NPRM Data and Models  -  Related Comments and 4.2 SNPRM Models, Variability/Uncertainty, and Assumptions/Defaults.
Section 4.1: NPRM Data and Models  -  Related Comments
 Data and Models; Dose response data and models (§30.2), requirements to the use of dose response data and models underlying pivotal regulatory science (§30.5), and additional requirements (§30.6)
 Definitions of Research Data (§30.2) and Data
Section 4.2: SNPRM Models; Variability/Uncertainty, and Assumptions/Defaults
          Modeling Definitions
          Models (General) and Best Science Practices
          Public Availability of Models/Data
          Weighting of Models
          Scope of Models Included
          Assumptions/Defaults-Variability/Uncertainty
          Impacts on Older/Existing Models
          Linear vs Nonlinear Modeling
          Impacts on Specific Models
Section 4.3 SNPRM Data, Variability/Uncertainty, and Assumptions/Defaults
          Clarity of Proposed Definitions
          Application of the Rule in Respect to Data
          Linear vs. Nonlinear Models
          Models (General) and Best Science Practices
          Assumptions/Defaults-Variability/Uncertainty
The specific comments provided in this Chapter reflect the comment letters that provide arguments supported by detailed rationale, examples, and references/documentation. The Docket ID numbers for these commenters are cited throughout this Chapter. In addition, some general comment letters may also include comments related to comments included and responded to in this Chapter. These comments are summarized in Appendix B. Note that points and issues raised in these general comment letters may also be raised by the more detailed comments summarized in this Chapter. The responses provided in this Chapter adequately address these more general comments.
4.1 NPRM Data and Models  -  Related Comments
4.1.1 Data and Models; Dose response data and models (§30.2), requirements to the use of dose response data and models underlying pivotal regulatory science (§30.5), and additional requirements (§30.6)
4.1.1.1 Dose Response Data
4.1.1.1.1 Availability of Underlying Data
Support the Proposed Rule Requirements that Data Be Made Publicly Available 
Comment: Commenter (6369) asserts that data access is crucial to reproducibility when conducting independent validation of EPA results as outlined in the proposed rule. The commenter adds that public access to information that the Agency deems critical to future or pending regulations is vitally important to the scientific process. The commenter also asserts that timely access to that information is just as important and provides examples to demonstrate how the lack of availability of underlying science used in decision-making can have significant impacts on EPA's chemical assessment conclusions.
For example, commenter (6369) states that they engaged directly with EPA's Integrated Risk Information System (IRIS) program to better understand the scientific information being relied upon to draw conclusions regarding formaldehyde carcinogenicity and had considerable difficulty in not only communicating with the IRIS scientists on the data that supports the program's conclusions regarding the carcinogenicity of formaldehyde and accessing the underlying data from studies that the IRIS program used in its draft chemical assessment. The commenter provides a chronology detailing their efforts and notes that, overall, it took six years to obtain all the necessary underlying data for evaluation/validation of the results from this study. The commenter provides that an evaluation of the study results determined that the changes reported by the study authors to be associated with formaldehyde exposure were not exposure dependent. The commenter contends that this highlights the critical need for public availability of results from studies that are key not only in dose-response modeling, but also in drawing conclusions regarding the potential health effects associated with exposure to formaldehyde. The commenter adds that data obtained via a Technology Transfer Agreement with NCI in 2014 to obtain information related to the Bean-Freeman (2009) study that was relied upon by the IRIS program to conduct dose response modeling to determine the potential for carcinogenic effects following formaldehyde exposure took several years before they were able to obtain the data. The commenter states that it is important to note that, in this case, the scientists gained access to underlying data related to individual subjects in a cohort. According to the commenter, this was accomplished while still allowing the protection of personal identifiers and other confidential information. Thus, the commenter concludes that, while concerns over confidential information are valid, they can be addressed while allowing transparency of critical scientific data. 
Response: Categorically excluding studies because the underlying dose-response data are not publicly available would exclude valid studies. Thus, rather than categorically excluding studies, EPA will give greater consideration to studies where the underlying dose-response data and models are available for independent validation.
 Types of Studies and Examples for which Underlying Data May Not Be Available/Accessible
Other Federal Agency Data
Comment: Commenter (6373) suggests that the EPA state whether it will decline to consider scientific studies that are based on data collected by federal agencies that require privacy protection. The commenter notes that EPA research and studies that inform EPA regulatory activities often rely on data collected by other federal agencies, including but not limited to Medicare and Medicaid, the National Institute of Health (NIH), the Census Bureau, and the Social Security Administration. The commenter asserts that the Rule will not bind these agencies' data collection and research policies and practices.
Response: EPA will consider these studies if it determines they are pivotal science but will give them greater consideration if the underlying data and models are available for independent validation.
Old Studies - Original Data Does Not Exist or is No Longer Accessible
Comment: Commenters (1945, 4596, 5176, 6125, 6362, 6873, 6898, 6914, 8317) provide that, for older studies, authors or data and information sources may no longer exist, or may not be accessible.
Commenter (4596) asserts that EPA regulations that reduce exposure to pollutants and chemicals that are toxic to the developing brain have for decades relied on studies that have included data and information that was not made fully public, including longitudinal epidemiologic studies, human clinical studies, and animal toxicologic studies. In many cases, the commenter provides that the original raw data does not exist or is no longer accessible. 
Commenter (6914) notes that epidemiological evidence and other accounts related to work-related illnesses and toxic releases often come from older, yet meaningful studies and sources, where the authors or data sources may not be accessible. 
Commenters (1945, 5176) contend that there are legitimate reasons why data underlying a study's findings are not readily publicly available but should not be excluded, including reasons related to the technology available at the time of data collection, outdated formats in which data sets from older studies are stored. Commenter (6898) states that the Proposed Rule could allow the elimination of historical data sets that are not stored in currently used electronic formats. The commenter adds that the availability of historical public data is essential to scientists' understanding of the health impacts of environmental exposures over time. 
Commenter (6125) states that, because many of the foundational lead studies analyzed children with higher exposure and blood lead levels than are commonly seen today, it would be unethical to run the same tests today, exposing children to levels of lead that we know are unhealthy. The commenter provides that, due to numerous factors related to their age, it may not be possible to recover the original data. The commenter adds that, if use of those older studies were successfully challenged because they do not meet the data requirement policy, they could not be used. The commenter contends that their exclusion would weaken the scientific support for EPA regulations that have clearly proved their worth in the dramatic declines in blood lead levels in American children.
Response: This rule does not categorically exclude any study. The rule requires that EPA give greater consideration to pivotal science where the underlying dose-response data are publicly available in a manner sufficient for independent validation. Thus, these studies will still be considered.
Confidentiality Requirements Legally Mandated by Institutional Review Boards (IRBs) and the Need for Reconsent of Privacy Agreements
Comment: Commenters (0017, 1999, 2283, 2909, 4591, 4595, 4596, 4848, 5013, 5015, 5115, 5165, 6111, 6351, 6897, 6905, 6919, 6922, 8277, 8281-PH26) state that, as articulated by numerous science and health organizations, many health studies, especially those that uncover the effects of exposure to toxins and pollutants/epidemiological studies, rely on medical records/location information that are sensitive covered information, and are confidential and private based on federal law and other patient privacy policies (i.e., including confidentiality agreements). Commenters (0106, 2259, 4591, 4596, 5015, 5180, 6873, 6905, 6919) provide that, as with all data (historic or otherwise), there may be circumstances where an individual participant's data cannot be made public because of confidentiality requirements legally mandated by IRBs and Health Insurance Portability and Accountability Act of 1996 (HIPAA). Commenter (6897) provides that, epidemiologic data often involves personal identifiers that can make full disclosure challenging  -  particularly retroactively, if participants have already agreed to be involved under more restrictive disclosure terms.
Commenter (6873) states that IRBs cannot release private data without consent, and getting this consent from study participants where the researcher has long since retired and participants have either died or moved multiple times would be extremely challenging and costly at best, and effectively impossible at worst, therefore possibly rendering "old" studies unusable.
Commenter (6919) provides that current federal and state law both require personally identifiable data to be de-identified due to the HIPAA and Colorado regulations protecting the confidentiality of Vital Statistics birth and death certificate data. The commenter provides that government agencies have a legal responsibility to protect residents from any risks associated with sharing personal health information  -  this includes discrimination. According to the commenter, researchers can only conduct high-quality and reproducible research by protecting the personal health information of study participants. Likewise, public health professionals must retain private personal data in their own study records in order to conduct accurate and useful surveillance to inform local, state, and national policies. 
Commenter (5015) states that access to private medical information is essential to conducting high quality and reproducible air quality and health research: 
 There are of course longstanding federal rules for protecting the privacy of individual medical information of the subjects of studies (HIPAA, Common Rule, etc.)
 Gaining access to data from older studies may be difficult, given the privacy commitments that were made to study subjects in the past.
 However, there are today several means to make such data available to investigators with appropriate privacy protections (e.g. Medicare, Federal Research Data Centers) and many investigators have been taking advantage of these.
 Although it is possible, as some have suggested, to create a "depersonalized" data set by stripping all personal identifiers, such as address, date of birth, etc., it is not possible to conduct a high-quality air pollution and health study without knowing the locations of those being studied, i.e. where they live, and what are the sources and levels of their air pollution exposure?
Commenter (8317) asserts that there is already a lack of scientific information about the occupational and environmental health risks farmworkers face. The commenter notes that because farmworkers used in occupational and environmental research are often migratory, moving for work across domestic and international borders, researchers may be unable to locate farmworkers they last encountered as study participants years ago, and thus unable to re-negotiate privacy agreements struck at the time the research was conducted. The commenter states that the EPA should base its regulatory decisions on the credibility of scientific evidence, not on arbitrary factors like the public availability of research data.
Response: This rule does not categorically exclude any study. The rule requires that EPA give greater consideration to pivotal science where the underlying dose-response data are publicly available in a manner sufficient for independent validation. Thus, these studies will still be considered.
Comment: Commenter (6125) expresses concern that lawsuits filed based on this proposal might force EPA to roll back current drinking water standards and drinking water health advisories for pollutants such as arsenic and nitrate. The commenter also expresses concern that EPA has not assessed how the requirements in the new policy might affect drinking water standards and advisories. The commenter states that both drinking water standards rely on epidemiological data, including confidential patient information, as well as on older studies. The commenter adds that data from such studies may have been discarded or lost or be in an unreadable form. The commenter adds that none of these studies could realistically or ethically be reproduced since they derive from a unique cohort, and it would be unethical to expose people to ingested pollutants.
Response: If a study is identified as pivotal science, EPA will still consider the study but may give it lesser consideration if the dose-response data are not available for independent validation.
Data Reduction Time Limits
Comment: Commenter (6362) states that, invariably there will be circumstances where underlying data no longer exist for studies and/or models that are high quality and reliable. For example, the commenter provides that most organizations have data retention policies that have resulted in the disposal of underlying data. Furthermore, according to the commenter (6362), Good Laboratory Practices (GLP) regulations include defined periods of time to retain data and study records. The commenter requests that EPA address how it will continue to use those studies and models considering these policies.
Response: If a study is identified as pivotal science, EPA will still consider the study but may give it lesser consideration if the dose-response data are not available for independent validation.
Proprietary Data
Comment: Commenter (4595) provides that some studies may rely on public and private-sector funding sources that limit access to underlying data for proprietary reasons. 
Response: If a study is identified as pivotal science, EPA will still consider the study but may give it lesser consideration if the dose-response data are not available for independent validation.
Studies are Often the Property of Scientific Investigators
Comment: Commenter (6922) notes that the Courts have recognized that raw data underlying studies are often the property of scientific investigators and are often not available because of reasonable confidentiality arrangements with study participants. 
Response: If a study is identified as pivotal science, EPA will still consider the study but may give it lesser consideration if the dose-response data are not available for independent validation.
Non-U.S. Studies
Comment: Commenter (1973) asserts that if the rule is finalized as proposed, all non-U.S. studies would be legally unable to meet EPA's requirements. The commenter provides that, for example, Canadian Community Health Study data would remain safe behind protected servers at Statistics Canada. Additionally, the commenter states that while the protocols, methods, and all other information necessary to judge the study are available, EPA would nevertheless ignore it. The commenter states that this would be the same for the ESCAPE study, the UK national cohort study, the Netherlands National Cohort study, the Danish National Birth Cohort and the Norwegian Birth Cohort (both funded by NIH), etc. The commenter adds that U.S. cohorts, given a choice between violating their promises in the informed consent documents they signed or their Data Use Agreements, or not making their data public, will choose the latter. The commenter contends that the EPA will achieve no greater "transparency", they will only deny themselves the ability to set standards based on the studies that the rest of the world relies on. The commenter provides that the EPA's own Science Advisory Board (SAB) has commented that this proposed rule" could have the effect of removing legal, ethical, and peer-reviewed studies of health effects as sources to support the agency's regulatory efforts".
Commenter (1657) states that if the proposed rule applies to data on human subjects, it could prevent the EPA from using any human subjects data collected from European Union (EU) member countries because after May 25, 2018 such data will be subject to the EU general data protection regulation (GDPR), which enshrines a "right to be forgotten." According to the commenter, while the results of analyses using EU human subjects data can be archived permanently, after that date the data itself could likely not legally be permanently and immutably archived. 
Response: If a study is identified as pivotal science, EPA will still consider the study but may give it lesser consideration if the dose-response data are not available for independent validation.
Comment: Commenter (4593) provides that the U.S. is obliged to accept data generated in accordance with the Organisation for Economic Cooperation and Development (OECD) Test Guidelines and Good Laboratory Practice (GLP) Standards. The commenter notes that the U.S. and other OECD member countries realized that differences in testing requirements among countries meant that companies would in some cases have to retest a chemical in order to market it in other areas. According to the commenter, this was needlessly costly and resulted in a delay in obtaining information needed for regulatory assessment. As a result, the commenters states that OECD member nations agreed to accept for regulatory purposes data generated in accordance with the OECD test guidelines, GLPs and quality assurance program. The commenter adds that GLPs specify the information that must be reported; and underlying data is not one of them. Therefore, the commenter is concerned that the proposed policy which requires underlying data to be made available would run counter to the obligations under the Mutual Acceptance of Data treaty.
Response: If the study is identified as pivotal science, EPA will still consider the study but may give it lesser consideration if the dose-response data are not available for independent validation.
Specific Examples of Studies and Supporting Data Sets Would Be Eliminated from Consideration
Comment: Commenters (6378, 8275) provide two examples of studies that may not be able to be used under the proposed requirements. They are papers from the University of Washington that involved studies that ensure confidentiality of the participants (Miller KA et al, Long-term exposure to fine particulate matter air pollution and cardiovascular events in women. New England Journal of Medicine 2007; 356:447-58; and Kaufman JD et al, Association between air pollution and coronary artery calcification within six metropolitan areas in the USA [The Multi-Ethnic Study of Atherosclerosis and Air Pollution]: a longitudinal cohort study. The Lancet 2016; 388:696-704). Commenter (8275) notes that both studies contain dose response data and models as anticipated in the proposed rule, and for which the proposed rule creates an expectation that data required to replicate the analysis be made available. In each case, the commenter states that the underlying studies are conducted with strict rules under the auspices of the NIH (National Heart, Lung, and Blood Institute), that preclude any potential identification of subject identity or confidential information. The commenter provides that the NIH and the approving IRBs would not permit release of the data in a way that would adhere to the letter of the rule. According to the commenter, it is possible that limited datasets could be created for replication analysis that would protect participant identity and confidential information, but funds are not available to create these datasets, and the rule is not clear that such a limited dataset would be acceptable. The commenter states that, as a result, the rule would lead to arbitrary exclusion of pertinent scientific evidence.
Commenter (4591) notes that data from the landmark "Six Cities" study that established a link between air pollution and human mortality, could not be made publicly available for several reasons, including promises that were made to the study participants at time of enrollment. Notably, the commenter provides that the outcomes of this study have been validated in several scientifically rigorous ways and by several independent studies. Commenter (8281-PH38) states that when the published landmark Harvard Six Cities Study was challenged, the EPA took a logical step and referred them to an independent third-party, the Health Effects Institute, for a deep dive review. The commenter reports that the autonomous reviewers examined the data and developed a report that confirmed the original findings. The commenter notes that there has been other research since then that has confirmed similar findings, including some studies that use publicly available data sets. 
Response: If a study is identified as pivotal science, EPA will still consider the study but may give it lesser consideration if the dose-response data are not available for independent validation.
Comment: Commenters provide and mention specific studies (not including the national ambient air quality standards (NAAQS) Supporting Studies) that would likely be impacted if the Proposed Rule were to apply.
Commenter (6170) provided a list of 50 example science references to some of the publications from their research (out of 265) that would not qualify for use in EPA rule making due to privacy concerns for human subjects if the Proposal is finalized. 
Commenter (5012) asserts that the Proposal would eliminate consideration of many major studies on the health impacts of environmental pollutants and cites the Children's Health Study conducted in southern California as an example of a study that would probably be eliminated from consideration under the Proposed Rule.
Commenter (6919) notes that Colorado is one of the top-producing states for oil and gas. The commenter provides that, as such, we are observing and recording exposures and possible health effects from oil and gas development that has increased dramatically near residential areas. According to the commenter, under this proposed rule, it is possible that studies derived from data on individual exposures could be suppressed since these studies do or may involve taking water and air quality measurements at people's homes, the use of personal air monitors, possibly even blood and urine samples, and portions of the data collected would need to be redacted to protect the study participants. The commenter asserts that it would be a disservice to public health if valuable information from these types of studies were not considered in rulemaking actions because personally identifiable health data could not be protected under the proposed transparency requirements. The commenter provided one example of a study by the Colorado School of Public Health that evaluated participants ranging in age from 0-24 years to study the rates of childhood hematologic cancer and residential proximity to oil and gas development. The commenter contends that, because oil and gas production is proliferating in Colorado, studies like these are critical for understanding the industry's impacts on public health, including the detrimental effects of elevated ozone levels and associated air toxic emissions along the Colorado Front Range. 
Commenter (6111) provides a 9-page partial list of studies that may contain protected health information that have been relied on by the EPA and cited in EPA documents. The commenter adds that there are many EPA rulemakings and decisions that have relied on studies that cannot be replicated and whose data likely could not be made publicly available. For example:
 PCBs: EPA's regulations establishing water quality standards for polychlorinated biphenyls ("PCBs") under the Clean Water Act (CWA) were based in part on long-term epidemiological studies of cancer rates in workers exposed to PCBs.
 Radionuclides: EPA's SDWA regulation for radionuclides relied on epidemiological studies of survivors from the Hiroshima and Nagasaki atomic bomb attacks.
 Particulate matter: EPA's 1997, 2006, and 2012 NAAQS for fine particulate matter all relied on studies using confidential data, such as the Six Cities Study.
 Methylmercury: EPA's reference dose for methylmercury in fish that will be consumed by humans relied on data from human exposures in the Faroe Islands. 
The commenter contends that precluding reliance on these and other studies for the sole reason that the underlying raw data has not been or cannot be released to the public is arbitrary, capricious, contrary to professional best practices, and antithetical to protection of public health and safety as required by the statutes. 
Commenter (6113) states that scientific evidence concerning the damage caused by pesticides was not generated by studies that were verifiable and public in the sense contemplated by EPA's proposal. (See generally Congressional Research Service, Pesticide Law: A Summary of the Statutes, Nov. 14, 2012 (overview of pesticide law). Little, if any of such evidence was generated by studies that were verifiable and public in the sense contemplated by EPA's proposed Anti-Environment Regulation.) According to the commenter, while much of the data included private medical data protected by privacy laws such as HIPAA, much of the data was private and or difficult to reproduce for other reasons (for example, endangered wildlife surveys may not be able to be repeated without impermissibly harming the wildlife).
Response: If a study is identified as pivotal science, EPA will still consider the study but may give it lesser consideration if the dose-response data are not available for independent validation.
Comment: Commenter (6916) notes there was no analysis of the number of studies that could be affected by the Proposal. According to the commenter, were this rule to be finalized as proposed, it would permit or even require the Agency to disregard certain studies placed in the record in future substantive rulemakings under the Clean Air Act (CAA), among other statutes
Response: The rule as finalized does not categorically exclude any study based on availability of data. If a study is identified as pivotal science, EPA will still consider the study but may give it lesser consideration if the dose-response data are not available for independent validation.
Comment: Commenters (6153, 6874, 8272) state that a recent example of important non-public data on a neurotoxicant is research on the pesticide chlorpyrifos. The commenters provide that researchers followed a cohort of children exposed to this pesticide before the current ban on residential use and found lower IQ and working memory to be associated with higher levels of prenatal chlorpyrifos exposure. The commenters state that, in a rulemaking process regarding agricultural use of chlorpyrifos, EPA requested the underlying data from the Columbia Center for Children's Environmental Health (CCEH). The commenters state that the response from CCEH explained that because of the detailed sociodemographic and health-related elements their data set contains, they did not believe they could submit extensive individual-level data to EPA in a way that would ensure participants' confidentiality. The commenters state that such concerns are not uncommon with the kinds of longitudinal data sets that allow identification of long-term consequences of environmental exposures. 
Response: If a study is identified as pivotal science, EPA will still consider the study but may give it lesser consideration if the dose-response data are not available for independent validation.
Comment: Commenters (6144, 6174) state that, as EPA continues its review of the insecticide chlorpyrifos, this policy might restrict its ability to consider several critical epidemiological studies conducted by Columbia University that showed that in utero exposure to the chemical was linked to negative neurodevelopmental impacts. The commenters note that the health data was collected along with confidentiality pledges to the mothers. The commenters state that cutting out the use of this research would mean that EPA would have little information on the direct human health impacts of chlorpyrifos exposure and could result in a policy decision that is less protective of public health than it otherwise would have been.
Response: If a study is identified as pivotal science, EPA will still consider the study but may give it lesser consideration if the dose-response data are not available for independent validation.
Comment: Commenters (1424, 1529, 4828, 4842, 5981, 6125, 6133, 6137, 6144, 6153, 6174, 6874, 6939, 8272) express concern that supporting studies for the lead rules would not be allowed as scientific evidence under the new transparency ruling since they include private medical records and/or could not be ethically replicated.
Response: If a study is identified as pivotal science, EPA will still consider the study but may give it lesser consideration if the dose-response data are not available for independent validation.
Comment: Commenter (6446) expresses concern that the proposed rule would likely prevent EPA from considering epidemiological studies of children exposed in utero to mercury through maternal consumption of mercury-contaminated fish or marine animals because individual health data could not be made public. The commenter states that EPA used these epidemiological studies in its 2001 report, Water Criterion for the Protection of Human Health: Methylmercury, which served as a basis for the California State Water Board's recently adopted mercury water quality objectives.
Response: If a study is identified as pivotal science, EPA will still consider the study but may give it lesser consideration if the dose-response data are not available for independent validation.
Would Limit New Studies
Comment: Commenter (6367) states that their members recognize that subjects in clinical toxicology studies often divulge highly sensitive information regarding the behaviors, occupations, avocational activities, and physical and neurocognitive function of themselves and their communities with the expectation that this information be maintained in a confidential manner. The commenter is concerned that subjects' voluntary participation in clinical toxicology studies, and the willingness of medical toxicologists to undertake such studies, will be constrained if such information will become publicly available if the study is used to advance environmental protection.
Response: EPA will not make personally identifiable information (PII) publicly available.
Invalidating Studies Based Solely on Public Availability of Underlying Raw Data
A Study Should Not be Invalidated Solely Based on the Underlying Data Being Unavailable
Comment: Commenters (0040, 0106, 0107, 0118, 0501, 1203, 1289, 1945, 2020, 2337, 2911, 4591, 4838, 5013, 6111, 6120, 6125, 6144, 6160, 6351, 6791, 6864, 6875, 6897, 6905, 6922, 8317, 9224) state that raw data may not be available for a variety of reasons, having to do with privacy, proprietary, and other issues, that require researchers to keep at least some data out of public view. Commenter (6196) states that, in principle, no one disputes the benefits of improving access to underlying data for research on chemicals and pollutants. The commenter provides that the goals of "open science" have received support from several organizations and leading scientific journals and research institutions have adopted practices and policies to maximize data access. Commenters (0040, 0107, 1011, 0118, 0501, 1203, 1289, 1420, 1945, 1973, 1999, 2020, 2259, 2283, 2337, 2909, 2911, 4591, 4595, 4838, 4848, 4884, 5012, 5013, 5019, 6109, 6111, 6113, 6119, 6120, 6121, 6133, 6144, 6145, 6156, 6159, 6160, 6164, 6195, 6196, 6351, 6355, 6367, 6373, 6791, 6875, 6896, 6898, 6899, 6905, 6906, 6908, 6918, 6919, 6920, 6922, 6928, 8281-PH16, 8281-PH26, 8281-PH38, 8317, 9224) assert, however, that a study should not be invalidated solely based on the underlying data and models being unavailable. The commenters express concern that, under the proposed rule, the available science and evidence that could be used to inform future regulations would be limited (and not reflect the body of the "best available science") by the requirement to make all datasets publicly available, limiting the science base for regulatory decision making and protection of public health. 
Commenter (6111) notes that releasing raw data will not improve the quality of the resulting report/study/analysis, and therefore will do nothing to render any individual study "better." The commenter provides that it has long been recognized by the professional public health, medical, and scientific research community -- and by EPA itself until now  -- whether the raw data underlying a study is released does not determine the quality of the study. 
Commenter (5180) notes that the Proposed Rule provides that when the EPA develops regulations, the "EPA should ensure that the data underlying those are publicly available in a manner sufficient for independent validation." The commenter asserts that this approach would allow the EPA to consider only those scientific studies where the underlying data is made publicly available. The commenter states that, under the Proposed Rule, the EPA would have had to reject studies simply because they involved confidential personal data protected under the HIPAA that had not been made public "in a manner sufficient for independent validation."
Commenter (4838) states that the requirement that significant regulatory decisions be justified only by studies based on dose response data and models that are available to the public limits EPA's ability to rely on otherwise peer-reviewed, scientifically-valid studies that do not, or cannot, make their data publicly available because of confidentiality concerns. For example, the commenter notes that the EPA used the landmark Harvard Six Cities Study, demonstrating a dramatic link between premature mortality and air pollution, as part of its justification for key clean air regulation. The commenter provides that the study has been rigorously, independently peer-reviewed but the subjects were promised confidentiality and the data is not public. According to the commenter, EPA should not provide blanket limits on the use of studies that cannot be made public because they contain confidential health or business information. The commenter states that scrubbing studies of such information may be impossible while keeping the study reproducible.
Commenter (0040) notes that, of the 17 publications she is a co-author for, only 5 would meet the proposed requirements that evidence come from publicly available data. The commenter states that the other 12 are from projects where the data are subject to data use agreements and cannot be made public due to confidentiality agreements. The commenter adds that the fact that the underlying data is not available to the public does not make these 12 publications any less trustworthy. The commenter states that data for every study do not need to be publicly-open for the results to be useful. 
Commenter (0106) states that studies must be accepted, if the peer review deems them worthy and "real" science, not just a study sponsored by a company or lobbying group with the conclusion written by that sponsor before the study begins. 
Commenter (6120) asserts that, while the Proposed Rule cites requirements of major scientific journals for the availability of data, the commenter provides that the editors of those journals have also issued a joint statement discussing the importance of including all relevant studies articulating circumstances where data cannot be fully shared, such as when it includes personal identifiers. Commenter (6896) notes, as editors of leading peer-reviewed journals stated, "[i]t does not strengthen policies based on scientific evidence to limit the scientific evidence that can inform them; rather, it is paramount that the full suite of relevant science vetted through peer review, which includes ever more rigorous features, inform the landscape of decision making. Excluding relevant studies simply because they do not meet rigid transparency standards will adversely affect decision-making processes." Commenter (9227) notes that numerous representatives of the scientific community -- including editors of the very scientific journals whose policies EPA cites to in the Proposal, the American Association for the Advancement of Science, members of the SAB, and other scientists cited to by EPA -- have already voiced serious concerns about the Proposal. According to the commenter, as these experts have recognized, it is not consistent with good scientific practice, and certainly not consistent with the Agency's responsibility to utilize "best available science," to deem certain scientific studies unworthy of consideration simply because these studies cannot meet an arbitrary public availability requirement. Commenter (6109) requests that EPA make use of the best available science, as it always has. The commenter provides that the best available science may include studies that do not have publicly available data and that the EPA should not arbitrarily dismiss these studies. Commenter (1011) states that EPA should avoid insisting on complete release of data if the security of the data would remain in doubt or if the failure to comply with the proposed transparency standard would prohibit EPA's use of the best available science. 
Commenter (8281-PH48) states that, if adopted the Proposed Rule would prevent EPA from considering the scientific studies that Rural Empowerment Association for Community Health (REACH) helps to conduct. The commenter notes that they cannot make all their data publicly available because we cannot risk compromising the confidentiality of the people who contribute to our work. According to the commenter, because they live in a rural community it would be relatively easy to identify study participants based on de-identified information like age, sex, occupation and number in households, even if the participants' names were redacted.
Commenter (6864) states that EPA is routinely in receipt of unpublished toxicity studies for chemicals designed for commerce. The commenter states that not all scientific findings are publishable, nor do scientific journals generally have enough space to include all data.
Response: Upon consideration of the comments, the EPA agrees that it is important not to categorically exclude any study because the data underlying a study at the stage required by this rulemaking may not be available for independent validation. Therefore, the EPA is not finalizing the proposal approach that would have categorically required that for studies to be considered pivotal science the underlying data would need to be available for independent validation. However, given that transparency is an important aspect of EPA's regulatory actions and assessments, it should be an important consideration in how the Agency considers pivotal science. As described in 40 CFR 30.5 of the final rule, the EPA will rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies --  based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements or quantitative analyses of an EPA significant regulatory action or influential scientific information will be identified as pivotal science.
Further, the EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted or tiered access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
No Reason to Prevent Use of Human Health Studies Because Underlying Health Data are Not Available to the Public
Comment: Commenters (0128, 2020, 4848, 6351, 6905, 6914, 6919, 6922) assert that there is no reason to prevent the use of health data simply because the underlying health data are not available.
Response: This rule does not categorically exclude any study because the underlying dose-response data are not available, either publicly or via restricted access, for independent validation.
Comment: Commenter (0128) provides that it is common practice for biomedical scientists to use data from randomized controlled trials published in the clinical oncology literature to inform clinical practice guidelines for the American Society of Clinical Oncology and to inform coverage decisions on emerging treatments and diagnostic tests by medical directors at Blue Cross and Blue Shield health plans. The commenter states that the confidentiality of individual patient data is ALWAYS strictly maintained. Nevertheless, the commenter notes that the aggregated outcomes data fully informed and optimally guided clinical and coverage decisions based on the best available scientific evidence and analysis. The commenter states that maintaining the confidentially of individual health records is not incompatible with subjecting scientific evidence to rigorous peer review.
Response: Nothing in this comment is incompatible with the requirements of this rule.
Comment: Commenter (4848) asserts that the EPA has presented no scientific reason to prevent use of human health studies simply because the underlying medical records are not available for public inspection and review. Commenter (6137) state that the Proposal only excludes health-based science (dose-response studies, epidemiological studies) where underlying data is not disclosed, generally because it cannot or should not be disclosed to protect individual participants' privacy and confidentiality. The commenter contends that the EPA cannot exclude whole categories of scientific data untethered from a specific context or study, but rather must assess each health study on a case-by-case basis to determine whether or not it should be considered in a particular rulemaking, under EPA's long-standing scientific guidelines and policies and its regular approach in CAA rulemakings. Instead, the commenter states that this Proposed Rule excludes health studies from consideration as a class, up front, before EPA is even in the rulemaking stage under its authority. And it does so not for scientific reasons, but, according to the commenter, simply due to industry preferences  -  most notably because industry cannot pick them apart by replicating decades of air pollution health effects, or by contacting individuals who shared private medical information to replicate the collection of data.
Response: This rule does not categorically exclude any study because the underlying dose-response data are not available, either publicly or via restricted access, for independent validation.
Comment: Commenter (2020) provides that a long-established approach to developing scientific knowledge is the peer-reviewed process that allows experts to propose logical arguments, test them with data, models and experiments, and determine the best explanations for the outcomes. The commenter notes that when human health is involved, the data required are often personal (names, addresses, illnesses, etc.) and are not needed to make aggregate assessments about the impacts of, for example, living nearer to a road where fine particulate (PM2.5) levels are high and rates of asthma or other health impacts. Thus, according to the commenter, to require that such data are public for EPA to use them is effectively the blocking of established approaches to doing solid science in a manner that also protects privacy. 
Commenter (6922) explains that, for decades EPA has relied upon epidemiological studies to help determine what pollutants or chemicals need to be regulated, and how rigorously to do so. EPA's reliance on those studies was reasonable  -  even though the underlying data was not public  -  because the studies underwent peer review to ensure these studies and their results were sound, replicable, and reliable. The commenter asserts that not having the raw data did not, and does not, make the science unsound. The commenter notes that the studies that include or rely on confidential health information often are among the best science that the Agency has to evaluate the effects of pollutant exposure on humans, and the costs imposed on individuals and society as a result of that exposure. 
Commenter (6922) notes that studies that include or rely on confidential health information often are among the best science that the Agency has to evaluate the effects of pollutant exposure on humans, and the costs imposed on individuals and society as a result of that exposure. Similarly, the commenter notes that EPA routinely relies on data provided by regulated industries that is treated as CBI and is therefore protected from public disclosure because it may include customer or other personally-identifiable information, including electric and gas consumption. According to the commenter, this is no different from relying on public health studies that rely on data that is protected from disclosure for various reasons. 
Commenter (6121) notes that much of the data showing the connection between chemicals and breast cancer risk comes from laboratory studies. However, epidemiological studies, and longitudinal studies, provide critical insights and important corroboration for these findings. The commenter adds that epidemiological studies also allow researchers to look at numerous factors in the lived experience of women to understand and parse out some of these complexities. According to the commenter, most of these critical epidemiological studies are funded by the federal NIH, which has a strong confidentiality policy intended to "protect the privacy of subjects by limiting the disclosure of identifiable, sensitive information." The commenter asserts that, forcing EPA scientists to disregard these critical studies because researchers have correctly protected the confidentiality of their participants' data will result in less scientifically sound conclusions and, most importantly, the public health benefits that the inclusion of these studies would provide.
Commenter (6145) asserts that, to have access to the best available science, studies must protect the privacy of those affected by environmental risks. According to the commenter, in its 2016 Plan to Increase Access to Results of EPA-Funded Scientific Research, the EPA recognizes the need to balance the interests of making information available to the public and protecting other interests, like privacy, as follows:
      While the Agency strives to increase access to its research results, it recognizes, consistent with the OSTP Memo, that Federal agencies have a responsibility to protect confidentiality and personal privacy, respect proprietary interests and property rights, and balance between the value of providing long-term access and its associated costs. It is important to recognize that some research data cannot be made fully available to the public but instead may need to be made available in more limited ways, e.g., establishing data use agreements with researchers that respect necessary protections. Whether research data are fully available to the public or available to researchers through other means does not affect the validity of the scientific conclusions from peer-reviewed research publications.
Commenter (6914) notes that under Toxic Substances Control Act (TSCA) and other laws, EPA relies on data and information that labor and public health organizations submit to inform rulemaking records. This data often is in the form of personal health data, exposure monitoring data, information on work practices and personal accounts; much of this data is not and would not be "publicly available in a manner sufficient for independent validation," as the proposal states. The commenter asserts that this is an unreasonable standard for research in general. 
Response: This rule does not categorically exclude any study because the underlying dose-response data are not available, either publicly or via restricted access, for independent validation. If a study is identified as pivotal science, EPA will still consider the study but may give it lesser consideration if the dose-response data are not available for independent validation.
No Reason to Prevent Use of Old Studies Because Underlying Data is not Available
Comment: Commenter (1481) asserts that the Proposal is over-broad in its restrictions and would ignore older studies while also setting unrealistic bars for entrance of new evidence. The commenter notes that open data requirements would be a new qualification for the field not in line with prior publication standards. The commenter provides that, as a result, many quality studies would be disregarded due to the old societal standards under which they were published. Furthermore, the commenter states that, until experts in the field decide this is a standard which is necessary to ensure quality research, it seems arbitrary and short-sighted for a non-expert government official to decide this is necessary.
Commenter (6111) notes that the proposal would prevent EPA from relying on studies conducted many years ago for which data are no longer available.
Response: If a study is identified as pivotal science, EPA will still consider the study but may give it lesser consideration if the dose-response data are not available for independent validation.
No Reason to Prevent Use if Underlying Proprietary or CBI Data is Unavailable to the Public
Comment: Commenter (6448) states that there are and will be situations in which EPA has a statutory requirement to make a determination for which scientific information will not be publicly available in a manner sufficient for independent validation. The commenter provides, for example, that during submission of new chemicals under Section 5 of TSCA, a manufacturer will submit proprietary information to the Agency whom will utilize this information in order to determine a risk finding; the outcome of which will be transparently disclosed but in a generic manner. The commenter adds that the registration of active ingredients under Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA) would follow a similar pathway as well. The commenter recommends that EPA ensure that any guidelines comply with applicable laws and encourage the use of all relevant scientific information, regardless if the underlying data can be publicly disclosed. According to the commenter, this is particularly important when utilizing data and information developed individually or via consortia under the requirements of FIFRA or TSCA.
Response: If a study is identified as pivotal science, EPA will still consider the study but may give it lesser consideration if the dose-response data are not available for independent validation. If there is a conflict between an environmental statute EPA administers and this rule, this rule will yield.
Scientific Mechanisms Intrinsic to the Scientific Process Sufficient to Ensure the Quality of a Study
Comment: Commenters (0040, 0732, 4838,4884, 5013, 6196, 6708, 6906, 6922, 9227) assert that public access to the underlying data is not necessary to ensure the validity of scientific information. Commenter (4591) notes that, in cases where it is not appropriate for data to be made publicly available, there are other mechanisms intrinsic to the scientific process for substantiating the relevance and validity of research results. 
Commenter (9227) asserts that the Agency's arbitrary, single-minded focus on considering studies for which certain data and models are publicly available (but only the dose-response studies relevant to health protective regulation, not the ones supporting registration of chemicals) stands in stark contrast to the way the scientific community validates research findings. The commenter provides that the scientific community, and scientific journals look to a range of attributes when assessing the quality of a scientific study, including whether the study has been peer reviewed, whether the scientists used rigorous scientific methods, and whether the study's results have been reproduced or replicated. According to the commenter, while scientific journals and other institutions have encouraged making data and models publicly available, there is widespread recognition in the scientific community that doing so is often legitimately constrained due to legal and ethical protections on the confidentiality and privacy of data, or because the data is unavailable. Moreover, the commenter states that no scientist or scientific organization supports the Proposal's approach of excluding research for which the underlying data cannot be disclosed. The commenter notes that none of the materials EPA cites support such an extreme approach. To the contrary, the commenter provides that the scientific community recognizes that the quality of a study is not determined by whether the underlying data is publicly available and has long utilized a variety of tools for ensuring the integrity and rigor of research findings.
Response: If a study is identified as pivotal science, EPA will still consider the study but may give it lesser consideration if the dose-response data are not available for independent validation.
Peer Review is Sufficient to Ensure Quality of a Study
Comment: Commenters (0040, 0732, 1203, 2020, 4595, 4838, 4884, 5013, 6108, 6196, 6708, 6875, 6922, 8281-PH16, 9224) suggest that existing peer-review processes are sufficient to ensure the quality of a study.
Commenter (4595) states that the Proposal sets an implied standard that peer-reviewed scientific research data that is not publicly available is not rigorous enough for use in decision making. Commenter (6196) contends that the unspoken premise of the Proposal is that unless EPA can guarantee full public access to a study's underlying data, the study must be deemed unreliable and should play no role in assessing a pollutant or chemical's effects on human health. The commenter states that this premise ignores the many ways in which the scientific community, regulators, and the public have traditionally determined the quality and relevance of study results.
Commenter (8281-PH16) notes that Congress intentionally directed EPA to consider peer reviewed research under the CAA, and mandates regular reviews of the science to ensure that EPA is reviewing and considering the most up to date science. 
Commenter (6922) provides that the EPA already utilizes a rigorous peer review process sufficient to ensure that Agency rulemakings are based only on sound, reliable studies, as described in EPA's Science and Technology Policy Council Peer Review Handbook (4th Edition). For instance, the commenter notes that the primary function of EPA's SAB is to review the quality and relevance of scientific and technical information being used by EPA or proposed as the basis for its regulations. The commenter notes that the Proposed Rule does not explain why the existing processes and tools for vetting scientific research, including epidemiological studies, are insufficient.
Commenter (4884) notes that peer reviewed studies as well as the necessarily cautious, deliberate process by which EPA has developed past regulations has ensured the sound basis of EPA rules. The commenter provides that valid health effect studies are often non-public because they rely upon confidential medical records. The commenter states that other studies useful in the past have relied upon confidential business sensitive industry data. According to the commenter, if the analysis of the raw data is available and correct, such studies must be used to achieve the legislative intent of these environmental laws. 
Commenter (2020) provides that most modern scientific journals require that data and models and methods required to reproduce the scientific findings are published when a peer-reviewed paper is published. The commenter states that this standard should be sufficient for EPA going forward. According to the commenter, the fact that past published papers were not held to this standard is something that current practices attempt to remedy, but this does not mean that decades of understanding should be ignored by EPA and EPA should apply best practices as does the scientific community. 
Commenter (0040) provides that publication study results need to go through the peer review publication and replication processes. As such, the commenter states the conclusions are based on strengths of evidences and are weighed according to their merit. The commenter adds that this methodology-based process leads to more robust research and stronger evidence. Commenter (0732) provides that scientists convene multi-scientist panels to extensively review literature from all known sources, evaluate each article for many criteria of well-done studies, and then factor it into their carefully assessed estimates that are used to create health standards. Commenter (4838) asserts that studies with confidential data can still be appropriately peer-reviewed and subject to rigorous scientific scrutiny over their methods and conclusions without making confidential health or business information publicly available and that EPA should not provide blanket limits on the use of studies where information cannot be made public. Commenter (4595) states that, when preparing studies with protected data for peer review and publishing, researchers summarize the analysis of raw data and draw conclusions based on that analysis and that these methodologies are not 'secret' but rather common practice.
Commenter (6196) provides that study reports typically explain the protocols used to gather data, the methods used for data analysis, the doses or exposure concentrations at which effects were and were not observed, the nature, severity, and incidence of such effects, and any unusual occurrences that may affect interpretation of the results. The commenter adds that this information plays an important role in the peer review process, informing the judgment of independent reviewers as to whether a study is worthy of publication in the scientific literature. According to the commenter, Agency reviewers likewise consider these indicators of reliability in deciding how much weight a study deserves in making judgments about hazard and risk. The commenter notes that in its narrow focus on a single criterion for study acceptability, the proposal departs from this comprehensive, multi-faceted approach for determining the "best available science" to inform decision-making. 
Commenter (1203) asserts that good science makes use of all verifiable data and if published in a peer-reviewed journal there is a paper trail that points back to the data, even if the raw data themselves are not available to every reader.
Commenter (5013) states that, in order to have reliable data and large sample sizes, researchers frequently study the records of patients treated in hospitals. Although all the underlying privacy data cannot be available for public review, reviewers wanting to reproduce a study in order to validate it can arrange to have confidential access to key data. Furthermore, the commenter contends that scientists can assess the merits of published research without seeing its data directly by considering such publicly released features as the study's research design, the methods used for data collection and analysis, and comparisons with previous results.
Commenter (6708) states that, if the EPA has an appropriate scientific peer-review process for its internal studies, studies should not be disqualified from consideration in rulemaking on the grounds that they could not release enough information to the public. The commenter adds that, in fact, the public's need for action on many issues regarding pollutants and the risk of exposure to dangerous substances should justify EPA rule-making based on some studies that require that some crucial information be kept private and thereby are rendered unrepeatable. The commenter contends that, instead of prioritizing this onerous, unnecessary, and potentially harmful attempt at transparency, the EPA should instead invest in having academic scientists available to its internal researchers and policymakers to review studies produced within and outside the agency to provide appropriate, timely, and independent feedback on findings regarding the health of the public.
Commenters (6875, 9224) states that, for data that is not produced by EPA, such as in scientific journals, and particularly when dealing with sensitive data, the process of peer reviewing should often be sufficient. According to the commenter, if multiple studies have gone through the peer review process and have come to the same conclusions, the Agency should not disregard these findings simply because the raw data is not available. 
Commenter (6108) asserts that, for policy decisions based upon peer-reviewed journals and literature, the articles should be available to the public (for a nominal fee, if necessary), but it is not necessary to provide every underlying data point for public inspection in rulemaking. According to the commenter, requiring every individual data point from such studies to be made available will greatly reduce the number of sound scientific articles available for rulemaking and the peer review process is widely accepted in the scientific community. The commenter provides that Journals that publish peer-reviewed articles have ethical guidelines for reviewers and reputable publishers use a robust peer-review process that has a series of quality controls.,
Response: This rule does not categorically exclude any study because the underlying dose-response data are not available, either publicly or via restricted access, for independent validation. If a study is identified as pivotal science, EPA will still consider the study but may give it lesser consideration if the dose-response data are not available for independent validation. Further, nothing in this rule precludes EPA from considering peer reviewed data. EPA will continue to consider peer reviewed data. Finally, if there is a conflict between an environmental statute EPA administers and this rule, this rule will yield.
Public Notice and Comment Process for Significant Rulemakings Sufficient
Comment: Commenter (6922) notes that the EPA follows a robust public notice and comment process for its significant rulemakings. According to the commenter, where the public has valid reason to doubt the assumptions or data underlying a particular study or is aware of a study that EPA has not considered, the public can share conflicting studies and other information with EPA during the public participation process of the rulemaking. The commenter adds that the EPA is obliged to exercise independent reasoned judgment to disregard information the Agency deems unreliable or biased. The commenter provides that this is the traditional, well-settled process of reasoned administrative rulemaking.
Response: EPA agrees with the commenter and will continue to critically evaluate studies as described in 40 CFR 30.5.
Inconsistency with EPA Policies on Data Access and Availability
Comment: Commenter (9227) states that the EPA has previously stated in several different forums that a scientific study can be valid even if the underlying dose response data and models are not publicly available. For example, the commenter notes that the EPA recently explained in its own Plan to Increase Access to Results of EPA-Funded Scientific Research that even though "some research data cannot be made fully available to the public but instead may need to be made available in more limited ways," the lack of full public availability "does not affect the validity of the scientific conclusions from peer-reviewed research publications." Under the plan, the commenter explains that the EPA must make publications resulting from EPA-funded research publicly accessible on NIH's PubMed Central (PMC). The commenter provides that the plan aims to "maximize access, by the general public and without charge, to digitally formatted data resulting from EPA funded research, while protecting confidentiality and personal privacy, recognizing proprietary interests, business confidential information and intellectual property rights, and preserving the balance between the relative benefits and costs of long-term preservation and access." The commenter adds that the plan recognizes important exceptions for when "the research data cannot be released due to one or more constraints, such as requirements to protect confidentiality, personal privacy, proprietary interest, or property rights." According to the commenter, it specifically declares: "The validity of scientific conclusions drawn from research publications or their associated research data, or EPA's ability to consider those conclusions and data in its actions, does not depend on compliance with this Plan."
Likewise, commenter (9227) notes that the EPA's Science Policy Council explains in A Summary of General Assessment Factors for Evaluating the Quality of Scientific and Technical Information that EPA's determination as to the quality and reliability of a particular scientific study does not depend on one single factor (e.g., the public availability of underlying data), but instead turns on the agency's consideration of five general factors. The commenter states that Congress implicitly endorsed this approach by including a directive for EPA to use these same five factors in evaluating science under the TSCA Amendments passed in 2016, and just last year this Administration included these same factors in a recent regulation implementing TSCA. The commenter adds that the factors comprise: (1) soundness; (2) applicability and utility; (3) clarity and completeness; (4) uncertainty and variability; and (5) evaluation and review. Of these, the commenter states that the only ones with any possible direct relevance to EPA's proposed approach are the third and fifth factors, but neither supports the elevation of public availability of data above all other considerations or the exclusion of studies with non-public data. The commenter adds that the third factor, "clarity and completeness" requires EPA to consider "[t]he degree of clarity and completeness with which the data, assumptions, methods, quality assurance, sponsoring organizations and analyses employed to generate the information are documented," and the fifth factor, "evaluation and review," requires EPA to consider "[t]he extent of independent verification, validation and peer review of the information or of the procedures, measures, methods or models." According to the commenter, even clear and complete "documentation" of the data used does not require that the data be made publicly available. Nor, according to the commenter, does factor five require either that a study's findings must have been replicated using the same data, or that the data must be available to allow for such replication. The commenter provides that, moreover, these are only portions of two of five key factors to consider.
Similarly, commenter (9227) asserts that the EPA's Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and Integrity of the Information Disseminated by the Environmental Protection Agency, ("EPA Information Quality Guidelines") issued pursuant to Section 515(a) of the Treasury and General Government Appropriations Act for Fiscal Year 2001 (Public Law 106-554; H.R. 5658) (the "Data Quality Act") make it clear that the public unavailability of underlying data or models does not render a study inappropriate for EPA's consideration. Specifically, the commenter notes that the EPA Information Quality Guidelines acknowledge that even with respect to science that will have "a clear and substantial impact on important public policies or private sector decisions," there will be circumstances where "access to data and methods cannot occur due to compelling interests such as privacy, trade secrets, intellectual property, and other confidentiality protections." Significantly, the commenter notes that the Guidelines do not instruct EPA to ignore such science. Rather, according to the commenter, the Guidelines instruct that if underlying data or methods are unavailable, "EPA should, to the extent practicable, apply especially rigorous robustness checks to analytic results and carefully document all checks that were undertaken." The commenter adds that the Guidelines further explain: "Original and supporting data may not be subject to the high and specific degree of transparency provided for analytic results; however, EPA should apply, to the extent practicable, relevant Agency policies and procedures to achieve reproducibility, given ethical, feasibility, and confidentiality constraints."
Commenter (9227) asserts that, far from instructing EPA not to consider scientific studies for which underlying data or models are unavailable, the EPA Information Quality Guidelines expressly acknowledge that EPA must balance a variety of important aims to fulfill its statutory obligations to protect public health and the environment. The commenter notes that the EPA explains in the guidelines that "most environmental statutes obligate EPA to act to prevent adverse environmental and human health impacts" and that "[f]or many of the risks that we must address, data are sparse and consensus about assumptions is rare." Thus, the commenter provides that rather than set rigid rules regarding what science and information EPA can rely upon in its rulemakings, EPA "seek[s] to strike a balance among fairness, accuracy, and efficient implementation." The commenter conveys that the EPA states: "Refusing to act until data quality improves can result in substantial harm to human health, safety, and the environment."
Commenter (9227) provides that even this Administration, in the context of promulgating regulations under TSCA, has adopted a regulatory definition of "best available science" expressly incorporating a multi-factor analysis, and that definition recognizes that public unavailability of data does not render a study incapable of being "best available science."
Response: This rule does not categorically exclude any study because the underlying dose-response data are not available, either publicly or via restricted access, for independent validation. If a study is identified as pivotal science, EPA will still consider the study but may give it lesser consideration if the dose-response data are not available for independent validation. Further, nothing in this rule precludes EPA from considering peer reviewed data. EPA will continue to consider peer reviewed data. Finally, if there is a conflict between an environmental statute EPA administers and this rule, this rule will yield.
As described in the preamble and provided in 40 CFR 30.5, EPA will consider a number of factors in determining the degree of consideration to give studies underlying pivotal science for which the dose-response data are not available for independent validation.
Strengthen Rule for Keeping Health, Personal and Proprietary Data Confidential
Comment: Commenter (6119) states that EPA should refine and strengthen the proposed rule for keeping health, personal, and proprietary data confidential to alleviate the apprehension of public access to personal or sensitive data while still ensuring that relevant epidemiological or toxicological studies can continue to be used to establish public health-based air quality and other standards. 
Response: EPA is not requiring that health, personal and proprietary dose-response data underlying studies considered to be pivotal science be made available in a manner that would violate confidentiality.
Court Cases Have Ruled that EPA Should be Allowed to Use Studies Where Data is Not Entirely Available
Comment: Commenter (0566) states that, in two cases, the court has ruled that EPA should be allowed to use studies where data is not entirely available. The commenter provides that, in American Trucking Associations, Inc. v. EPA, 283 F.3d 355, 372 (D.C. Cir. 2002), the court said, "[i]f EPA and other governmental agencies could not rely on published studies without conducting an independent analysis of the enormous volume of raw data underlying them, then much plainly relevant scientific information would become unavailable to EPA for use in setting standards to protect public health and the environment .... [S]uch data are often the property of scientific investigators and are often not readily available because of ... proprietary interests ... or because of [confidentiality] arrangements [with study participants].'" 
Commenter (0566) adds that, in 2010 the court reaffirmed its view that "raw data often is unavailable due to proprietary interests of a study's scientific investigators or confidentiality agreements with study participants" and that EPA is entitled to rely on published study results instead (Coalition of Battery Recyclers Ass'n v. EPA, 604 F.3d 613, 623 (D.C. Cir. 2010)). 
Commenter (6922) notes that the Courts rejected a challenge to a CAA rulemaking, the U.S. Court of Appeals for the District of Columbia held that requiring agencies to obtain and publicize the data underlying all studies on which they rely "would be impractical and unnecessary.",  The commenter provides that, where EPA relied on the results of the scientific studies in the rulemaking record rather than on the raw data underlying those studies, the public was not denied meaningful participation in the rulemaking process.
Response: This rule does not categorically exclude any study because the underlying dose-response data are not available, either publicly or via restricted access, for independent validation. If a study is identified as pivotal science, EPA will still consider the study but may give it lesser consideration if the dose-response data are not available for independent validation. 
Alternative Approaches to Data Availability
Comment: Commenter (6362) recommends that greater weight be given to studies using validated test methods and procedures, models, and approaches when and where those data are based on publicly accessible data, and transparent computer algorithms. The commenter states that other scientifically relevant and reliable studies and data should not be eliminated from consideration, but rather, accorded less weight when integrating evidence from multiple studies within and across different lines of evidence. 
Commenters (6918, 9224) recommend that when data unavailable to the public is used, it should be acknowledged in an open process, including in the proposed or final rule's preamble, and that the Administrator should take the unavailability of underlying data and/or models into account in deciding what weight to give to a study. 
Response: The commenters' recommendations are generally consistent with this rule. If a study is identified as pivotal science, EPA will still consider the study but may give it lesser consideration if the dose-response data are not available for independent validation. 
4.1.1.1.2 Disclosability of Underlying Data
Privacy Data 
Does the Handling of Privacy Data Conflict with Federal Laws?
Conflicts with Federal Privacy Laws/Policies and Confidentiality Agreements/Case Law
Comment: Commenters (0569, 0777, 1476, 1791, 1973, 1997, 2002, 2004, 2171, 2413, 2431, 2907, 2908, 4410, 4542, 4588, 4590, 4595, 4840, 4872, 5012, 5173, 5176, 5180, 6098, 6111, 6117, 6125, 6133, 6137, 6170, 6181, 6188, 6191, 6195, 6446, 6788, 6898, 6916, 6919, 6924, 6932, 8275, 8281-PH20, 8281-PH46, 9227) assert that, in many cases, releasing privacy data in the manner EPA proposed could be a violation of health privacy laws and confidentiality agreements with human subject participants. Commenters cite health privacy laws and/or confidentiality agreements legally mandated by IRB regulations, HIPPA, 21st Century Cures Act, Health and Human Services' "Federal Policy for the Protection of Human Subjects" regulations, the Food and Drug Administration (FDA) "Protection of Human Subjects" (also known as the Common Rule) and other FDA requirements, Procedures for Disclosure of Records Under the Freedom of Information Act (FOIA) requirements, and the National Death Index. 
Commenter (4590) states that virtually all studies of population-level human health effects of pesticides, air pollutants, water contaminants, drugs, and nutrients, as well as post market surveillance for adverse reactions to medications, require a commitment to the confidentiality of the involved or impacted persons. According to commenter (6181), it is not possible for a host of epidemiological studies and long-term health studies to open their datasets without publishing sensitive information that cannot, by law, be made public. Commenter (6916) states that the proposal provides no discussion of how underlying data of relevant studies can be released consistent with the various laws and regulations protecting privacy and prohibiting disclosure. The commenter asserts that the failure to "consider an important aspect of the problem" renders the proposal arbitrary and capricious. Commenter (6133) expresses concern that the proposal conflicts with decades of statutory, regulatory and institutional progress to simultaneously protect public health and privacy. Commenters (5176, 6898) notes that the proposed rule would require researchers to make all data publicly available, which in some circumstances would not be ethical because of important patient confidentiality protections.
Commenters (2171, 5012, 6111, 6133, 6188, 6924, 8281-PH20, 8281-PH46, 9227) assert that the proposal conflicts with requirements of HIPAA. Commenter (6111) states that, for any research carried out by healthcare providers that involves the handling of individually identifiable health information, the HIPAA Privacy Rule imposes strict confidentiality requirements. Commenter (6188) states the proposal would require the sharing of data protected under HIPAA and otherwise barred from disclosure. 
Commenters (2171, 4410, 4542, 4590, 4840, 6111, 6133, 6919, 6924, 8281-PH20, 9227) express concern that disclosure of analytic data sufficient to fully replicate study analysis could breach confidentiality requirements upheld by public and private research through IRBs. Commenters (4840, 9227) assert that under the Federal Policy for the Protection of Human Subject (also known as the Common Rule), for all federally-funded studies involving human research subjects, researchers must first obtain IRB approval and informed consent from study participants. Commenter (4590) states that Institutional committees for human subjects' protection routinely require non-disclosure of the subjects' identities; when disclosure is permitted, it is only in the most urgent and rare circumstance, usually when it is essential to the protection and safety of an individual subject. Commenter (6133) states that the proposal's provisions concerning the public sharing of underlying data from these studies directly contradict the requirements researchers adhere to under the purview of IRBs, which typically require investigators to ensure study participant confidentiality and data security. Commenter (6916) states that the criteria for IRB approval of research are at the very least in tension with the proposal's directive that "dose response data" are "publicly available in a manner sufficient for independent validation." Commenters (6924, 9227) provide that IRB reviews each human subjects research project to ensure that the specific research protocol protects individual rights. The commenters add that participants much be notified about the degree to which the confidentiality of their records will be maintained and must receive appropriate notification and give consent if study data is to be shared outside the research team. Commenter (9227) suggests that for studies that had received IRB approval prior to finalization of this proposed rule, there may be no practical opportunity to make the data publicly available. The commenter contends that, even for new studies, it may be extremely difficult, require additional (often unavailable) funding for elaborate protective measures, or simply impossible to obtain IRB approval for protocols that would allow the data to be made publicly available.
Commenter (6098) provides that FDA regulations and Procedures for Disclosure of Records Under FOIA requirements prohibit the sharing and disclosing of CBI and personal data: 
           FDA- 21 CFR Chapter 1 Part 20 subpart D 
               a. 61 
 "Trade secrets and commercial or financial information which is privileged or confidential."
               b. 63 
 "Personnel, medical, and similar files, disclosure of which constitutes a clearly unwarranted invasion of personal privacy.
 (a) "The names or other information which would identify patients or research subjects in any medical or similar report, test, study, or other research project shall be deleted before the record is made available for public disclosure."
 (b) "The names and other information which would identify patients or research subjects should be deleted from any record before it is submitted to the Food and Drug Administration. If the Food and Drug Administration subsequently needs the names of such individuals, a separate request will be made."
         2. 6 CFR Chapter 10, Part 1001.4(a) (Procedures for Disclosure of Records Under the FOIA, Categories of Exemptions)
 (4) "Trade secrets and commercial or financial information obtained from a person and privileged or confidential;"
 (6) "Personnel and medical files and similar files the disclosure of which would constitute a clearly unwarranted invasion of personal privacy;
 (7)(iii) "Could reasonably be expected to constitute an unwarranted invasion of personal privacy;"
 (7)(iv) "Could reasonably be expected to disclose the identity of a confidential source, including a state, local, or foreign agency or authority or any private institution that furnished information on a confidential basis, and, in the case of a record or information compiled by a criminal investigation, or by an agency conducting a lawful national security intelligence investigation ,information furnished by a confidential source;
 (7)(vi) "Could reasonably be expected to endanger the life or physical safety of any individual"
 (9)" Geological and geophysical information and data, including maps, concerning wells."
Response: This rule does not require the release of any privacy or other confidential data. This rule does not conflict with HIPPA or any other privacy statute.
Does Not Conflict with Federal Laws 
Comment: Commenter (6179) states that neither HIPAA nor the Privacy Rule place a blanket prohibition on releasing dose response data and models when protected health information (PHI) is de-identified. The commenter contends that the proposal does not explicitly claim new authority under existing environmental statutes to disclose protected sensitive and confidential information. According to the commenter, instead, they understand the proposal as seeking to provide the public with information that contextualizes pivotal regulatory science so comments on significant regulatory decisions can be more meaningful, while still protecting private, sensitive, and confidential information. The commenter suggests that EPA can clarify this position by explicitly stating that any final regulations arising from the proposal do not assert authorization under the law to disclose protected sensitive or confidential information, and that any such claim must be independently based on a statutory grant of authority from Congress. 
Response: This rule does not require the release of any privacy or other confidential data. This rule does not conflict with HIPPA or any other privacy statute.
International Privacy Data Laws
Comment: Commenters (1973, 6873) express concern that the proposal does not address conflicts with international requirements for data disclosure. 
Commenter (1973) states that European and Canadian privacy laws do not allow personal information from participants in research studies to be made public. The commenter states that Europe has just tightened its data privacy laws with the General Data Protection Regulation. According to the commenter, regulation limits movement of private data outside of the EU, which would prohibit the type of data access EPA proposes. The commenter states that it defines private data to include medical, physical, physiological, genetic, mental, economic, cultural or social identity of that natural person and it states that the data controller must "demonstrate that the data subject has consented to processing of his or her personal data." According to the commenter, if the data were made public, even (if possible) in unidentifiable ways, the data controller could still not demonstrate that it was only being used for the purposes and by the people to which the participant in the study consented. The commenter provides:
      As an example of the highly confidential personal information included in these types of studies, consider the Canadian Community Health Survey Cohort, which followed a cohort of 300,000 people and looked at the association of PM2.5 with mortality. The participants were linked to their tax records to obtain individual level information on income, which was used as a variable in the analysis. The data includes individual education, marital status, age, sex, immigrant status, minority status, weight, smoking, diet, alcohol consumption, and neighborhood level census data. Because of the highly confidential nature of the data, even the investigators did not have the data. They performed their analyses on the computers at Statistics Canada, where the data resides. Yet this study is a critical study for EPA to consider as it reviews the adequacy of its current 12 μg/m3 PM2.5 standard, because essentially all the participants lived in locations below that standard (the 95th percentile of exposure was 11.3 μg/m3). The Canadian Statistics Act prevents these data from being made public, yet how can EPA set standards based on the best available science without considering it?
      Or consider the recent study of the Medicare cohort by Di and coworkers. One key result came from a restricted analysis including only people residing in U.S. ZIP Codes with annual PM2.5 concentration below the 12 μg/m3 NAAQS. That analysis included 247,682,367 person-years of follow-up and 11,908,888 deaths. They found a 1.36% (95% Confidence Interval (CI) 1.31%, 1.41%) increase in mortality rate per 1 μg/m3 increase in PM2.5 below 12 μg/m3. Yet their Data Use Agreement with Medicare prohibits them from providing these data to anyone. How can EPA set a standard based on the best available science when such a huge study, with hundreds of millions of observations in the range relevant for determining the adequacy of the standard is excluded from consideration? Particularly when anyone else can apply to the Centers for Medicare and Medicaid Services for access to the data?
      Or consider the ESCAPE study. This study combined 22 cohorts across Europe, estimated address specific concentrations of multiple air pollutants for each participant, and examined the association of air pollution with mortality in these participants. European data privacy laws would certainly prevent the data from these people being made public as EPA proposes requiring before considering the results in setting standards for air pollution.
Response: This rule does not require the release of any privacy or other confidential data. 
Comment: Commenter (6873) states that there are well-established requirements and procedures for protecting the privacy rights of research participants encoded in the international Helsinki Declaration of the World Medical Association. The commenter contends that coming up with new guidelines for releasing private data, even if the intent is to mask this data and make it available only to government scientists, will require review and approval by the U.S. (the FDA and other agencies) and the global research community.
Response: This rule does not require the release of any privacy or other confidential data. This rule does not conflict with HIPPA or any other privacy statute.
Handling of Privacy Data Under the Proposal 
Comment: Commenters (0040, 1331, 4872, 5122, 6098, 6126, 6376, 6708, 6898, 6919, 6920, 8281-PH10, 8281-PH24, 8817) urge EPA to ensure the protection of individuals' privacy data. Commenters (0040, 6920) state that research participants enter into non-disclosure confidentiality agreements that require that their information be protected. Commenters (1006, 1011, 1459, 4588, 6191, 6932, 6935) express concern that it may not be reasonable to expect that all data can be made publicly available without inadvertently exposing privacy data. Commenter (6126) suggests that, in order to protect individual's privacy data, more attention is needed to assess the risk of re-identifying data before public releases occur and to ensure important disclosure avoidance practices. 
Commenter (4588) states that researchers could redact and anonymize personal health data, but that (1) redaction would involve considerable time, cost, and administrative burden, (2) there are limits as to how much an individual's identity can be protected, especially in small communities where basic identifiers may be enough to recognize an individual. If the number of affected individuals in a study is small -- and this is not unusual with rare or serious outcomes -- then there is virtually no capacity to maintain anonymity for individuals who have consented to participate.
Commenter (1331) notes that, in the proposed rule, the mechanisms used to determine whether data can be made available or not do not go far enough to ensure regulations involving public health data will not be available to the public. The commenter states that the rule suggests the transparency requirement could be waived if not practical to implement in certain cases, but given the similarities between the data used in environmental epidemiology, a more specific protocol for the handling of this data in relation to the transparency rule would ensure privacy. The commenter states that a section specific to exempting public health data in the rule would lower regulatory costs significantly compared to case-by-case mechanism currently proposed.
Commenter (6098) encourages the EPA to coordinate and work with the other science based regulatory bodies to create an appropriate system of protecting and ensuring patient data to ensure that studies dealing with impact on human health are not discounted nor are subjected to privacy violating policy decisions as in this proposed regulation. The commenter states that patient data demonstrating the effects of substances on human health must not be excluded from any assessment performed by the EPA on determination of the hazards of a substance. The commenter further encourages the EPA to coordinate health data efforts with the FDA, OSHA and other Health and Human Services bodies to ensure a comprehensive system and consistent treatment and verbiage across all regulatory bodies when handling such data.
Commenter (6376) suggests that scientific studies involving or including sensitive health data should not be at risk of exclusion over concerns related to personal information being exposed. The commenter requests that EPA clearly establish guidelines to ensure transparency does not come at the cost of widespread exclusion due to inadequate protections for the variety of sensitive data included in different pertinent studies. The commenter states that the EPA has already shown sensitivity on this issue but should expand the mechanisms to include more than CBI, so as not to exclude widely accepted, peer-reviewed studies, which will now be entered into the public domain.
Commenter (6919) states that government agencies have a legal responsibility to protect residents from any risks associated with sharing personal health information  -  this includes discrimination. According to the commenter, while transparency is important, health-related data still holds a special status that must be maintained. The commenter contends that researchers can only conduct high-quality and reproducible research by protecting the personal health information of study participants. The commenter asserts that public health professionals must retain private personal data in their own study records in order to conduct accurate and useful surveillance to inform local, state, and national policies. 
Commenter (8275) states that the principal focus of the proposed rule is to require full access to original data for scientific studies in order for those studies to be used for regulatory decision-making. The commenter suggests that, while the preamble to the rule indicates that this could be done while maintaining the protection of privacy and confidentiality of research participants, the proposed rule does not include specific provisions to make these protections possible.
Commenter (9227) states that even EPA's own Science Advisory Board (SAB) voiced concerns to making even limited releases of privacy data, saying:
      The proposed rule oversimplifies the argument that "concerns about access to confidential or private information can, in many case, be addressed through the application of solutions commonly in use across some parts of the Federal government." For studies already completed or underway, the participation of human subjects is undertaken according to terms approved by the cognizant IRB. These terms can vary from study to study. In some cases, the data cannot be released simply by redacting portions of it. For example, data may have been collected with an assurance to the participating individuals that their data would be kept confidential.
Response: This rule does not require the release of any privacy or other confidential data. 
Comment: Commenters (2171, 2182, 2495, 6133) assert that EPA has not presented a compelling rationale for why the standard privacy needs to be violated. According to commenter (2495), understanding public health impacts requires only conducting valid statistical analyses on available data, and the disclosure of privacy information is not required to conduct these analyses.
Response: This rule does not require the release of any privacy or other confidential data. This rule does not conflict with HIPPA or any other privacy statute.
Comment: Commenters (1997, 5011, 6788, 9227) generally assert that the proposal would require researchers to choose between protecting study participant privacy (which is their legal responsibility) or informing the EPA's efforts to protect public health and safety. Commenter (6788) asserts that if researchers respect the privacy of their subjects, their research cannot be used by the EPA. Commenter (9227) states that some researchers might respond by choosing to work only on public administrative data sets, but this would harm rather than strengthen science quality by curtailing scientific inquiry. Thus, the commenter believes that the effects of EPA's proposed approach would cause some researchers to choose not to pursue research with human subjects, stifling scientific discovery, while others would forgo compliance with EPA's regulatory requirements and have their research ignored by EPA. As a result, the commenter contends that the EPA's proposal would both discourage the development of best available science as well as EPA's use of it.
Response: This rule does not require the release of any privacy or other confidential data. This rule does not conflict with HIPPA or any other privacy statute. Thus, this rule would not "require researchers to choose between protecting study participant privacy (which is their legal responsibility) or informing the EPA's efforts to protect public health and safety."
Comment: Commenters (0569, 2477, 4590, 4878, 5173, 6111, 6446) assert that, even when medical data is masked or coded, subjects can be identified if the population size is small enough or the defining characteristics described are rare enough (e.g., age, gender, income, illnesses, geographic location, specific profession). 
Response: This rule does not require the release of any privacy or other confidential data. This rule does not regulate how medical or other private data are to be handled.
Comment: Commenter (6935) suggests that a final rule should encourage, but not mandate, that future studies provide transparency through confidentiality protections such as data masking.
Response: This rule does not regulate how medical or other private data are to be handled.
Comment: Commenter (0559) states that the NIH has established effective policies governing the sharing of data among researchers and with appropriate access by the public at large. The commenter further states that many journals, including the Endocrine Society, are establishing guidelines and standards for the submission of datasets to repositories that are consistent with NIH policies and the FAIR (findable, accessible, interoperable, re-usable) principles for scientific data management and stewardship. According to the commenter, research that abides by these policies and has been published in high-quality peer-reviewed journals should be considered independently validated and sufficiently available to the public.
Response: This rule does not regulate how medical or other private data are to be handled or require that they be shared.
Balancing Transparency and Handling of Privacy Data
Comment: Commenters (2788, 4589, 5129, 6111, 6125, 6133, 6188, 6366, 6909) suggest that the EPA rule needs to provide a balanced approach between transparency and privacy data privacy concerns of study participants and legal and ethical restrictions on the sharing of sensitive data. Commenter (6908) states that transparency is a virtue but not the only virtue and the agency should not be tempted to equate transparency with quality. The commenter states that, in addition to its own quality standards, the objective of the transparency is to improve quality through additional higher levels of trust or higher levels of scrutiny (which in turn generates higher trust). 
Commenter (6111) asserts that transparency does not mean violating the confidentiality of study participants or making all raw data publicly available. According to the commenter, the proposed rule does not comport with the fundamental approach to conducting scientific and medical research that is the standard practice for experienced, advanced scholars and researchers. The commenter states that, in the professional scientific and medical research community, "transparency" means clear and detailed disclosure of all methods, data, assumptions, and uncertainties. Studies are considered "transparent" when the study design and methodology are clear enough to allow other scientists to challenge assumptions, test hypotheses, and either reproduce or replicate the study to determine whether the results obtained are consistent with the original study.
Response: EPA agrees that the clear and detailed disclosure of all methods, data, assumptions, and uncertainties is important. This rule does not require the disclosure of privacy data or otherwise violating confidentiality.
Commenter (4589) expresses concern that the proposal places undue emphasis on the need for "transparency" in the assessment of published studies and does not establish clear guidelines regarding the use of studies that include privacy data that have important relevance for risk assessment. The commenter requests that the proposal state explicitly that data "transparency" must necessarily be waived regarding the medical status of individuals who may have suffered adverse health problems as a result of occupational and environmental exposures. The commenter further suggests that the proposal clarify how medical confidentiality will be protected under considerations of expanded disclosure.
Commenter (6125) states that the National Research Council analyzed the balancing of transparency and disclosure of privacy data as one of many competing factors that needs to be balanced in the interest of making gradual progress over time. According to commenter, the report presents a summary of recommendations which include:
      No one way is optimal for all data users or all purposes. To meet society's needs for high quality research and statistics, the nation's statistical and research agencies must provide both unrestricted access to anonymized public-use files and restricted access to detailed, individually identifiable confidential data for researchers under carefully specified conditions. 
      Ultimately, decisions about how much disclosure risk is acceptable in order to achieve the benefits of greater access to research data involve weighing the potential harm posed by disclosure against the benefits potentially foregone, as well as a judgment about who should make those decisions.
Commenter (6188) states that, while EPA specifically solicits comment on balancing "increased transparency" with protection for copyrighted or CBI, EPA has asked for input only on how to "provide protected access," not how to balance that access with appropriate privacy protections, creating the impression that no such balancing calculus is required in the case of individual health data. According to the commenter, this focus overlooks the significant personal privacy pitfalls of the proposal. 
Commenter (2788) urges EPA to not censor science in the name of transparency. The commenter quotes former Associate Director for Environmental Health Emergencies at CDC Dr. Scott Deitchman:
      "Transparency in science is a movement whose time has come, but it can't be used to hobble science. Such a rule needs to explain how relevant data that can't be made public, particularly health data containing personally identifiable information (PII), can be used to contribute to the regulatory process. Such a rule also should call for a level playing field, requiring that ALL parties participating in the discussion of proposed regulation must transparently disclose the existence of all relevant studies - you can't bury a study whose conclusions don't support your position - as well as making the data available within necessary constraints to protect PII. The rule also should explain how the agency will transparently comply with the Office of Management and Budget's guidelines on the selection of peer reviewers, who will assess whether the data were correctly interpreted."
Similarly, commenter (5129) states that transparency should not be prioritized over the best conclusive data. The commenter notes that EPA should not create a false conflict between violating the privacy of the people who participate in research studies and protecting the public's health. The commenter contends that EPA's job is not to help industrial studies that are inherently biased to help their profits but to support the health of the American people.
Commenter (6909) states that this rule is based on false premises about transparency and scientific reliability. The commenter states that, in general, transparency in science is a good thing, however, EPA seems to have proposed this rule under the false premise that only scientific studies for which all underlying data is made publicly available can be adequately evaluated. The commenter states that this is simply not the case, and any discussion about mandating transparency through a bright line rule such as the one currently being proposed should be grounded in this fact. 
Response: This rule does not regulate how medical or other private data are to be handled or require that they be shared.
Potential Harms Posed by Disclosure of Privacy Data
Harassment, Stigma and/or Retaliation
Comment: Commenters (5176, 6119, 6125, 6153, 6874, 6896, 6943, 8281-PH13) express concern that the proposal does not address the issue that the proposal requirements for open access to study data could make those who provide this information vulnerable to harassment and/or retaliation.
Commenter (5176) asserts that research participants often agree to provide their information on the condition that it remain confidential, out of fear of retaliation for participating in a study that may negatively impact the interests of powerful parties (i.e., landlords, employers, industry leaders, advocacy groups, etc.). Commenter (6119) states that protection of individuals', households', and families' private information cannot be assured if such information is openly available to the public. 
Commenter (6125) states that the NRC report recognizes that protecting confidential information is critical to assuring research participants that they do not need to fear harm:
      It is essential that respondents believe they can provide accurate, complete information without any fear that the information will be disclosed inappropriately. Indeed, if the information was disclosed, harm might come to an individual respondent. Many government- sponsored surveys ask about sensitive topics (e.g., income or alcoholic beverage consumption), as well as about stigmatizing and even illegal behavior. The disclosure of such information might subject a respondent to loss of reputation, employment, or civil or criminal penalties. 
Commenters (6153, 6874) state that privacy is of concern where data sets include information that could lead to participants facing stigma if it were to be made public. Commenters express concern that in reproductive health research, such information includes pregnancy outcomes such as abortion and miscarriage, and developmental delays that can result from environmental exposures during pregnancy or early life. Commenter (6943) states that study participants should not be subjected to additional stress over legitimate concerns that their personal health information could be released, be put at risk even if redacted, or, worse still, interpreted incorrectly. 
Commenter (6896) states that protecting sensitive worker information is of concern in the construction industry as construction is a highly physical occupation; hiring is done quickly and often without much oversight; and construction employees are concerned about being blackballed when employers have reason to view them as physically compromised or vulnerable to health or safety risks on the jobsite. The commenter states that, for example, construction employers confirmed that they would be likely to refuse to hire employees whose medical records revealed decreased lung function. The commenter further states that construction workers have also expressed reticence to report injuries and illnesses because they feared employer retaliation or loss of future work opportunities. According to the commenter, when construction workers participate in research, they risk individual entitlements, impairment of family relationships, job retention, peer pressure, promotion opportunities, and obtaining and retaining medical and health benefits.
Commenter (8281-PH13) states that people talk to them about being exposed to pesticides, and that they are afraid to make a report because they fear losing their job, or fear retaliation.
Response: This rule does not regulate how medical or other private data are to be handled or require that they be shared. Dose-response data that contains privacy data or any other confidential data would be considered available for independent validation if it were available through restricted access consistent with all privacy laws.
De-identification of Data May Reduce or Negate the Usefulness of the Data
Comment: Commenters (1420, 1945, 4542, 4878, 5170, 5173, 5176, 6111, 6896, 6919) assert that there are many studies where it is impossible to de-identify data to a level where the data is both usable and participants are properly protected. Commenters (0100, 1954, 5419, 6120, 6125) assert that, in epidemiological studies, redacting information consistent with privacy requirements may render health effects data meaningless/unusable.
Commenter (6919) asserts that researchers can only conduct high-quality and reproducible research by protecting the personal health information of study participants. Likewise, the commenter states that public health professionals must retain private personal data in their own study records in order to conduct accurate and useful surveillance to inform local, state, and national policies. The commenter suggests that the proposed rule would effectively keep critical studies in the dark  -  the opposite of transparency.
Commenter (6120) expresses concern that the proposed rule will prevent the use of key public health data whenever epidemiologists keep personal data confidential despite publishing in peer reviewed journals. The commenter suggests that, while the proposed rule cites requirements of major scientific journals for the availability of data, the editors of those journals have also issued a joint statement discussing the importance of including all relevant studies articulating circumstances where data cannot be fully shared, such as when it includes personal identifiers.
Commenter (4542) states that de‐identification methods can degrade the accuracy of data for multi‐variate statistical analysis or data mining, potentially masking important relationships, such as racial differences in outcomes. Commenters (0100, 1954, 5176, 6125) assert that since an address is identifying information, it cannot be released with the epidemiological data; but without that information, the health effect data is meaningless as there is no longer a means of correlating the effects to degree of exposure. Commenter 5149 contends that de-identification of data it develops to a form that would be acceptable for sharing would preclude their use in meaningful air pollution epidemiological research because identifying information such as the ZIP codes of residence is required to assign pollutant exposures, and identifying information such as age, gender, and race is required to allow adjustment for these factors in epidemiological models. 
Commenter (4878) states that, while a study showed that consuming water from a private well located in an area with historical pesticide use is associated with an increased risk of Parkinson's disease, due to the nature of wells  --  typically serving a relatively limited number of people within a very small radius  --  the detail needed to perform the study renders proper de-identification impossible. Commenter (6111) states that, for some studies (e.g. prospective cohort studies that include extensive personal data; environmental health effects studies), it is impossible to de-identify the data without negating its scientific value. 
Commenter (6133) states that the 2009 Integrated Science Assessment for PM2.5 notes that "[a]ppropriate statistical adjustment for confounders requires identifying and measuring all reasonably expected confounders." Commenter states that exclusion of some potentially sensitive confounding variables from an underlying dataset likely would lead a different team of investigators to a different result -- not as a result of any true difference in the quantitative exposure-effect relationship, but because the original work relied on complete data sets and the new analyses would not -- due to the proposal. Commenter states that the resulting discrepancies in quantitative findings could serve as motivation to call the original study results into question due to faulty and incomplete re-analyses. 
Response: This rule does not require that in order to be used dose-response data be anonymized to the point that it is not informative. Dose-response data that contains privacy data or any other confidential data would be considered available for independent validation if it were available through restricted access consistent with all privacy laws.
Comment: Commenter (6125) states that at a 2014 NAS workshop on data sharing, Greenbaum of the Health Effects Institute (HEI) noted that sharing de-identified data files would not be useful for air pollution cohort studies "because a de-identified one would not allow you to have location and a variety of other things that are absolutely essential." 
Response: This rule does not require that in order to be used dose-response data be anonymized to the point that it is not informative. Dose-response data that contains privacy data or any other confidential data would be considered available for independent validation if it were available through restricted access consistent with all privacy laws.
Comment: Commenters (2336, 2429, 6133, 6158, 6373, 6446, 6924, 9227) assert that, to protect against re-identification as required by HIPPA, it would be necessary to remove so much demographic information from the dataset that other scientists would not be able to perform a meaningful reanalysis or validation of the data. Commenter (2336) states that, under HIPAA, researchers would have to redact the participants' names, birthdates and death dates, among other information, before they could release their health-outcomes data publicly. 
Commenter (9227) cites the HIPPAA guidelines document: "Of course, de-identification leads to information loss which may limit the usefulness of the resulting health information." 
Commenters (6158, 6924) state that a workshop organized by the National Research Council in 2000 highlighted that the more identifiers within a dataset are redacted, the less useful the resulting data become in terms of reanalysis. Commenters assert that this indicates that in many cases, the results of a re-analysis completed with de-identified or masked data would necessarily be different than the original study. Commenters argue this runs counter to the rule's stated intent of using these techniques to allow public disclosure of data and validation of scientific findings. Commenters assert that the agency can't make data available (given its need to be de-identified) and simultaneously have its reanalysis be meaningful. 
Response: This rule does not require that in order to be used dose-response data be anonymized to the point that it is not informative. Dose-response data that contains privacy data or any other confidential data would be considered available for independent validation if it were available through restricted access consistent with all privacy laws.
Comment: Commenters (2429, 5015, 6909, 6925, 8281-PH20) express concern that the proposed rule does not address impacts on the quality of analysis that arise due to information loss.
Commenter (2429) asserts that, after simple data masking, coding, and de-identification, it is not guaranteed that statistical and inferential reproducibility will be achieved. Commenter (5015) expresses concern that it is not possible to conduct a high-quality air pollution and health study without knowing the locations of those being studied, i.e. where they live, and what are the sources and levels of their air pollution exposure. Commenter (8281-PH20) states that data de-identification and masking techniques can degrade the accuracy of data for further analysis. 
Commenter (6909) states that de-identification techniques have the unfortunate side-effect of degrading the data's analytical integrity for purposes of further analysis  -  exactly the kind of further analysis EPA says it wants to enable by promulgating this rule. Commenter states that any claim that it is a simple matter for health data underlying a study to be anonymized and made public should be met with deep skepticism.
Commenter (6925) states that, requiring that health study data be made public, while protecting privacy, would likely dilute the results by introducing noise into the measurements. Commenter states that one of the most critical pieces of information that would certainly be protected from disclosure is patient addresses  -  this could potentially be "de-identified" by providing a larger geographic identifier, such as a zip code or county of residence. Commenter states that this process of deidentifying also leads to greater misclassification of exposure, and the higher likelihood that any true effect that exists will not be detected, simply due to imprecise exposure estimation. 
Response: This rule does not require that in order to be used dose-response data be anonymized to the point that it is not informative. Dose-response data that contains privacy data or any other confidential data would be considered available for independent validation if it were available through restricted access consistent with all privacy laws.
Participation in Research Studies
Comment: Commenters (1335, 1831, 1973, 2171, 2904, 4410, 5012, 6111, 6125, 6133, 6170, 6171, 6446, 6872, 6876, 6882, 6884, 6896, 6898, 6909, 8281-PH6, 8281-PH10, 8281-PH13, 8281-PH35, 8281-PH-10, 8281-PH48, 8818) express concern that this proposal will decrease people's willingness to participate in future studies of environmental exposures, due to concerns about trust and privacy.
Commenter (2904) states that, while transparency is important in science in order to maintain rigorous scientific method, a balance must occur to protect individuals involved in that research. The commenter states that the proposed rules will jeopardize research in small vulnerable communities who are often the very ones needing EPA and public health research. According to the commenter, a change in current standards will greatly impact research in these communities and their participation in much needed public health and environmental science. The commenter notes that tribes will not participate in research if their confidentiality and intellectual property rights are not ensured. The commenter adds that many important and disadvantaged populations are located in rural and other small communities that make de-identification of individuals and their data a significant challenge. The commenter explains that many participants or subjects in research studies come from protected groups who have a history of exploitation by the United States government, including Native American and Alaska Natives, and that Native peoples in the U.S. are unduly burdened by health disparities and by health disparities related to environmental hazards. 
Commenter (1395) states that, if researchers are allowed to reveal confidential medical records, no one would agree to have their medical records used for such studies, and then this research could not be conducted. Commenter (1831) states that if, for instance, someone could not participate in a drug trial for a cure for genital herpes without announcing his affliction to the world, he would likely not participate; as a result, knowledge would not advance, suffering would go unrelieved, and medical research becomes impossible. Commenter (8281-PH35) states that the proposal would make public health research impossible, or much, much more difficult. 
Commenters (2171, 4410) assert that it is well documented that privacy assurances are essential to recruit and maintain population-based cohorts. Commenter (1973) states that concern about privacy would have widespread ramifications for health research in general, since many of the cohort studies of air pollution (e.g. Framingham Heart Study, American Cancer Society Study, Nurses' Health Study, MESA study) were originally funded for other purposes and continue to do important research on cardiovascular disease and cancer.
Commenters (1420, 6125, 6133, 6872, 8281-PH6, 8281-PH10, 8281-PH24, 8281-PH13, 8281-PH48, 8281-PH48, 8817) assert that the proposal does not consider the negative effects it would have on recruitment for future epidemiological studies if members of the public had to permit access to sensitive personal and health information as a condition for study participation. 
Commenter (1420) states that, while the proposed regulation sounds well-intentioned, the reality is that public health research relies heavily on personal health data and few people will allow their data to be used if those data were publicly available. Commenter (6125) expresses concern that informing prospective survey participants that data that could be used to identify them might be made publicly available could make it more difficult to enroll subjects in future studies. Commenter (6872) states that the threat of confidential data disclosure would create a strong disincentive for people considering participation in such studies going forward. Commenter (8281-PH6) states that it may be more challenging for researchers to recruit participants for their studies because of the fear that personal data could be shared. Commenter (8281-PH24) states that volunteers will cease to come forward for scientific research if their medical history and geographic information will be made public, thus putting critical scientific research at risk. Commenters (8281-PH10, 8281-PH13, 8281-PH48, 8281-PH48, 8817) assert that, if we could not assure patient confidentiality, we would not have their participation. 
Commenters (6133, 6170, 6171, 6884, 6896, 6898) assert that human subjects will not be willing to provide sensitive information without a guarantee of confidentiality from the researchers for this information. 
Response: This rule does not require that dose-response data that contains privacy or other confidential data be made publicly available. Dose-response data that contains privacy data or any other confidential data would be considered available for independent validation if it were available through restricted access consistent with all privacy laws. Thus, the rule should not have a negative effect on study participants.
Comment: Commenter (6170) asserts that, in epidemiologic studies, most, if not all, human subjects will not be willing to provide such sensitive information without a guarantee of confidentiality from the researchers for this information. Commenters (6171, 6898) state that the inability of a scientist to guarantee confidentiality would be a major barrier to being able to recruit an appropriately diverse population of participants. Commenter (6884) states that farmworkers agree to participate because they are guaranteed that their information will remain anonymous, so that they do not risk retaliation for participating. Commenter (6896) states that construction workers will not participate in scientific research unless they know they will be protected from employer retaliation. Commenter (6896) states that sharing a dataset publicly as required by the proposed rule would directly violate the participant's consent. 
Response: This rule does not require that dose-response data that contains privacy or other confidential data be made publicly available. 
Comment: Commenters (6446, 6909) state that the fact that the public would be able to de-anonymize much epidemiological data, after de-identification processes have been applied, would almost certainly have a chilling effect on voluntary public participation in important research. Commenter (6909) states that, while it is not perfectly clear what EPA's intent is, it is possible given the vagueness of the proposed rule's language that EPA will see fit under some circumstances to make data public itself when it wishes to use a study as a basis for a regulatory action. Commenter (6909) states that, even where it is possible to recruit study participants, it could become challenging to recruit enough participants who adequately represent the population to make the study useful from a statistical standpoint. 
Response: This rule does not require that dose-response data that contains privacy or other confidential data be made publicly available. Dose-response data that contains privacy data or any other confidential data would be considered available for independent validation if it were available through restricted access consistent with all privacy laws. Thus, the rule should not have a negative effect on study participants.
Comment: Commenters (1973, 6909, 8281-PH6) express concern that states that, if the proposed rule is implemented it is possible that EPA will conduct less epidemiological research and scientists, organizations, and research institutions will be less inclined to participate in EPA-funded research because of the risk of improperly disclosing personal information. Commenter (1973) states that concern about privacy would prevent future research on environmental exposures.
Commenter (6125) states that the NRC report recognizes that protecting confidential information can be critical to getting good data: "The reason for confidentiality pledges and for stringent procedures to prevent disclosure is that they improve the quality of data collected from individuals, households, and firms. It is essential that respondents believe they can provide accurate, complete information without any fear that the information will be disclosed inappropriately."
Response: This rule does not require that dose-response data that contains privacy or other confidential data be made publicly available. Thus, this rule will not create a disincentive to new research. 
Other Potential Harms
Comment: Commenter (6137) states that, as the Seventh Circuit has explained, people may have privacy interests in unidentifiable personal information: "Imagine if nude pictures of a woman, uploaded to the Internet without her consent though without identifying her by name, were downloaded in a foreign country by people who will never meet her. She would still feel her privacy had been invaded." Commenter states that, according to a 1993 study, more than 60 percent of Americans want hospitals, pharmaceutical companies, and researchers to obtain patient consent before using medical information -- even if that information has been de-identified. 
Response: This rule does not require that dose-response data that contains privacy or other confidential data be made publicly available. 
Comment: Commenter (6147) states that there are sensitive ecological, cultural, and archeological data which must be kept private to assure their safety from vandalism, terrorism, and wanton destruction.
Response: This rule does not require that dose-response data that contains privacy or other confidential data be made publicly available. 
Comment: Commenter (4877) states that many studies also contain data that is sensitive in terms of security, business confidentiality, and intellectual property reasons, and this data should also be exempt from blanket data sharing requirements. Commenter (6373) states that, because an increasing number of epidemiological studies rely on spatial (e.g., residential address) and temporal (e.g., date of treatment) information to identify risk factors associated with exposures to environmental pollutants, releasing more data publicly can itself compromise national security. 
Commenter (8913) states that an unintended consequence of the rule is the national security hazard which flows from the requirement of publication of data sets featuring personal identifiers. Commenter states that the proposal would open a portal into troves of personal confidential information, including the medical, mental health and lifestyle records of cohorts, including individuals associated with military activity and critical infrastructure (frequently the subject of study due to exposure to dangerous chemicals and radiation) who may then be subject to blackmail. Commenter states that hacking and other cybersecurity incidents in recent years demonstrate the inability to fully safeguard such information.
Response: This rule does not require the public disclosure of any sensitive or confidential data. 
Confidential Business Information
Does the Handling of Confidential Business Information (CBI) Conflict with Legal Requirements?
Comment: Commenter (6098) states that trade secrets, commercial or financial information that is privileged or confidential are all protected by laws and regulations.
Commenter (6133) contends that the proposed rule's handling of CBI is unlawfully vague and arbitrary and capricious. According to the commenter, the differential treatment of health data and CBI submissions relevant and even integral to EPA's rulemakings, is the essence of arbitrary and capricious action.
Commenter (6196) expresses concern that industry-conducted studies may contain CBI required to be withheld by law.
Response: This rule does not require the public disclosure of any sensitive or confidential data, regardless of whether it is CBI or PII.
Comment: Commenter (6143) states that TSCA §14 provides explicit protections for CBI that is submitted to EPA. The commenter contends that TSCA itself recognizes that specific, limited classes of information should remain confidential, even if they play a role in decisions that the Agency makes pursuant to TSCA. 
Response: This rule does not require the public disclosure of CBI. CBI will be handled in accordance with applicable statutes and regulations.
Handling of CBI Under the Proposal
Comment: Commenters (2401, 2787, 6133, 6148, 6196, 6361, 6872, 6920) oppose or express concern with the proposal's approach to CBI. 
Commenter (6148) states that in some circumstances full transparency may not be achievable due to CBI. Commenter (2787) states that industry representatives have expressed concerns about requiring public disclosure of data, such as CBI, citing the potential for improper use of such data by competitors. 
Commenter (6361) expresses concern that many studies performed using data from industrial sources would be excluded because they contain CBI, such as product formulations. The commenter states that, under federal law, there are criminal penalties for disclosing CBI, and without demonstrating how compliance will be achieved in tandem with making research data publicly available, it creates serious liability concern for the EPA's intentions.
Commenter (2401) opposes the proposed rule because it will have a negative effect on the relationship between private businesses across a variety of sectors and the EPA. The commenter states that, for proprietary reasons, many firms cannot publicly share results from research they have conducted (in companies in diverse sectors including construction, pharmaceutical, and retail products). The commenter states that this rule would effectively gag companies from sharing such research with the EPA because the work would then have to be made public in its entirety, effectively stopping private companies from cooperating and reaching out to the federal government to help inform regulation and best practice. 
Commenter (6872) states that, while CBI data is the least transparent information utilized by EPA and legitimate questions exist regarding the scope of CBI designations, the mandatory public release of appropriate CBI risks undercutting U.S. businesses that are increasingly working to formulate greener, less toxic chemical products that provide a market advantage. The commenter states that mandatory public release of data appropriately considered CBI would create disincentives to the generation and submission of such data to EPA, hindering TSCA and pesticide assessments and could also jeopardize US environmental business innovations.
Commenter (6175) states that agricultural landowners have unique concerns as many also maintain places of residence on their farms, and data disclosed could include physical addresses and other identifying information. Any new data disclosure requirements should maintain consistency with established statutory privacy protections and judicial precedent.
Response: This rule does not require the public disclosure of CBI, nor does it include any requirements for the use or management of CBI data. CBI will be handled in accordance with applicable statutes and regulations.
Comment: Commenters (6857, 6935, 9243) suggest that any final rule should include a process for safeguarding CBI. Commenter (3971) contends that confidentiality can be protected in all the studies upon which EPA relies. Commenter (6935) reports that many states have CBI strategies that might be evaluated. Commenters (6857, 9243) state that, in some instances, there could be legal requirements preventing CBI from being disclosed while in others there may be concerns centered around competition in the marketplace. Commenters (6857, 9243) state that, in either instance, EPA should consider and provide some sort of mechanism for entities to share information with the agency when necessary without suffering unnecessary adverse impacts. 
Commenter (6362) suggests EPA consider the following strategies to protect CBI.
 If necessary, when specialized expertise is needed, EPA could contract with an independent third-party science reviewer to confirm those findings. 
 EPA might also consider an approach followed under FIFRA where Data Evaluation Records of studies are made publicly available, but not full studies. 
 Another approach is that of the European Union's REACH program, which makes Robust Study Summaries (RSS) publicly available, while protecting from disclosure the competitively sensitive underlying data of health and safety studies. 
 When protecting data while also promoting data access, the NIH guidelines should be consulted. Many of these guidelines could be applied in EPA's implementation of this proposed policy under each of the statutory programs EPA administers to ensure the guidelines adopted suit the specific needs of each statute. 
 In circumstances where company CBI and other intellectual property may be implicated, EPA should confer with the CBI data owner to determine how to make that data available to the greatest extent possible without disclosing the CBI within that data, study, or model. The commenter provides the following example:
            For example, under TSCA, while the summarized study results, analysis, and final report may be publicly available, the underlying data in a health and safety study may qualify as CBI when the underlying data are not in the public domain and that data provides a commercial value to its owner. In such circumstance, it is the availability of the underlying data that determines whether or not an unpublished study can be used by a competitor to support its notification or registration of a substance overseas without obtaining ownership or citation rights to use such data, depriving the data owner of the value of its investment in the underlying data. Current EPA regulations require chemical manufacturers to submit health and safety studies under some circumstances. However, it is noteworthy that none of these regulations routinely require study submitters to submit underlying data along with a final report. This indicates that the final report likely communicates sufficient information about the potential health and environmental effects to the public when a company has submitted health and safety studies in which it has a commercial interest in protecting.
Commenter (6376) commends the EPA for understanding the need to blind CBI in the proposal. The commenter states that the ability to shield CBI is essential for the business community to share relevant scientific studies under the new proposal. The commenter adds that similar measures should be provided for other sensitive data or information from studies unrelated to CBI. 
Response: This rule does not require the public disclosure of CBI, nor does it include any requirements for the use or management of CBI data. CBI will be handled in accordance with applicable statutes and regulations.
Comment: Commenters (0559, 6116, 6130, 6133, 6147, 6920) generally support increased access to CBI. Commenter (0559) states that responsible authorities, including state agencies, the EPA, and independent scientific reviewers should be able to request and have access to CBI when necessary to evaluate hazards due to chemical exposures. Commenter (6130) states that refinement of EPA dose-response models would be more likely accomplished by increased disclosure of CBI, not by the increased protection of CBI and decreased availability of scientific information. Commenter (6147) asserts that the codes for chemical formulations which are patented or are protected by CBI should be open to scrutiny by scientists and the community. 
Commenters (6116, 6133, 6446, 6920, 6922) express concern that many studies performed using data from industrial sources would be excluded because they contain CBI. Commenter (6920) states that the proposed rule establishes increased protections for CBI, diminishing the amount of information available to the public to inform policy, whether from the scientific community or from the business community. The commenter asserts that these proposed provisions reduce rather than increase public transparency.
Commenter (6133) states that industry stakeholders may submit studies, data or information for which CBI redactions would prevent EPA from considering those materials, because the information is not "publicly available in a manner sufficient for independent validation." According to the commenter, this could prevent EPA from adopting standards, exclusions, or other regulatory provisions informed by that information. Commenter (6446) states that EPA has long recognized that, while CBI submitted to EPA cannot be shared with the public, the Agency routinely relies on that information, including scientific studies and other scientific information, as the foundation for important regulatory decisions. 
Commenter (6133) states that other industry stakeholders opposed to the appeals and demands sought by the first set of stakeholders, would be harmed if EPA nonetheless considers the latter industry's submissions, notwithstanding redacted CBI that is not "publicly available in a manner sufficient for independent validation" -- while at the same time EPA refuses to consider confidential non-business information submitted by the opponent-stakeholders. 
Response: This rule does not require the public disclosure of CBI, nor does it include any requirements for the use or management of CBI data. CBI will be handled in accordance with applicable statutes and regulations.
Comment: Commenter (6922) states that any rule should continue to protect CBI. The commenter asserts that, because the proposal risks disclosure of CBI, the proposal could discourage industry and research institutions from working with EPA and contributing information if the Agency cannot ensure that intellectual property, trade secrets, and customer data would be protected from disclosure. According to the commenter, if companies are reluctant to submit CBI to EPA, the Agency's rules will not be based on the best available data. 
Response: This rule does not require the public disclosure of CBI, nor does it include any requirements for the use or management of CBI data. CBI will be handled in accordance with applicable statutes and regulations.
Balancing Transparency and Handling of CBI
Comment: Commenter (6143) states that TSCA itself recognizes that specific, limited classes of information should remain confidential, even if they play a role in decisions that the Agency makes pursuant to TSCA. The commenter asserts that EPA should ensure that any final action it takes on the proposed rule respects this balance, encouraging transparency while making limited exceptions to accommodate legitimate, protected privacy interests such as CBI. 
Commenter (6366) states that there is a need to address consideration of private and confidential information that is contained within the pivotal regulatory science being used. The commenter contends that a final rule can be crafted that balances the responsibility EPA has under existing statutes for private and confidential information. 
Response: This rule does not require the public disclosure of CBI, nor does it include any requirements for the use or management of CBI data. CBI will be handled in accordance with applicable statutes and regulations.
De-Identification Procedures and the Potential for Re-identification
De-identification Procedures Have Limitations
Comment: Commenters (0569, 0100, 1008, 1346, 1420, 1461, 1476, 1481, 2171, 2477, 2548, 2687, 4542, 4872, 5019, 5176, 6111, 6125, 6137, 6153, 6125, 6158, 6170, 6446, 6772, 6868, 6873, 6877, 6909, 6924, 6932, 8281-PH20, 8281-PH46, 9227) are generally opposed to the proposed disclosure provisions because de-identification procedures to anonymize personal health data and information can be burdensome and may not sufficiently mask personal identifiable information. 
Commenter (6111) states that EPA's suggestion in the proposed rule that "simple data masking, coding, and de-identification" will be able to overcome these confidentiality concerns is incorrect. The commenter asserts that EPA's SAB states "[i]n some cases, the data cannot be released simply by redacting portions of it. For example, data may have been collected with an assurance to the participating individuals that their data would be kept confidential."
Commenter (9227) asserts that EPA ignores the real concerns, based in empirical evidence, about reidentification of individuals through cross linking with existing public datasets and the ensuing breach of privacy. The commenter states that experts have observed that even the disclosure of redacted or "de-identified" data sets have become more fraught as public health studies have become more rigorous, because these studies are relying upon greater quantities of ever more granular personal information. 
Response: This rule does not require the disclosure of PII or mandate any procedures on the use or management of PII.
Comment: Commenter (6125) asserts that the proposal's unsecured promissory note that it will "make all reasonable efforts to explore methodologies, technologies, and institutional arrangements" for making data available and preserving privacy both fails to say how and fails to acknowledge the extreme difficulties in doing so. Commenter states that, by ignoring all the issues raised (including in the very sources it cites), the agency arbitrarily ignores key issues and record evidence. ​State Farm, 463 U.S. 43.
Response: This rule does not require the disclosure of PII or mandate any procedures on the use or management of PII.
Comment: Commenters (0100, 6873, 6896) explain there are barriers with respect to older studies. 
Commenter (0100) states that each IRB committee usually specifies a fixed time period after completion of the study at which the researcher must destroy his or her copies of the raw data and, thus, would no longer have it available to supply to the EPA after this fixed destruction date. Commenter (6896) states that many data sharing and informed consent agreements include clauses for destroying or returning raw data sets after a study's completion, and so data sharing may not be possible. Commenter (6873) states that, for older studies, improving data access to the degree necessary to pass the openness requirements of this proposed rule could end up being almost impossible given that consent forms written 10-20 years ago were largely silent on the issue of broad sharing of data. The commenter adds that IRBs cannot release private data without consent and getting this consent from study participants where the personal information has long since retired and participants have either died or moved multiple times will be extremely challenging and costly at best, and effectively impossible at worst, therefore possibly rendering "old" studies unusable. Commenter (6896) states that data confidentiality laws have changed to increase protections for research subjects throughout the years and older datasets may contain data that would be considered illegal under current laws. The commenter states that requirements that subjects be notified on how their personally identifiable information is used and shared may prove so cumbersome as to make data sharing impossible for older datasets. The commenter adds that the individuals may be no longer living or be reachable through the contact information originally provided to researchers. 
Commenter (6915) asserts that the proposal's suggestion that concerns about access to confidential or private data can simply be addressed through the application of tools used by other federal agencies, id. at 18,770-71, will be unworkable or impracticable for many past and even future studies. For example, the commenter provides that the proposal cites to guidance regarding methods to de-identify protected health information under the privacy rules of the 1996 HIPAA, Pub. L. No. 104-191, 110 Stat. 1936. Id. at 18,771 n.17. The commenter states that the cited guidance document is 28 pages, contains detailed instructions for de-identification, including how experts are to assess the risk of identification of information, and emphasizes the importance of data-sorting systems to manage protected health information for the de-identification process, including use of a one-way cryptographic function to obscure personally identifiable information. Among other things, the commenter states that the guidance provides that various identifiers of individuals -- including all names, geographic subdivisions smaller than a state (with one exception), dates directly related to an individual, telephone numbers, biometric identifiers, and so forth -- must be de-identified. HIPAA Guidance at 4-5. According to the commenter, the guidance thus highlights that, in fact, it is not easy to address confidentiality concerns: the de-identification process is complex and must be designed into the overall study process, something that cannot be done for historic studies. Moreover, the commenter asserts that, to the extent that one of the purposes of the proposal is to enable persons to replicate studies, this may not be possible where the de-identified data is critical to the studies' findings and conclusions.
Response: This rule does not require that dose-response data that contains privacy or other confidential data be made publicly available. It does not mandate any procedures on the use or management of PII. Dose-response data that contains privacy data or any other confidential data would be considered available for independent validation if it were available through restricted access consistent with all privacy laws.
Possibility of Re-identification
Comment: Commenters (1973, 2171, 2548, 2906, 6125, 6446, 6868) assert that data masking, coding and de‐identification techniques have limitations because re‐identification of participants is still possible. 
Commenter (2548) states that data transparency and data sharing are not always viable options when researchers study human beings and must estimate exposures as a function of location and time. Commenter (5170) states that when individual characteristics are combined by linking datasets (for example, death records, age, sex, and exposure information), individuals can be identified. Commenter (6133) states that research has documented that de-identification techniques to render data anonymous are not "simple" as the proposal characterizes and can lead to the publication of protected confidential or private data. Commenter (4542) states that re‐identification of participants is also made possible by using external information available online and through other sources. Commenter (6909) states that even when "direct identifiers" like names, addresses, and birth dates are removed, "the data values themselves may possess some degree of subject specificity or uniqueness" that could potentially be used to identify individual study participants. Commenter (6133) asserts that, with the shift away from paper to electronic medical records in recent decades, "the potential for individuals to access, use, and disclose sensitive personal health data" has increased.
Commenters (2477, 3135) suggest that it is not always possible to de-identify the data set. Commenter (2477) states that Deoxyribonucleic Acid (DNA) samples/data can be delinked from identity, but the ability to re-link that later is inherent in DNA. Commenter (3135) states that the geneticist Craig Ventner and colleagues reported the ability to identify persons using their genetic code alone (without needing to do a DNA match).
Commenters (2171, 6446) specifically state that data masking, coding, and other procedures suggested to protect privacy, particularly in the context of epidemiological research, would not work. Commenter (6446) explains that even when medical data is masked or coded, subjects can be identified if the sample size is small enough, the characteristics described are rare enough, and/or the data includes, for example, subjects family structure, geographic location, dates of birth, sex/gender, medical conditions, occupations, and/or dates and causes of death. Commenter (2171) states that data availability, as proposed under this rule, would disclose information such as geocoded latitude/longitude residential locations; demographic covariates; and repeated measures in longitudinal population-based studies. According to the commenter, if made publicly available, as required by the proposed rule, the specificity of these non-private covariates would facilitate, with relative ease, the re-identification of individuals and their protected health information. The commenter states that it is this very specificity in geographic, demographic, and health information that contribute to epidemiological studies with the most robust quantification of dose-response and uncertainty relationships. 
Commenter (1973) states that the arguments that data can be redacted to be non-identifiable are belied by modern data science. Re-identification can easily happen. The commenter provides the following examples.
      Consider the Harvard Six City study. To get good exposure estimates participants were recruited not across the cities involved but from one neighborhood in each city. In Boston that neighborhood was Watertown with a population of about 35,000. From data obtainable from the MA department of Public Health, the average number of deaths per year in Watertown is 208. Applying the average U.S. mortality rate per 1000 people to the population of Watertown would give very similar results. This is less than one death per day. Obviously knowing the date of death alone would uniquely identify most of the participants. But even if the data made public were restricted to providing only the year of death (which is required to replicate the analysis or do any reasonable analysis of the effect of air pollution on annual mortality) the analysis of such data also requires knowing the age, race, sex, and cause of death of each individual. Again, with only 200 deaths in a year, it is likely that most people can be identified from knowing those facts. After all, that is only 100 deaths per year for each sex, and those deaths occur across a range of at least 40 ages at death, and many different causes. Combining this with publicly available obituary data, or publicly available death certificate data, would complete the identification of each person.
      The Medicare cohort which followed over 60 million Medicare participants might seem less susceptible to this issue. But consider that exposure was assigned to people by Zip code. From the data provided in the paper, the average number of deaths per year in a Zip code was only 23. If one knew the age, race, and sex, age at entry into Medicare (people who work after 65 often enter at a later age), year of death, and Zip code of those people, it would not be difficult to identify them. Since all of that data except the Zip code would have to be made public under EPA's proposal, the question of whether the data are sufficiently de-identified becomes how difficult would it be to identify the Zip code of the participants, given all the information used in the analysis, which EPA would like to make public? Since everyone in a given Zip code was assigned the same exposure and same covariates in the same year, it is easy to group observations into Zip codes, even without their identity being known. After this, can they be identified? The data that was used in the analysis of the Medicare cohort included region of the country. So, the matching is already made easier by knowing which of 10 regions each Zip code is in. Then, for each Zip code, the annual average of ozone and PM2.5 is part of the analysis data. There is considerable variability in both of these variables, and, moreover, there are public maps showing how they vary across the U.S., including in the published paper of Di and coworkers. EPA also has ~1800 monitoring sites to provide guidance on what levels prevail where, in what year. Using those, it should be easy to assign each Zip code to a sub region that corresponds to that range of PM2.5 and ozone. But over 13 years, these exposures changed differently over time in different Zip codes, and since annual analyses were done in the paper, that data would have to be available under the proposed EPA rules as well. Looking at levels and trends can assign the unknown Zip codes to much smaller sub regions. Then consider that the analysis controlled for, and therefore under EPA's proposal, should make public, percent of the Zip code that is age 65 or older, is black, is Hispanic, percent of people 65 and older living below the poverty level, or not having completed high school, percent of owner occupied housing in the Zip code, population density in the Zip code, median house value in the Zip code, percent of owner occupied housing in the Zip code, and the annual average temperature in the Zip code. All of these data are, of course publicly available from the U.S. Census with the Zip codes identified. How hard is it then to match? Consider median housing value. This varies greatly in the U.S., and even between nearby neighborhoods. Restricted to a sub-region, this provides a lot of information on where a Zip code could possibly be. So does mean annual temperature, and all of the other variables listed. The analysis data includes additional variables that were controlled for, including county level percent smokers and mean body mass index, and percent of people age 65 and over who had hemoglobin A1c and LDL cholesterol tested each year, and percent who had an annual checkup, by hospital service area. All of this data is publicly available with the location identified. How hard would it be to match? Indeed, consider the simple calculation that if each of the 15 area level variables were classified as high vs low, there would be 32,768 unique combinations in each region which contains only about 3900 Zip codes. And, of course those variables are continuous, giving many more unique combinations. And this is without the ozone and PM2.5 concentrations. Given all this information, we believe that most and likely all of the individual Zip codes, and hence most of the individuals who died, would be identifiable. About a third of the Medicare participants died during follow-up in that study. This would certainly violate rules of the Center for Medicare and Medicaid Services for use of their data, as well as general prevailing standards of individual privacy.
Commenters (1973, 6111, 6125, 6133, 6137, 6874) reference several other studies where researchers have demonstrated the potential for "re-identification" of data. 
Commenters (1973, 6111) reference a recent peer-reviewed study which examined the identifiability of records from an environmental health study in Northern California. Commenter (1973) states that, using data considered by HIPAA to be sufficiently de-identified to be made public, which involved far fewer variables than would be required to make public in the cohort studies, they were able to correctly identify over 25% of the participants. Commenter (6111) concluded from this study that even if EPA or the researchers spend money and time to de-identify data to comply with the proposed rule, that effort will not necessarily protect patient or research subject confidentiality. 
Commenters (1973, 6874) reference another study which searched the Lexis-Nexis database for stories that mentioned hospitalization. Commenter (1973) states that by matching that with age, race, sex and Zip code from a supposedly anonymized hospital admissions data base they were able to match 43% of the people named in the news stories to their medical records. Commenter (1973) states that, since large numbers of obituaries are printed every day, the numbers that could be identified by such a process is even larger. Commenter (1973) states that is before considering the other data available to help matching. 
Commenters (6133, 6874) reference a study which examined the identifiability of records. Commenter (6133) states the study explains that "any data that is even minutely useful can never be perfectly anonymous." Commenter (6874) states that the study has demonstrated the potential of "re-identification" techniques, which combine seemingly innocuous data from multiple publicly available sources, to identify individuals.
Commenters (6125, 6133) refer to a study that explained "87% (216 million of 248 million) of the population in the United States had reported characteristics that likely made them unique based only on [5-digit ZIP, gender, date of birth]." Commenter (6125) states that the level of detail in the data supporting many of these studies precludes simply redacting portions of the information to protect privacy as suggested in the proposal. 
Commenter (6125) states that, as Herring notes, the better-characterized the cohort in studies (e.g., personal information on potentially confounding contexts and behaviors, well-characterized exposures, clinically-confirmed outcomes), the more limitations on data sharing due to federally-mandated personal privacy protections (Common Rule, HIPAA Privacy Rule, informed consent restrictions). 
Commenter (6133) states that anonymizing data (through techniques such as data masking, coding, and de-identification techniques) might not adequately protect confidentiality or privacy. The commenter contends that various studies have documented that de-identification techniques render data anonymous is not "simple," despite what the proposal suggests, and can lead to the publication of protected confidential or private data. According to the commenter, that one study explained that "[b]y linking demographics to public records such as voter lists, and mining for names hidden in attached documents, we correctly identified 84 to 97 percent of the profiles for which we provided names." 
Commenter (6137) states that de-identifying personal information has thus far proven to be ineffective. The commenter states that, under the Privacy Rule, the entity may remove multiple enumerated categories of information, including patient names, social security numbers, full face photographs, and biometric identifiers (such as fingerprints). Id. § 164.514(b)(2). The commenter explains that an entity following this approach must generalize each patient's birth date to the relevant year and may include only the first three digits of a patient's zip code. Id. § 164.514(b)(2)(i)(B), (C). According to the commenter, despite these seemingly thorough requirements for de-identification, the Privacy Rule is significantly less protective than it appears. The commenter states that, in the years since its adoption, publicly available personal information has proliferated, and new databases are created every day. The commenter contends that to "reidentify" de-identified data, an adversary need only discover an individual's "data fingerprint" -- that is, the combination of values shared by nobody else in an anonymized data set. The commenter states that the adversary can then link this fingerprint to publicly available, non-anonymized information to discover the individual's identity. 
Commenter (6909) states that the NAS, Engineering, and Medicine (NASEM) have discussed this problem in a recent report, agreeing that "it has become clear that even anonymized data can reveal private information about the human subjects." The commenter states that as the NASEM Report went on to explain, there is a significant challenge presented by the fact that "even attributes that are not labeled as personally identifiable may still contain sensitive information that associates an individual, and that by linking those data to other publicly available resources, individuals can be re-identified." According to the commenter, a recent case study cited by NASEM which looked at a dataset containing 2.8 million hospital records showed that "even after removing from the dataset all information except the procedures received by a patient, the percentage of patients with a unique set of procedures is still 42.8 percent; in other words, as the investigators state, `an adversary would have about a 42.8 percent chance of linking the anesthesia record to the hospital database, thereby discovering the patient's sensitive information.'" The commenter states that "anonymizing" techniques cannot guarantee a prospective study participant that his or her confidential patient information will in fact remain confidential.
Commenter (6137) states that adversaries need not resort to such sophisticated methods. The commenter states that, as the Seventh Circuit explained in the context of medical records relating to abortion, once a patients' de-identified records are made available, "persons of their acquaintance, or skillful `Googlers,' sifting the information contained in the medical records concerning each patient's medical and sex history, will put two and two together, `out' the . . . women, and thereby expose them to threats, humiliation, and obloquy."
Response: This rule does not require that dose-response data that contains privacy or other confidential data be made publicly available. It does not mandate any procedures on the use or management of PII. Dose-response data that contains privacy data or any other confidential data would be considered available for independent validation if it were available through restricted access consistent with all privacy laws.
Relationship to Existing Data Sharing Practices
Comment: Commenters (6117, 6125, 6170) expresses concern that the proposal does not align with existing data sharing policies
Commenter (6170) asserts that IRBs do not allow for such data to be made publicly available i.e. in any form that may allow one to identify the research subject. The commenter states that this is often impossible to achieve since researchers need geographic identifiers (geocoded addresses) to go along with the health data in order to identify exposures. According to the commenter, even so called de-identified human data would have to be made available at a level of detail for replication studies - as proposed by this rule  -  such that it would allow for individuals to be identified especially for rarer diseases such as some cancers, autism and Parkinson's disease. 
Commenter (6125) states that HHS's guidance on de-identification of protected health information provides safe harbor as an acceptable method, requiring that certain identifiers are removed. The commenter states, however, that it is unclear whether this method would be adequate/suitable to de-identify information from data relied upon by EPA. Commenter (6868) expresses that, while the proposed rule cites guidance from the Department of Health and Human Services on proper techniques for deidentifying protected patient data, the very webpage EPA cites includes the following text about the risks of data de-identification: "Both methods, even when properly applied, yield de-identified data that retains some risk of identification. Although the risk is very small, it is not zero, and there is a possibility that de-identified data could be linked back to the identity of the patient to which it corresponds." According to the commenter, even sources that EPA cites in support of de-identification of data recognize that patients' protected information could still be breached. 
Commenter (6117) refers to the following policies as not aligning with EPA's proposal: 
 NIH policies, including the policy that researchers conducting agency-funded research pre-register clinical trials, pre-specify study endpoints and statistical plans, and report research results. 
 Many federal agencies, including the EPA, require that any published articles based on federally-funded research be made available via the PubMed Central (PMC) website.
 The International Committee of Medical Journal Editors (ICMJE) has adopted a policy requiring pre-registration of clinical trials, and publication of a data-sharing statement.
 In 2015, the Institute of Medicine (IOM) published a report recommending that trials be registered prospectively (i.e., at or before the time of first patient enrollment), that summary-level results be shared with the public after trial completion, and that the metadata and patient-level data be responsibly shared with researchers 6 months after scientific publication, or 30 days after regulatory approval.
Response: This rule does not prescribe any data-sharing methods. The rule simply requires that EPA give greater consideration to pivotal science where the underlying dose-response data are available in a manner sufficient for independent validation.
Comment: Commenters (1973, 4542, 6125, 6909, 6924, 9227) refer to work of the National Academy of Sciences (NAS) and assert that de-identification procedures have significant limitations.
Commenters (6117, 6125, 6133) assert that the proposal's reference to NIH data sharing policy does not acknowledge certain limitations.
Commenter (6125) states that the proposal's reliance on NIH is misplaced. The commenter states that NIH policies regarding data sharing generally recognize the need for restricting access to protect privacy or CBI and note that in some cases access may need to be restricted to data enclaves or other limited sharing agreements available only to qualified investigators. Commenters (6117, 6125) state that NIH requires a data sharing plan for extramural research projects requesting $500,000 or more in a single year and some institutes of the NIH share data with researchers via a secure website. Commenter (6125) asserts, however, that this requirement is not absolute, since NIH will also accept an explanation for why data sharing is not possible. The commenter asserts that, under the proposal, EPA would not accept such an explanation as anything but a basis for a waiver request, either for its own science or for NIH's. The commenter states that, for research with such plans, NIH funds the costs of necessary data and methodology sharing arrangements as part of the project. The commenter states that EPA's proposal provides no such funding and does not even acknowledge that it may be needed. 
Commenter (6133) states that the NIH policy cited in footnote 21 of the proposal states that "[t]he investigator must be a tenure-track professor, senior scientist, or equivalent, to be able to submit" a data access request. The commenter contends that this fatally undermines the notion in the proposal that data must be available to all members of the public in order to meet the reproducibility threshold. The commenter states that the Census Bureau resource, also cited in footnote 21 of the proposal, describes the Federal Research Data Centers. According to the commenter, these centers restrict access to certain individuals, who "must obtain Census Bureau Special Sworn Status  -  passing a moderate risk background check and swearing to protect respondent confidentiality for life, facing significant financial and legal penalties under Title 13 and Title 26 for failure to do so." The commenter states that, again, this fatally undermines the notion in the proposal that data must be available to all members of the public. According to the commenter, while the proposal simply says that members of the "public" can access these centers, the reality is that access to such controlled spaces is carefully restricted and not accessible to all members of the public. The commenter contends that EPA does not seriously confront the significant challenges involved in enabling access. 
Response: This rule does not prescribe any data-sharing methods. The rule simply requires that EPA give greater consideration to pivotal science where the underlying dose-response data are available in a manner sufficient for independent validation.
Comment: Commenter (2906) states that de-identification of study subjects remains a difficult, costly, and time-consuming process. Particularly where study data intersects with consumer data collected by data brokers, de-identification may not be possible, which could dramatically limit the amount of data available to EPA regulatory officials.
Response: This rule does not require the de-identification of study subjects or the public disclosure of personPII. This rule does not prescribe any data-sharing methods. The rule simply requires that EPA give greater consideration to pivotal science where the underlying dose-response data are available in a manner sufficient for independent validation. For PII, CBI and other sensitive data, the dose-response data can be made available for independent validation through restricted or tiered access.
Comment: Commenters (6125, 6133) note that the proposal cites to various agencies and documents to support its general statement "that concerns about access to confidential or private information can, in many cases, be addressed through the application of solutions commonly in use across some parts of the Federal government." The commenters state that the proposal includes references to examples from HHS, National Institute of Standards and Technology, U.S. Department of Education, the U.S. Census Bureau., and the Bipartisan Commission on Evidence Based Policy. According to the commenters, the documents cited in the proposal do not support the position taken in the proposal. 
Response: These documents are consistent with the restricted or tiered-access approach proposed in the 2020 SNPRM and finalized in today's rule.
Lack of Discussion of Implementation in the Proposal 
Comment Commenters (2241, 3135, 6116, 6125, 6133, 6188, 6896, 6924) contend that EPA has not identified the specifics of how it will implement the rule. Commenter (6133) asserts that the proposal does not propose or analyze any strategies it notes EPA should consider, even though it seeks to implement a binding legal change. Commenter (2241) states that the proposed rule has the following critical shortcoming that disqualifies it as regulation: it fails to explain how PII, such as health information, can be used in the rulemaking process. The commenter states that, without clear guidelines on the use of PII, such data may be excluded from regulatory analyses, which could lead to epidemiologically biased data sets and incorrect conclusions.
Commenters (3135, 6116, 6924) express concern that the proposed rule does not address who would assume potential liability if confidential information is disclosed to comply with the rule. Commenter (3135) states that, for human studies, to manage potential risks of disclosure of sensitive human data, the EPA would not be able to rely solely on data submitters to have de-identified the data but, to avoid liability, would have to perform checks to assure that EPA would not inadvertently disclose any personal health information. Commenter (6116) expresses concern that, under federal law, there are criminal penalties for disclosing CBI  -  a serious risk of liability for EPA that the agency does not address except to say generically that all such laws will be followed, without demonstrating how compliance will be achieved in tandem with making research data publicly available.
Response: This rule does not require the disclosure of confidential information.
Comment: Commenters (6125, 6133) state that the proposal does not explain what would happen if an available de-identifying method does not meet the applicable standards. 
Response: This rule provides for independent validation of dose-response data that contains PII through restricted access. Thus, the dose-response data do not have to be anonymized.
Comment: Commenter (6133) states that the proposal does not identify a plan for EPA to manage mandatory agreements to maintain confidentiality, data encryption, electronic firewalls and locked storage facilities, password authentication of users, audit trails, disaster prevention and recovery plans, or security measures for backup tapes. The commenter states that the NIH Data Sharing Workbook specifies that "[s]ome investigators withhold parts of the sample; others block access to specific variables, especially items with low prevalence rates that make it easier to identify participants with unusual characteristics." The commenter states that, within this policy, the "measures used to minimize the risk of breaching the confidentiality of data" are unworkable given the depth and breadth of peer-reviewed research that would fall under the rule. 
Response: EPA will handle this consistently with how it currently manages PII, CBI and other sensitive data.
Comment: Commenter (6896) expresses concern that the proposal does not recognize or address the extensive efforts needed for data to be shared with the EPA and maintain confidentiality of sensitive health and employment information. The commenter states that the appropriate IRB would have to be consulted on the approach to de-identification and each individual record would have to be personally reviewed by a subject expert and appropriately de-identified. According to the commenter, for some datasets, multiple IRBs would have to be consulted. The commenter asserts that the requirements and dedication to protecting research participants who served our country in Department of Energy (DOE) complexes would result in countless hours spent notifying all participants, gaining informed consent, and de-identifying each individual record before underlying data could be shared with EPA. The commenter states that the Building Trades National Medical Screening Program (BTMed) dataset characterizes the strenuous challenges to protecting sensitive worker information. 
Response: This rule provides for independent validation of dose-response data that contains PII through restricted access. Thus, the dose-response data do not have to be anonymized.
Comment: Commenter (6188) states that the rule's scant text entirely omits a discussion of privacy protections or how EPA would navigate those protections to adequately shield the identities of study participants while still providing "the information necessary for the public to understand, assess, and replicate findings." According to the commenter, the proposal amounts to a proposal to shirk EPA's statutory obligations and ignore patient privacy laws administered by other agencies. The commenter states that the proposal only says that data will be made publicly available "in a fashion that is consistent with law, protects privacy, confidentiality, CBI, and is sensitive to national and homeland security." According to the commenter, it is entirely unclear how EPA means to satisfy this direction, which is diametrically opposed to the proposal's requirement that EPA make an unprecedented amount of protected scientific data publicly available.
Commenter (6133) states that the proposal refers to a statement of the National Academies that "[n]othing in the past suggests that increasing access to research data without damage to privacy and confidentiality rights is beyond scientific reach." According to the commenter, this is not fully supported as experience shows increasing access to data can damage privacy and confidentiality rights. The commenter asserts that the proposal does not say which, if any of these techniques the EPA will use, or how the EPA will use them. The commenter states that, while the National Academies may believe that increasing access to data without damage to privacy and confidentiality is not beyond scientific reach, the proposal does not explain how this belief translates to past, present, and future scientific studies EPA considers in regulatory decision making. According to the commenter, this document does not explain how EPA will address concerns about confidential or private information and does not support EPA's proposal to preclude consideration of those studies that do not make public underlying data for those, or other reasons.
Commenter (6133) also states that the proposal suggests that EPA should identify strategies in the future and that these strategies should be cost-effective. The commenter asserts that the proposal does not say what cost-effective means, nor what EPA should do if it does not identify any cost-effective strategies, yet it still seeks to alter legal obligations and regulatory decision making in reliance on this unexplained suggestion. The commenter also states that the EPA also does not point to any authority for the proposition that the agency's consideration of peer reviewed scientific studies depends on the cost-effectiveness of some strategy the agency develops for publicizing and protecting the underlying data. According to the commenter, the proposal suggests that it will exclude a large class of scientific studies from regulatory decision making but contains a vague assertion that it will look for "cost effective" ways in the future to exclude less them.
Response: These comments relate to proposed 40 CFR 30.8 which was not finalized.
Other Data Disclosure Issues
Proposal Regarding How Privacy Data/CBI is Handled Could Bias Study Selection
Comment: Commenters (2171, 4410) assert that it is well-documented that privacy assurances are essential to mitigate nonrandom selection bias. Commenter (8817) asserts that studies done on human subjects where the subjects' privacy is not protected may produce false or biased results. Commenters (6171, 6898) state that the inability of a scientist to guarantee confidentiality would be a major barrier to being able to recruit an appropriately diverse population of participants, potentially causing significant bias in research findings. Commenter (2171) states that the proposed rule does not make clear how it would assess or address bias or systematic errors in population studies that were able to release unmasked analytic data to the public. 
Response: The rule does not require the disclosure of sensitive data, including PII and CBI. The rule requires that the Agency give greater consideration to pivotal science based on dose-response data that include CBI, proprietary information or PII if these data are available through restricted access in a manner sufficient for independent validation. Thus, there should not be the bias suggested by the commenter.
Comment: Commenter (2548) states that exposure estimates based on highly-aggregated data typically lead to biased and flawed analyses upon which decisions cannot be reliably based. 
Response: The requirements of this rule will not lead to the generation of highly-aggregated data as suggested by these comments on the 2018 proposed rule. Thus, there should not be the bias suggested by the commenter.
Comment: Commenter (4590, 4595) expresses concern that an incomplete pool/smaller pool of scientific evidence creates an inherent bias that could skew regulatory decision-making toward studies that are less rigorous and trustworthy. 
Response: No study will be categorically excluded. Thus, there should not be an incomplete pool/smaller pool of scientific evidence and no resulting bias.
Transparency Proposal Creates a Double Standard Between How Privacy Data and CBI Data Would Be Handled
Comment: Commenters (0044, 1012, 1388, 2907, 5170, 6109, 6133, 6152, 6157, 6895, 6919) express concern that the proposed rule would require independent public and private institutions to release confidential data behind environmental health studies while at the same time seeking to exempt private industry from being required to do the same. Commenter (6133) asserts that, this would result in EPA not being able to rely on the best available science, while industry-funded research would not be subject to similar data release requirements, or even peer-review and independent reevaluation. Commenter (6133) states that this approach is asymmetric and favors selective, opaque, and questionable research methods over the consensus of robust peer-reviewed scientific investigation. 
Response: The rule does not require the disclosure of sensitive data, including PII and CBI. The rule requires that the Agency give greater consideration to pivotal science based on dose-response data that include CBI, proprietary information or PII if these data are available through restricted access in a manner sufficient for independent validation.
Comment: Commenters (1012, 2907, 5170, 6109, 6919) state that, unlike the Proposal does for CBI, it does not seek comments on how to balance patient privacy or confidentiality with requirements for increased transparency, which is also protected by federal law. The commenters state that this omission makes it seem that protections for CBI are being given higher priority by the EPA over protections for personal health information. According to the commenters, prioritizing the confidentiality of private entities could lead to the publication or release of data that is supported by industry and could bias the usable body of evidence for the EPA towards a particular industry position. Further, commenter (6109) requests that EPA clarify what is meant by the phrase "balance appropriate protections" for CBI and why EPA has not used the same phrase for protecting private health data? 
Commenter (6895) states that, given that the confidential data held by independent institutions are also protected by law, EPA should be seeking the same transparency privilege as that afforded the private sector. According to the commenter, otherwise, this rule would grant authoritative power to the regulated industry to direct public and environmental health regulation since much of the private sector's data is not "publicly available in a manner sufficient for independent validation." Similarly, commenter (6152) states that, while they support protections for justifiable claims of CBI, any requirements that EPA puts on research must be evenly applied. 
Response: The rule does not require the disclosure of sensitive data, including PII and CBI. The rule requires that the Agency give greater consideration to pivotal science based on dose-response data that include CBI, proprietary information or PII if these data are available through restricted access in a manner sufficient for independent validation.
Comment: Commenter (2012) states that that the exclusion of available data on any basis that is not statistically justified can result in a biased perspective and, consequently, ill-informed policy. Commenter states that EPA should promote rules for increased data transparency across their fellow governmental agencies, while still recognizing the reality of the system in which they operate, and the need for as much data as possible in creating any public policy that governs human health. Commenter states that when considering any question of interest pertaining to the EPA's focus on the dose-response relationship of potential pollutants, regulators should consider the proportion of data and studies that are available exclusively through the new proposed method of "publicly available data" - as opposed to the data that would be available if incorporating non-public data. Commenter states that this partition is to be considered after employing any of the tools mentioned in the proposal - data masking, de-identification, etc. - to include as much data as possible under this heading. Commenter states that, if this difference is anything more than obviously negligible then the agency should opt to include all sources of data, at the expense of using only data that can be made publicly available.
Response: The rule does not require a categorical exclusion of any study based on data availability. The rule incrementally increases transparency by requiring that EPA give greater consideration to pivotal science where the underlying dose-response data are available in a manner sufficient for independent validation.
4.1.1.1.3 Competing Models, Linear, Threshold, U-Shaped, J-Shaped, Bell-Shaped
Default Linear Non-Threshold (LNT) Model 
Model Selection Objectivity 
Comment: Commenter (4881) contends that research funded by industry and research funded by federal agencies often disagree on evidence for non-linearity, with industry generally preferring models that incorporate a threshold below which the pollutants are harmless, while independent research often finds harmful effects even at low doses. The commenter asserts that who makes the decisions which studies have given "appropriate consideration to...a broad class...of models" of dose response becomes critical in guaranteeing balanced consideration of public health risks and industry concerns.
Response: This rule does not apply to models, including dose-response models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Support Proposal to no Longer Use the LNT as the Default Model
Comment: Commenters (1010, 1273, 4541, 6148, 6151, 6362, 6449, 6942) support EPA's Proposal to no longer use the LNT as the default model in cancer risk assessment and agree with EPA's contention that there is growing empirical evidence of non-linearity for the concentration response for specific pollutants and health effects. Commenter (1273) asserts that, until now, the EPA has enshrined the LNT as the basis for its regulations, ignoring the overwhelming evidence of biological defense mechanisms indicating thresholds and/or hormetic dose-responses. Commenter (4541) suggests that the EPA has had a troubled history with scientific models as a general matter. The continued use of default models, without consideration of alternatives or admission of model uncertainty, creates a false scientific justification for EPA actions, policies, and regulatory burdens.
Response: This rule does not apply to models, including dose-response models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
IRIS Assessment 
Comment: Commenter (6369) asserts that an example where an IRIS assessment has led to EPA toxicity values below natural background levels and not reflective of real-world human exposures is for formaldehyde. The commenter states that the published literature provides considerable evidence of observed thresholds for effects from formaldehyde exposure.,,. Commenter (6911) also expresses concern that EPA's draft risk assessment on this chemical will result in a "safe dose" determination that would likely be incomprehensible below background level.
Commenters (6369, 6911) state that other international agencies have also recognized alternative models other than the default linear modeling approach for formaldehyde exposure. The commenters conclude that the importance of using non-linear or biologically-based dose-response modeling, when the published data supports it, cannot be overstated and further ensures that the most robust science is the foundation for regulatory decisions.
Response: This rule does not apply to models, including dose-response models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data. Further, formaldehyde is not the subject of this rulemaking.
LNT Dose Response Model
Comment: Commenter (0039) expresses that they have long been concerned about politicized distortions of dose-response science. The commenter provides that a notable example is the status of the LNT dose-response model for the biological effects of nuclear radiation. The commenter provides that the prominence of the model stems from the June 29, 1956 Science paper. "Genetic Effects of Atomic Radiation"· authored by the AS Committee on the Biological Effects of Atomic Radiation. The commenter states that this paper is now widely questioned and has been seriously critiqued in many peer-reviewed publications, including two detailed 2015 papers. According to the commenter, these criticisms are being taken seriously around the world. as summarized in a December 2, 2015 Wall Street Journal commentary. The commenter contends that this is a consequential matter that bears on a great deal of national public policy, as the LNT model has served as the basis for risk assessment and risk management of radiation and chemical carcinogens for decades. The commenter asserts that a reassessment of that model could profoundly alter many regulations from the Environmental Protection Agency, the Nuclear Regulatory Commission, and other government agencies. Yet, according to the commenter, due to the complacency of the scientific establishment and, no doubt, the prestige of some of the scientific bodies implicated in the cover-up, no one in a position or authority until now has been willing to challenge the linear no threshold dose-response orthodoxy.
Response: This rule does not apply to models, including dose-response models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Comment: Commenter (0062) states that the use of the LNT as the default model for radiation-induced cancers by the EPA and other regulatory agencies worldwide during the past several decades, in spite of accumulating evidence supporting radiation hormesis, has not served the public well because of the unjustified fear of even small amounts of radiation it has generated, and the resulting expensive and harmful defensive actions. According to the commenter, it has increased regulatory burden on the enterprises that utilize radiation without any corresponding benefit of avoided diseases. The commenter contends that there are indeed many adverse effects due to the use of the LNT model. According to the commenter, this is an example of the harm to the public from the use of default models without supportive evidence.
Response: This rule does not apply to models, including dose-response models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
LNT Zero Tolerance Risk Paradigm
Comment: Commenter (4541) contends that the EPA's campaign to regulate away all risks is doomed to fail since risk is inherently subjective. 
Response: The comments are beyond the scope of this rulemaking.
Oppose Proposal Requirement to Use Alternate Dose-Response Models and Justification for Use of LNT Model
Comment: Commenter (6191) believes that the historic approach that EPA has taken to err on the side of protecting human health when there is insufficient evidence to rule out harm is correct and states that EPA's reference to "considering the breadth of dose response data and models used" would seem to reflect a change to the EPA's traditional practice of opting for a protective stance. Commenter (3840) asserts that the dose-response part of the Proposal is essentially a challenge to the default hypotheses and models used by EPA in cancer risk regulation, as developed over the years, and approved by scientific review bodies. The commenter contends that only believers in alternate dose-response models claim that there exists science to justify a change in long-standing practice-- science that will withstand judicial review.
Commenter (6914) provides that the proposal states "the widespread occurrence of non-linear dose responses in toxicology and epidemiology for chemicals and radiation and the need to incorporate such data in the risk assessment process." According to the commenter, the chemical industry long has attempted to convince regulatory agencies of non-linearity in order to ignore real dose-response relationships and weaken regulatory requirements that protect people. The commenter asserts that this is an attack on science and on credible and peer-reviewed risk assessment methodology. The commenter states that the chemical industry specifically targets low dose exposures and uncertainty factors that are more protective for vulnerable populations, such as workers. (The TSCA law specifically named workers as a "potentially susceptible subpopulation" that deserved specific considerations because of their unique exposure profiles and disproportionate adverse health effects to chemicals.) Commenter (5022) asserts that the pro-industry orientation of the proposal is revealed once more in its assault on the linearity assumption in the dose/concentration-response function. (This approach assumes that there is no safe threshold at the population level for most chemical pollutants.)
Commenter (6117) contends that it is inappropriate to establish major changes in research priorities through insertion of additional language in a proposed regulation ostensibly on another topic, as was done here with regards to dose-response model selection, rather than to use the normal channels of consultation with stakeholders, advisory boards, and reference to prior commissioned studies from the National Academies of Sciences. The commenter asserts that the proposed changes are antithetical to the governing law, existing regulations, and well-established agency practice. 
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Comment: Commenters provide that (1) EPA provides no evidence to justify the proposed requirement that it consider threshold models in dose-response assessments and that there is "growing empirical evidence of non-linearity in the concentration-response function for specific pollutants and health effects" (5022, 6117, 6137, 6144, 6191, 6881, 6924, 9227); (2) the proposed requirements to consider threshold models ignores essential science, namely the approach to dose-response assessment recommended by the National Academy of Sciences (NAS) in Science and Decisions: Advancing Risk Assessment (6117, 6125, 6137, 6158, 6881, 6924, 8281-PH46, 9227); and (3) EPA's proposed requirement disregards the Agency's own Guidelines for Carcinogen Risk Assessment (6125, 6137, 6915). Commenter (6924) adds that EPA's conclusions on linearity vs. non-linearity in dose response are incorrect. 
Commenter (6446) asserts that the proposed approach to non-linear cancer dose-response modeling suggested by the proposed rule is inconsistent with scientific guidance for determining when defaults can be replaced by alternative modeling approaches that are backed by chemical-specific data. The commenter states that EPA cancer modeling generally assumes that no exposure threshold exists for carcinogenesis unless data to the contrary exist. Commenter states that this assumption includes the recognition of the study design of most cancer bioassays, which use a relatively small sample size, resulting in a low detection power. Commenter states that this low detection power often makes it difficult or impossible to determine the existence of a threshold for carcinogenicity for a specific chemical. Commenter states that this current EPA cancer modeling practice was developed in documents that received public comment and peer-review. Commenter states that, in contrast, the proposed rule's requirement for re-justification of established and accepted modeling practices has not undergone any scientific peer review, including peer review by EPA's own SAB, lacks scientific consensus, and is not scientifically justified in the rule.
Commenters (1589, 6137, 8281-PH46, 8771) state that, contrary to the Proposal's statement about the growing evidence of non-linearity in concentration response functions, there is a body of empirical evidence that points to the opposite. Commenter (1589) asserts that there are very few, highly specific cases in which data supports any non-linear relationships between concentration of pollutants and health effects, suggesting that the data are not statistically significant or that they are outliers. According to the commenter, conversely, there is an enormous preponderance of data supporting the linear relationship for a variety of pollutants. Commenter (6137) states, for example, that EPA has found strong empirical evidence of no-threshold concentration-response function for lead and reduced IQ, and particulate matter and increased mortality. Commenter (8281-PH46) contends that for most chemicals and pollutants there is likely no safe threshold on a population level because of ongoing exposures and preexisting vulnerabilities.
Commenter (8771) states that there's growing theory and evidence that at the population level, more and more dose-response functions are linear in the range of interest of actual community and occupational exposures in the real world (Crawford and Wilson 1996; NAS 2009). The commenter asserts that a regulatory agency should be completely uninterested in the behavior of the dose-response curve, threshold or otherwise, below the exposure level that it seeks to regulate to. The commenter provides that the possibility that there may be a "threshold somewhere" in no way implies that there is a threshold in the narrow window between (a) the current level of exposure to the population and (b) the new level EPA or another agency seeks to achieve through regulation. The commenter also states that, depending entirely on where the threshold is, threshold and linear models may well predict the same amount of risk reduction for a regulation, because the dose-response is linear where it counts.
Commenter (9227) provides that, as discussed by NAS, individual risk is determined by both the chemical exposure and an individual's unique circumstance of factors (e.g., co-exposures and susceptibilities). Cancer incidence in the population illustrates the significance of these additional factors in considering actual individual risk to a particular chemical exposure. Individual lifetime risk of developing cancer is 1 in 3, and 1 in 5 for dying from cancer, indicating a substantial population baseline risk resulting from a large number of exposures and other risk factors. The commenter asserts that assuming that there is somehow a threshold for everyone cannot be supported by the evidence. Therefore, given that the mission of EPA is to protect public health, the commenter provides that the linear approach is most appropriate unless there is strong evidence in favor of an alternative as recommended by the NAS in Science and Decisions: Advancing Risk Assessment.
Commenters (6137, 9227) note that, historically, researches applied no-threshold dose response models to carcinogens and threshold models to non-carcinogens. Commenter (6137) provides that because thresholds vary by individuals, for some, effects will have high thresholds, while for others, effects have practically no threshold due to increased susceptibility or background exposures. Based on these findings, the commenters (6137, 9227) provide that the NAS concluded that no-threshold models should be applied to both carcinogens and non-carcinogens unless reliable data/strong evidence affirmatively support a threshold model based on detailed assessments of mode of action, susceptibility, and background exposures. Commenter (6158) states that applying the linear assumption is designed to avoid an arbitrary, case-by-case treatment of potential exceptions without justification to do so. Commenter (6144) is concerned that the Proposal gives unjustified weight to non-linear models to assess relationships between pollutants and heal in EPA policy decisions that will likely disproportionately affect the sensitive populations that the EPA is charged to protect. The commenter provides that for ozone and particulate matter, for example, evidence suggests that health effects (including mortality, cardiovascular effects, and respiratory effects) below the current NAAQS are more pronounced for the elderly, children, low-income individuals, and African Americans.
Commenters (6137, 6915) provide that the Agency's own Guidelines for Carcinogen Risk Assessment expressly state that, in cancer risk assessments, no threshold should be assumed unless the mode of action is known, and the chemical does not cause cancer by inducing DNA mutations that initiate tumor development. Commenter (6915) states that these guidelines state that EPA's default dose response modeling approach for carcinogenic substances is linear extrapolation from the point-of-departure (essentially the lower limit of the range of the experimental data) to the origin (zero exposure, zero risk). (This default no-threshold approach assumes that any dose of a carcinogen results in some level of risk, making it the most protective of human health.) The commenter provides that, threshold models, by contrast, assume that there is some dose of a carcinogen at which there is no cancer risk, an assumption that is less health protective and that has not been conclusively established in most cases. The commenter notes that, in most cancer risk assessments, dose response data within the low risk range, which is the range of interest for regulatory purposes, are lacking. Thus, the commenter provides that low-dose extrapolation is used to estimate risks in the lower dose range where data are unavailable. For estimation of risks below the range of the data, the commenter states there are an infinite number of possible threshold and non-threshold assumptions regarding the shape of the dose response curve that can be envisioned, with no substantive basis for assuming the general superiority of one assumption over another. The commenter contends that, to deviate from the default assumption that any dose of a carcinogen results in some risk in the absence of chemical-specific data that demonstrate a threshold mode-of-action of carcinogenicity would be mere speculation and would assume, with no scientific support, that Americans can be safely exposed to those substances. The commenter adds that, to "evaluate the appropriateness" of the linear, non-threshold approach for low-dose extrapolation by also considering non-linear and threshold models would provide no cognizable benefit in modeling accuracy or clarity, but instead could result in the manipulation of results, delay, and obfuscation. The commenter states that it is unclear what EPA means by "explicit consideration," or what EPA would consider to be "high quality studies," but insofar as those terms are intended to mean that EPA will give preference to studies utilizing threshold models, such a preference would be inconsistent with EPA's 2005 Guidelines for Carcinogen Risk Assessment as well as generally accepted practice in the field of toxicology. Commenter (6137) asserts that EPA's reversal of longstanding policy is unjustified and arbitrary and should not be permitted. 
Commenter (6915) states that, in the limited circumstances where the data support threshold modeling, the EPA's 2005 Guidelines for Carcinogen Risk Assessment already provide for departure from the default linear extrapolation in risk assessment and instead allow for the use of threshold modeling. Id. at A-8. In fact, the commenter asserts that the EPA has used a threshold approach for carcinogen risk assessment when there is clear, chemical-specific, empirical evidence of a threshold mode of action. The commenter notes that this careful, well-founded approach is generally considered both scientifically supportable and protective of public health, as opposed to the proposed rule's requirement for justification of the default linear approach on a case-by-case basis and "explicit consideration [of] high quality studies that explore various threshold models across the dose or exposure range." 83 Fed. Reg. at 18,774. According to the commenter, without a well-founded and substantiated scientific basis, EPA should not entertain such a fundamental departure from accepted, public-health protective risk assessment practices.
Commenter (6125) contends that the Guidelines for Carcinogen Risk Assessment and NAS both encourage careful evaluation of all scientific information in determining the possible shape of dose response curves (based on mechanistic concentrations) as well as the application of a number of mathematical models to one or more specific sets of dose response data to determine which model provides the best fit and prediction. The commenter provides that raw data is not required for these analyses and analyses are done on a case-by-case basis dependent on the nature of the data examined. According to the commenter, this can result in an outcome that concludes that the dose-response for the observed adverse effect exhibits non-linearity or that it doesn't and requires considerable scientific judgment.
Commenter (6133) opposes the case-by-case evaluation of the appropriateness of using default assumptions, including assumptions of a linear, no-threshold dose response. The commenter asserts that "case-by-case" provisions inject additional arbitrariness into the rule, in that the Proposal may be applied unevenly.
Commenter (3135) states that the Proposal seems to naively assume that single studies are used to inform risk assessors of the possible shape of dose response curves. The commenter explains that this is not the case and that the first step of the dose-response modeling process is to evaluate all of the scientific information to gain a biological understanding of how each type of toxicity or response (adverse effect) occurs; the understanding of how the toxicity is caused is called the "mode of action." The commenter notes that via this evaluation the EPA identifies a sequence of key events and processes that result in the effect. The commenter explains that frequently the data do not conclusively prove mode of action. According to the commenter, in such cases, EPA often applies default assumptions such as low dose linearity for carcinogens unless the carcinogens can be shown to have a mode of action for which a threshold would be expected. The commenter explains that such defaults have been developed to assure that, in the face of uncertainty, the EPA will protect the public's health. The commenter adds that, in recent years, it has also been determined that often noncancer effects (e.g., lead and neurotoxicity) also have no threshold.
Commenter (1589) expresses concern that the Proposal places an emphasis on threshold models. The commenter provides that a threshold model is a form of a spline, a model where, for example, the slope of a linear association is allowed to change at a certain point. The commenter provides that in a threshold model, one considers a piecewise linear dose-response, where the slope is assumed to be zero below some dose "T", and some positive number above "T". According to the commenter, spline models are quite common in environmental epidemiology, but rather than, in the example above, forcing the slope below T to be zero, they allow the slope to change at "T", but allow the slope below "T" to take on any value the data dictates. The commenter asserts that the EPA has provided no reason to justify their proposed approach, or for constraining the slope to be zero when the data might differ. Moreover, the commenter disagrees with the EPA's assertion that there is growing evidence for nonlinear dose-response relationships and the implication that science is finding more evidence of thresholds. According to the commenter, in human studies the opposite is true. The commenter explains that there is a clear biological reason for this  -  most of these toxins act on pathways in the human body that produce the diseases of aging and that any incremental stimulus along these pathways would be expected to produce incremental damage to the body. The commenter asserts that this is why the National Academy of Sciences committee on risk assessment "recommends that cancer and non-cancer responses be assumed to be linear as a default."
Response: This rule does not apply to models, including dose-response models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Consideration of High-Quality Studies
The proposed rule requires that EPA give explicit consideration to high quality studies that explore: A broad class of parametric dose-response or concentration-response models; a robust set of potential confounding variables; nonparametric models that incorporate fewer assumptions; various threshold models across the dose or exposure range; and models that investigate factors that might account for spatial heterogeneity. 
Parametric and Nonparametric Dose-Response Models with Adjustments for Confounding Variables
Comment: Commenter (6148) supports the use of parametric and nonparametric dose-response models that include adjustments for confounding variables (e.g. contaminants with health impacts distinct from the target pollutant being modeled). The commenter provides that these confounding factors must be quantitatively factored into the selection of a threshold or non-threshold no observed-adverse-effect-level (NOAEL). According to the commenter, the EPA has not historically adjusted for model uncertainties to optimize low dose risk estimation based on competing models. The commenter asserts that such adjustments should be applied utilizing all available science, including, linear, threshold, U-shaped, J-shaped, and bell-shaped models. 
Response: This rule does not apply to models, including dose-response models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Comment: Commenter (6122) asserts that EPA deliberately mischaracterizes science in the Proposal statement that the "EPA should give appropriate consideration to high quality studies that explore a broad class of parametric concentration-response models with a robust set of potential confounding variables" and those that examine "spatial heterogeneity." According to the commenter, this approach would allow the EPA to give priority, regardless of the best available science, to studies that have data that fit a certain distribution (i.e. that use parametric tests). The commenter provides that not all data fit the assumptions for parametric models, and thus data may well be of high quality and constitute the best available science regardless of whether it fits into those assumptions. In addition, the commenter states that the proposed rule recommends EPA consider studies with a "robust set of potential confounding variables." The commenter asserts that, with many confounding variables, it is difficult for a study to attribute results to any specific chemical; instead studies control for confounding variables- holding them constant while varying the target chemical. The commenter states that controlling for confounding variables is the only way to detect any causation of the experimental variable on the dependent variable. Instead, the proposed rule would give priority to poorly designed studies with confounding variables such as, for example, age, weight, race, location, etc. According to the commenter, only studies that control for confounding variables can detect causation or high correlation with the pollutant, such as a pesticide or air pollutant, in question. 
Response: This rule does not apply to models, including dose-response models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Comment: Commenter (6915) states that for approximately 20 years, EPA has employed parametric modeling in risk assessment by providing and using benchmark dose response modeling software. According to the commenter, although the proposal implies that this is not the case, this software already allows investigation of the most appropriate parametric model(s) for risk assessment and currently provides "a broad class of parametric dose response, concentration-response models." The commenter states that there is no obvious benefit to adding an additional layer of analysis -- nonparametric modeling -- on top of this longstanding approach. The commenter asserts that nonparametric models are useful only when the quantity and quality of the data are sufficient to infer a clear and plausible estimate of the overall pattern. The commenter states that when there are few data and/or data are of poor quality, as is often the case in situations to which the proposed rule would apply, nonparametric models can produce a wide variation of results with few, if any, constraints on plausibility. The commenter states that, in such cases, the use of nonparametric models is not scientifically supportable, and moving forward, little would be gained from considering them in terms of accuracy and clarity of the predictions, while the potential for delay and obfuscation would again multiply. The commenter contends that the proposed rule's requirement that nonparametric models be explicitly considered, without regard to the applicability of a particular model, is therefore misguided and scientifically unsound.
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Comment: Commenter (6955) contends that the non-parametric modeling technique can be abused. According to the commenter, it is a misnomer, because one or more smoothing parameters must be assumed or fit to the data using debatable assumptions. The commenter provides that there are a large number of smoothing methods to choose from, especially when there are covariates, increasing the effective number of degrees of freedom. The commenter asserts that the analysis is hardly transparent. The commenter contends that such approaches are vulnerable to overfitting, which is why sophisticated error analysis is essential, but not always carried out. The commenter states that different analysts can come up with very different conclusions as is exemplified by contradictory fits to A-bomb survivor data by Sasaki et al. (hormesis result) and by Dropkin (supralinear result). The commenter notes that only Dropkin had error analysis on his outcomes (count rate errors only). Additionally, the commenter states that "uncontrolled confounding" is an objection commonly raised by those unhappy with the outcome of a study. The commenter asserts that they have learned that many who use the term "uncontrolled confounding" may not realize that confounding requires correlation with both the outcome and exposure under study. In any case, the commenter states that there are always data sets in the world that will so correlate, if a stakeholder searches long enough. The commenter contends that such searches are a form of post-hoc analysis and represent a form of multiple comparison that must be accounted for, if making a scientific claim. The commenter also states that there are also legitimate questions about confounding, including residual confounding, that are a problem for most studies. The commenter concludes that there are no perfect studies and that strengths and weaknesses of studies are the rule, which is why studies need to be assessed all together.
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Nonparametric Models that Incorporate Fewer Assumptions
Comment: Commenter (6122) states that the proposed rule also recommends the EPA consider studies with "nonparametric models that incorporate fewer assumptions." According to the commenter, while this allows for data that do not follow a certain distribution, it is unclear what "fewer assumptions" means in this case. The commenter provides that scientists state their assumptions in work submitted for peer review and follow the assumptions laid out in the model they choose to analyze data; the best model chosen based on other peer-reviewed literature or expert experience is finally deemed either appropriate or ruled out by peer-review. Accordingly, the commenter contends that the number of assumptions is itself not a separately relevant factor in evaluating the strength of a study, since it is a factor already considered in study design, data analysis, and the peer review process. The commenter asserts that work that has inappropriate assumptions will not make it past peer review and thus not be used for EPA regulations regardless.
Commenter (6373) states that encouraging nonparametric models with fewer assumptions will not necessarily lead to "better" or more accurate models. The commenter provides that one could argue the opposite: models with fewer assumptions are created by persons who are scientifically less familiar with the subject matter and/or statistical assumptions underlying the model. The commenter states that more scientifically sophisticated models are created by researchers who are more familiar with limitations and acknowledge these limitations via specific assumptions. The commenter notes that researchers should always acknowledge assumptions when defining and presenting their models and when defining existing controls during peer-review and, in the case of studies being considered for regulatory value. According to the commenter, this provides sufficient processes to determine whether the assumptions that a study relies on are sound and appropriate.
This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Methods for Assessing the Shape of a Concentration-Response Relationship
Comment: Commenter (1973) states that the EPA's Proposal details very specific criteria for judging what types of analyses provide valuable information about what concentration or dose-response curves look like, instead of relying on the judgment of the scientific community and its own science advisory committees. The commenter asserts that the Proposal puts an inappropriate emphasis on studies looking at broad classes of parametric forms for dose response which is an approach little used in environmental epidemiology, because those parametric forms by their nature can force dose-response curves into shapes not indicated by the data. According to the commenter, environmental epidemiology prefers penalized splines and similar non-parametric approaches and no epidemiology society recommends EPA's proposed approach. The commenter asserts that the methods for assessing the shape of a concentration-response relationship is a scientific question and better left to scientists and EPA's science advisor, without stating preferences with no scientific justification given, and no apparent input from either the scientific community or EPA's own scientific advisors. According to the commenter, the EPA's attempt to prescribe how a scientist should assess the shape of concentration-response relationships constrains the EPA from considering studies most scientists think are relevant, rather than to serve any public health purpose.
Commenter (6881) states that the proposed rule (§30.6) is overly prescriptive regarding the modeling approaches that should be applied and demonstrates misunderstanding of the concept of model uncertainty. The commenter asserts that this component of the proposed rule relates to epidemiological studies, and more specifically, appears to target air pollution epidemiology. Commenter notes that by following this approach, EPA would place greater weight on studies that force the data into shapes that may not be indicated by the data (i.e., by using defined parametric models and threshold models). The commenter asserts that this might prioritize studies which report a range of statistical approaches even if they yield poorly fitting curves, as compared to a study which uses a widely-accepted nonparametric modeling approach that performs well.
Response: EPA has not finalized proposed 40 CFR 30.6 which is the subject of these comments.
Model Optimization
Comment: Commenters (0062, 1273) support EPA incorporating the concept of model uncertainty when needed as a default to optimize low dose risk estimation based on major competing models, including linear, and U-shaped, J-shaped, and bell-shaped models. 

Response: This rule does not apply to models, including dose-response models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Comment: Commenter (6446) asserts that the statement in the proposed rule that "EPA should also incorporate the concept of model uncertainty when needed as a default to optimize low dose risk estimation based on major competing models" is unclear. 
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Comment: Commenter (1971) asserts that over the last two decades it was extensively shown that U-shaped, J-shaped, and bell-shaped (i.e. hormesis) dose responses are induced by chemical agents and radiation in hundreds of biological models, as reflected in hundreds of endpoints (Agathokleous et al.; Calabrese,; Calabrese and Baldwin,, ; Calabrese and Blain,; Calabrese et al.; Costantini et al.; Kim et al.). The commenter states that, more recently, it was shown that several environmental factors can induce hormesis in plants and animals and that hormesis can occur not only at the individual level and at the community level (Agathokleous; Agathokleous et al.,). The commenter, therefore, agrees with the optimization of low dose risk estimation based on linear, threshold, and U-shaped, J-shaped, and bell-shaped models. According to the commenter, failure to consider U-shaped, J-shaped, and bell-shaped dose-responses would be equal to a failure to conduct a realistic and validated risk assessment with potential consequences to animals, including humans, and to the environment.
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Comment: Commenter (3840) contends that the implication in the language of the Proposal is that the default models are too conservative, as can be gleaned from the language of the proposal, such as "model optimization," a phrase used in a comment for this rule and other papers by the hormesis supporter, E.J. Calabrese (Calabrese et al. 2018b, Calabrese et al. 2018a). The commenter states that this is also inferred by the descriptors of other models the EPA should consider: threshold, U-shaped, J-shaped, and bell-shaped model primarily imply sub-linearity. The commenter states that only a bell-shaped model could qualify as a supralinear model, but there are many other types. According to the commenter, without inclusion of the full range of supralinear models, the consideration of non-linear models will shift the balance to weaker standards, with subsequent increases in chemical and radiation exposures in the workplace, home, and outdoor environment, including the vicinity of superfund sites. The commenter provides that, if supralinear models are left out of the mix, as proposed by commenters, Calabrese et al. (Calabrese et al. 2018a), the analysis will be biased low. On the other hand, the commenter states that if supra-linearity is included, the mid-point curve will likely be (roughly) linear, possibly with a modest DDREF (Rühm et al, 2015)  -  interpreted not as an indication of truth but as an indication of the mid-point of an uncertain region.
Furthermore, the commenter (3840) asserts that when it comes to consideration of dose-response models other than linear, whether sub- or supralinear, it should be recognized that they are generally not mature enough to be considered for regulatory analysis, because they do not prescribe how they behave over the full range of temporal dose-delivery patterns for which regulators must deal. The commenter notes that there already is a standard form of model optimization for uncertain situations, namely, uncertainty risk analysis. The commenter explains that dose-response models cannot be optimized at low doses, because there is no data on which to check their performance, which is why a risk analysis with uncertainty up and down is necessary. The commenter concludes that they would certainly like to see more consideration of uncertainty ranges at low doses and low dose rates, but they state that they doubt those who think regulations are too strict would be happy with the outcome, which would likely focus on risks greater than linear, out of a desire to be prudent.
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Comment: Commenter (1010) suggests that, the best answer, for the foreseeable future, from theoretical data-support, and public health perspectives is the use of dose response model uncertainty, that is, using the leading dose response models and determining where they optimally converge (i.e., dose where health benefits are optimized and risks are minimized). The commenter provides that, in practice this involves finding a practical and scientific means to integrate the threshold, LNT, and hormetic dose response models, the three models with the most toxicological gravitas based on the peer-reviewed published literature. According to the commenter, each model has its strengths and limits, its advocates and detractors, but authors favor the hormesis model over the LNT model. The commenter asserts that the combination and integration of these three models into a dynamic risk assessment framework works best because it has the potential to integrate the best scientific features of the three models while limiting/minimizing the possibility of error. The commenter provides that the case for this integrated dose response approach has been published in several peer-reviewed chemical and radiation health risk assessment publications (Calabrese, 2015a; Calabrese, et al., 2015, 2016). The commenter states that the attractive features of this integrative approach are that the nadir of the hormetic dose response, based on a large number of studies in the hormetic data base (Calabrese and Blain, 2011), and the "safe" exposure estimate using the threshold dose response model with a standard 100-fold uncertainty factor yield essentially the same value. Thus, the commenter provides that these two models provide an agreement even though they offer a different toxicological interpretation (i.e., no effect/safe threshold interpretation versus beneficial hormetic interpretation). The commenter concludes that the integration of the three most credible scientific models within a model uncertainty suggest that more research still needs to be undertaken to improve the reliability of model-based low dose estimates. However, the commenter contends that despite the remaining uncertainties of this approach, it offers considerable scientific and societal advances and should be adopted by the U.S. EPA and other environmental regulatory agencies in other countries.
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Comment: Commenter (6911) states that a tool to optimize exposure assessments is probabilistic risk assessment (PRA). The commenter states that PRA is an analytic tool used to incorporate information regarding variability and uncertainty into analyses to provide insight regarding the degree of certainty of a risk estimate. The commenter notes that, as the best estimate of risk reduction benefits should be used in a benefit-cost analysis, EPA should adopt PRA as the preferred risk assessment method to provide the benefits estimates for its benefit-cost implementing regulations. The commenter asserts that EPA's traditional approaches, such as deterministic risk assessment, produce unrealistic, overly conservative outputs. Commenter includes the following additional information regarding PRA.
      EPA's use of compounded conservative assumptions is not only improper as a scientific matter; it is also contrary to EPA's own policy documents, which recognize that the Agency does not need to use excessively conservative input values on all equation parameters in order to yield adequately protective regulations. Indeed, EPA's 2014 Risk Assessment Forum White Paper on probabilistic risk assessment (PRA) The White Paper identifies situations in which PRA "may be particularly useful," including situations in which "uncertainty in some aspect of the risk assessment is high, and decisions are contentious or have large resource implications," as well as situations in which "the scientific rigor and quality of the assessment is critical to the credibility of the EPA decision." (http://www.gpo.gov/fdsys/pkg/FR-2014-08-12/pdf/2014-19065.pdf.)
      Other scientific disciplines use PRA. For example, in safety engineering, the designers of chemical manufacturing plants, nuclear power plants, highways and air traffic control systems use probabilistic approaches to assess risk. Fisheries biology uses Monte Carlo (probability) simulation to study alternate management practices. Financial analysts commonly use Monte Carlo simulation. EPA should acknowledge that probabilistic methods are well defined mathematically and have long been used in other professions and disciplines. The simple fact that EPA still uses deterministic methods in most of its human health risk assessments indicates that EPA does not recognize the benefits of adopting probabilistic methods. Deterministic methods hide a potentially great deal of uncertainty about the actual risk and also hide policy judgements. The lack of transparency involved in these judgements affects the credibility of scientific outcomes and results in poorer policy decisions. The potentially higher resource costs that are required for PRA are well justified by the far greater certainty and transparency of the risk-reduction benefits and ultimately the efficiency of the optimal regulatory standard.
Response: These comments are not within the scope of this rulemaking.
Spatial Heterogeneity
Comment: Commenter (6122) states that spatial heterogeneity is also something that is already appropriately addressed through scientific peer review, as spatial heterogeneity is incorporated into data collection methods, data analysis models, and in uncertainty analyses.
Response: The fact that spatial heterogeneity is considered during peer review is not sufficient justification to exclude it from consideration in determining the degree of consideration to afford pivotal science for which the dose-response data are not available for independent validation.
Commenter (6373) states that encouraging models that include spatial heterogeneity (that is, variability across geographic regions) may potentially lead to models that dilute the effects of a toxic exposure if the exposure was concentrated in one geographic region -- leading to scientifically invalid conclusions (e.g., that an exposure was not associated with negative health effects). 
Response: This rule does not apply to models and thus does not encourage models that include spatial heterogeneity.
Comment: Commenter (6881) states that the call to consider spatial heterogeneity is largely pertinent to only national-scale air pollution epidemiology and does not consider whether such an analysis is indicated by the study design and available data. 
Response: The proposed regulatory text the commenter refers to is part of proposed 40 CFR 30.6. Proposed 40 CFR 30.6 was not finalized. The term "spatial heterogeneity" is included in 40 CFR 30.5 as part of one of the factors EPA will consider in determining the degree of consideration to afford pivotal science for which the dose-response data are not available for independent validation. The commenter has not provided a rationale for why spatial heterogeneity's pertinence to air pollution epidemiology should exclude it as a factor in 40 CFR 30.5 (or in the original proposed 40 CFR 30.6).
Linear versus Non-Linear Dose Response Relationships
Comment: Commenter (6373) states that the Proposal repeatedly uses the term "dose response" (i.e., concentration-response), but neglects to recognize that "dose" cannot be measured in observational epidemiological studies. The commenter is concerned that this has implications for EPA's establishment of NAAQS. The commenter provides that the issues with linear versus non-linear "dose response" relationships are different between animal toxicological studies and epidemiological studies. Observational epidemiological studies often do not have control over the potential heterogeneity in vulnerability in the population being studied or the spatial variation (and/or error in measurements) in pollutants' concentrations where the subjects reside. Thus, the commenter states that the apparent "dose response" relationships in epidemiological studies do not simply reflect biological response function, making the interpretation of "dose response" curves in observational epidemiological studies not just a matter of fitting a "right" statistical model. The commenter states that the Proposal narrative describes this issue as merely statistical, which is not correct, and therefore the Proposal's application to epidemiological research is unclear and contrary to scientific practice.
Commenter (6373) states that, because there is frequently significant variation within the exposure to a pollutant and the reaction of different individuals in a study group to that exposure, and because it is impossible to test the impact of high levels of toxic exposure to humans, modeling practices may require that linear dose response assumptions are applied despite outlying data or uncertainty about high-exposure predictability. The commenter states that this does not invalidate best efforts to create responsive and predictive models, and current scientific best practices already require a thorough explanation of modeling practices, assumptions, and options for dose response curves within a study.
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Support Dose-Response Model Selection Guidance/Guidelines Over a Rule
Comment: Commenter (6449) states that they support EPA considering and expanding the variety of dose-response models to include threshold and nonmonotonically-increasing functions, but they suggest that it should be part of guidance or EPA Guidelines, and not part of the rule. The commenter suggests that EPA may want to provide a method for interested parties to submit papers on these issues that would be publicly available prior to workshops and other methods of discussion. The commenter contends that this could provide EPA not only with multiple options but also with some of the strengths and weaknesses of these models. The commenter states that a draft of how EPA is considering use of such models, including some examples, would provide further comment that could be incorporated before guidance is final.
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Inquiry Regarding Researcher Model Use Requirements/Reanalysis of Studies to Fit Criteria
Comment: Commenter (6881) asks, given most journal articles would not include such a wide range of models, if EPA is proposing that individual researchers should produce additional models and provide them to EPA, or that EPA plans to reanalyze studies to fit these specific criteria? 
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
4.1.1.2 Models
4.1.1.2.1 Use of Proprietary Models
Models Developed by EPA Should be Constructed in Freeware
Comment: Commenter (6449) recommends that models developed by EPA be constructed in R or other freeware, and the code posted with the document in which they were used. The commenter states that renting or buying proprietary software can impede the transparency advocated in this proposal. 
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Use of Commercially Available Computer Software
Comment: Commenters (0562, 6108, 6941) assert that commercially available computer software is acceptable for rulemaking. Commenters (0562, 6941) take the position that it is acceptable to use results from commercially available models for regulatory decisions without the need to release proprietary code or internal algorithms and that requiring commercial model developers to release this intellectual property would likely restrict their ability to continue developing and supporting their models. Commenter (0562) states that, in some cases, availability of an open source model or use of both commercial and open source models together may be the "best available approach". 
Commenter (6108) states that industry often uses commercially available computer code and software to design, build, and operate chemical facilities, and to predict emissions. The commenter provides that it is unlikely that commercial software makers would be willing to surrender their extensive code, as that could give competing software companies an advantage. The commenter states that the key is to use the computer code and/or software within its intended applicability range and with appropriate assumptions. The commenter defines "commercially available" as published software or code with a sound scientific basis, available for use by the public or government. The commenter suggests that any commercially available models or computer codes that are used in rulemaking decisions include a disclosure about the basis, applicability limits, and assumptions of the parameters used with the computer code/software. The commenter states that making the complete computer code or software code available for review is not necessary. The commenter suggests that if, however, computer code is used in a novel way specific only to the study at hand, such code be made public.
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Grants and Cooperative Agreements Should Stipulate that Models Used be Publicly Available 
Comment: Commenter (6356) asserts that EPA grants and cooperative agreements should stipulate that models developed as the basis for studies and the generation of data EPA intends to use to support regulatory actions must be available for the public to utilize, even if they are copyrighted. The commenter states that any use should be available only if the user is willing to agree to copyright protections, which could be encoded into software for downloading such information from public servers, that would identify the user and his agreement to be liable if he or she evades copyright protections for his or her own gain. The commenter contends that this general usage agreement is no different in fields of science than in the fields of art and music or other intellectual property. 
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
4.1.1.2.2 Consideration of Alternative Models
Support Proposal to Consider Alternative Models
Comment: Commenters (1011, 1971, 6911) support EPA's proposed rules for consideration of different models. 
Commenter (1971) agrees with the proposal to consider "high quality studies" that explore: i) "A broad class of parametric concentration-response models with a robust set of potential confounding variables"; ii) "nonparametric models that incorporate fewer assumptions"; iii) "various threshold models across the exposure range"; and iv) "spatial heterogeneity". The commenter asserts that this requirement is an important step towards a more dynamic and flexible risk assessment which will not face animals, plants, and ecosystems as static entities but as biological entities that display complex responses which differ upon environmental conditions/constraints. The commenter states that this need is highlighted for vegetation response to ground-level ozone which can be highly driven by soil water or nutrient availability. The commenter contends that it would be ironic to have the same ozone critical levels for a northern region with high water availability and a southern region with notable drought. Thus, the commenter agrees that high quality studies should be considered which select the proper model among a set of statistically tested models. 
Commenter (6911) states that recognizing alternative models should not be a polarizing process and the selection of the model used to estimate exposure-response relationships of chemicals can have significant impacts on estimated regulatory benefits. The commenter provides that, in 2007, Office of Information and Regulatory Affairs (OIRA) and the Office of Science and Technology Policy (OSTP) observed that a "high degree of transparency with respect to data, assumptions, and methods will increase the credibility of the risk analysis, and will allow interested individuals, internal and external to the agency, to understand better the technical basis of the analysis."
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Do Not Support Proposal to Consider Alternative Models
Comment: Commenters (2907, 6122, 6168, 6373, 6872, 6881, 8281-PH46) do not support EPA's proposed rules for consideration of alternative models. 
Commenters (2907, 6168, 6881, 8281-PH46) assert that dictating and fitting numerous specified curves to study data without an empirical basis is needlessly exhaustive and costly. Commenter (2907) states that there are literally an infinite number of possible dose-response models, as the proposal points out; for example, linear, threshold and non-threshold, U-shaped, J-shaped, and bell-shaped. 
Commenter (2907) states that the EPA has historically taken the reasonable position that toxic chemicals and pollution are deleterious to human health, and that an increase in pollution will lead to an increase in adverse effects. The commenter states that the proposal discards those default health-protective assumptions out, and instead invites literally an infinite number of model options from which the EPA can choose.
Commenter (2907) states that the proposal requires the EPA to give equal consideration to many dose-response models, including those that presume that there are safe levels of a toxic chemical -- or even that it may be beneficial. The commenter states that this may be appropriate when testing medical drugs but not for toxic pollution.
Commenters (6168, 6881, 8281-PH46) assert that dictating and fitting numerous specified curves to toxicological study data without an empirical basis would remove expert judgement and biological context and does not represent best scientific practice.
Commenters (6122, 6872, 8281-PH46) provide that simply using/considering virtually all model alternatives as the proposal preference is unlikely to improve results without considering the models' assumptions and whether they fit the data set, the goals of the analysis, and other issues. 
Commenter (6881) states that most available toxicological data sets do not have the resolution to differentiate among many different model forms that can have extremely different low-dose extrapolations. The commenter provides that many parametric model forms would fit the data equally well but result in a wide range of estimated risks. The commenter adds that, where the data allow for such differentiation, those models that do not fit the data well should be down-weighted, and those models that are not mechanistically supported should not even be considered. The commenter contends that best scientific practice involves using model averaging, wherein numerous mechanistically justified models are fit, and the better-fitting models receive a higher weight than more poorly-fitting models. 
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Comment: Commenters (6122, 6168, 6872) assert that this proposed requirement would further delay an already slow process for completing toxicity assessments that will defer life-saving steps to address pollution. According to commenter (6168), prioritizing model-fit in this way counteracts a health-protective timeframe, endlessly extending the timeline for determining which chemicals are safe or not. Commenter (6168) asserts that a recent example of this negligent mode of operation culminated in the court order filed August 9th, requiring EPA to finalize the ban on the pesticide chlorpyrifos.
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Comment: Commenter (6881) contends that while one can fit many curves to a set of data and show that the resulting risk estimates differ, this is not necessarily reflective of the true level of uncertainty and would thus further obscure the assessment and impede decision making. 
Response: The comment refers to dose-response models. This rule does not include requirements for dose-response models.
Comment: Commenter (6373) asserts that §30.6 of the proposed rule improperly allows EPA to second guess the statistical model selected by researchers, which could motivate researchers to improperly tailor their methodologies so that their work may be "considered" during EPA's regulatory process, to the detriment of the study's validity. The commenter asserts that, while the proposal would improperly allow EPA to prioritize specific statistical models, the proper choice of statistical model must be determined based on the data being collected and analyzed and the study design. 
Response: EPA has not finalized the proposed 40 CFR 30.6 requirements.
4.1.1.2.3 Model Evaluation Guidance/Criteria
Comment: Commenters (1011, 6446) express concern that the proposed rule does not address guidance for evaluating models.
Commenter (1011) states that changes in the use of underlying models should be further explored through appropriate guidelines that allow for appropriate input from the scientific community and NAS prior to implementation. 
Commenter (6446) states that the proposed rule does not address guidance for evaluating the scientific plausibility and usefulness of alternative dose-response models. The commenter states that the proposed rule generally does not provide any guidance or direction on how alternatives to current EPA dose-response modeling practice would be evaluated for their scientific plausibility and usefulness. Commenter states that, while the preamble to the proposed rule states, "EPA should give appropriate consideration to high quality studies that explore...nonparametric models that incorporate fewer assumptions", nonparametric dose-response models lacking a biological background can be manipulated to result in output that does not correspond to the biological reality. The commenter expresses concern that the proposed rule does not compel such models to be consistent with biology and to be evaluated for biological plausibility.
Response: This rule does not apply to models. See the preamble of the rule which explains the decision to limit the scope of the rule to dose-response data.
Comment: Commenter (6362) notes that EPA should adopt model evaluation criteria to apply the greatest weight and credibility to models that are open access, describe the endpoint predicted clearly, are based on unambiguous open access computer algorithms, have a defined domain of applicability, have been transparently verified with publicly available datasets, and are shown to be robust and scientifically sound for the intended use. 
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Comment: Commenter (6449) suggests that all new models be vetted separately, preferably before they were used and before they are used in another analysis. The commenter adds that exceptions and additions to existing EPA guidance may allow for improved analyses, but the strengths and limitations of those changes should undergo the same level of review, e.g., interagency comment and public comment, as the original guidance document. When performed as part of a chemical-specific analysis, the commenter contends that there is insufficient time and expertise available for a full discussion of the potential utility and limitations of the changes.
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
4.1.1.2.4 Model Documentation
Comment: Commenter (6375) suggests that, with respect to dose - response models, EPA require documentation of the choices made at each step of the process and to include a scientific rationale for the selected approach. The commenter provides suggestions for documentation related to, but not limited to, data and model selection, uncertainty and variability/sensitivity analyses, model parameter estimation and use, methods/assumptions, and specific requirement considerations when documenting epidemiological studies and reporting of dose or concentration response assessments.
Commenter (6891) supports the proposed regulatory text that the underlying data or models must include "the information necessary for the public to understand, assess, and replicate findings." The commenter agrees that such necessary information may include associated protocols necessary to understand, assess, and extend conclusions; computer codes and models involved in the creation and analysis of such information; and detailed descriptions of how to access and use such information. The commenter encourages the EPA to present data and models with adequate explanatory materials so that the public can readily understand their relevance and how they were used in reaching a decision. The commenter adds that EPA consider document format and legibility (including the use of acronyms/headings) as part of providing meaningful public access.
Commenter (6935) states that, consistent with the intent of this regulation, EPA should provide more information about the inputs, assumptions and outcomes from the Agency's own proprietary models so that stakeholders can assess and independently validate the results, which often provide the exclusive scientific justification for new regulations. 
Commenter (0555) supports the requirement that EPA describe and document its assumptions and methods and show how sensitive modeled results are to those and alternative assumptions. The commenter states that assumptions and judgments can bias the ultimate advice provided to decision-makers and the public. The commenter states that documenting those assumptions and estimating how sensitive predicted outcomes are to them and alternative assumptions and judgments could greatly improve the transparency and quality of EPA's decisions. Commenter states that policymakers will face more difficult choices when faced with a range of reasonable estimates but argue "that is how it should be." Commenter states that this requirement comports with recommendations from various cited sources.,,,,,,
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
4.1.1.2.5 Impacts on Current Agency Tools and Models
Comment: Commenter (6935) states that EPA should consider possible requirements of a final rule from the perspective of impacts on current Agency tools and models (e.g., IPM) used to support regulatory proceedings, especially those tools and models used not only by EPA but also by states and private entities. 
Response: EPA is pursuing an incremental approach to maximizing transparency in the pivotal science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. As further described in the preamble to the final rule, the EPA is focusing on the underlying dose-response data because of the influence these data have on particularly impactful decisions at the Agency. Risk assessments and regulations that target emissions and risk reduction of one or more pollutants, contaminants, or substances are integral to the Agency's mission and the underlying dose-response data that inform the quantitative value used to evaluate and mitigate potential risk are critical to understanding the assessment or regulatory action. In addition, the data underlying the dose-response assessment are more distinct than the broad range of data and models informing an entire risk assessment.
Comment: Commenter (6133) lists the following examples of currently used EPA models and tools that may not meet the transparency requirement of the proposed rule:
 Integrated Planning Model (IPM)
 National Electric Energy Data System (NEEDS) Database
 AVoided Emissions and GeneRation Tool (AVERT) (developed by Synapse)
 Co-Benefits Risk Assessment Tool (COBRA)
 Community Multi-scale Air Quality (CMAQ) Modeling System
 U.S. EPA MARKet ALlocation (MARKAL) model Database
 Emissions & Generation Resource Integrated Database (eGRID)
 National Emissions Inventory (NEI)
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
4.1.1.2.6 Uncertainty and Variability
General Comments on the Requirement to Describe Variability and Uncertainty
Comment: Commenters (0068, 6150, 6913) provide general support for requirement to describe variability and uncertainty.
Commenter (6150) requests that EPA evaluate quantitatively, where possible, the effect of variability and uncertainty on results and give preference to studies that evaluate a range of models, including threshold models, and that account for spatial heterogeneity. 
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Comment: Commenter (6913) supports the requirement that EPA explain the scientific uncertainty that underlies their decision making. The commenter asserts that, during the most recent update for ozone, the models that EPA relied on had clear inconsistencies. The commenter asserts that the prevailing example was that some areas showed an increase in mortality with decreasing ozone concentrations. The commenter states that EPA directly linked ozone concentrations with asthma, even though research on this issue is not clear-cut. According to the commenter, the significant impact to industry and the adverse economic consequences of the lower ozone standard plainly outweighed the unverified health benefits, but EPA nevertheless implemented the lower concentration. 
Response: EPA will characterize the uncertainty in its pivotal science. EPA notes that this rule does not apply to models.
Comment: Commenter (6168) states that unconventional models and novel research approaches to predict adverse health outcomes are often mischaracterized as lacking transparency, but that the proposed measures on data and model availability and use facilitate not transparency but the culture of science denialism: dismissing important evidence and dredging data to endlessly produce false uncertainty and delay scientific advances regarding the harmful effects of chemical exposures. According to the commenter, this false uncertainty is used to justify failure to protect environmental and public health.
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Uncertainty and Variability in Risk Assessment
Comment: Commenter (6155) suggests that the problem of uncertainty in risk assessment should be divided into smaller and more manageable topics. The commenter states that, for dose-response analyses, smaller and more manageable topics would include: 1) extrapolation of tested doses to human doses, or otherwise setting forth adequate and accurate measurement of doses in the study compared to actual human doses; 2) adequate definition of "positive responses" in a given study; 3) identification of parameter estimation assumptions and techniques; 4) identification of dose-response sets of data, including quantities, qualities, and types; 5) explanation and justification of the model selection for low-dose extrapolation, including identification of the type of dose-response relationship chosen (e.g., threshold vs. linear), the role of time (e.g. dose frequency, rate, and duration), 6) explanation of the pharmacokinetic model used (including clarification of the effective dose as a function of applied dose in the study in question), and 7) the impact of competing risks (i.e., addressing the correlation vs. causation issue in the particular model used and how that may have influenced the study's overall conclusions). 
Commenter (6155) suggests that an alternative approach to uncertainty would incorporate by reference the "Guidelines for Exposure Assessment," 57 Fed. Reg. 22888 (May 29, 1992) (the 1992 guidelines). The commenter states that the 1992 guidelines address four areas that any final rule's data transparency requirements should address: Gathering and Developing Data for Exposure Assessments, Using Data to Determine or Estimate Exposure and Dose, Assessing Uncertainty, and Presenting the Results of the Exposure Assessment. The commenter also states that the 1992 guidelines also break down uncertainty in dose-response exposure assessments into the following three general categories and contain guidance on how to address each of the three types of uncertainty that the guidelines identify.
 Uncertainty regarding missing or incomplete information needed to fully define the exposure and dose (scenario uncertainty).
 Uncertainty regarding some parameter (parameter uncertainty).
 Uncertainty regarding gaps in scientific theory required to make predictions on the basis of causal inferences (model uncertainty).
Response: This rule applies to independent validation of dose-response data underlying pivotal science. The areas suggested by the commenters are outside the scope of this regulation.
Comment: Commenter (6932) suggests that EPA identify and include variability in human exposures and vulnerability into risk assessments. The commenter states that risk assessment conclusions can differ depending on what data get used or emphasized. The commenter states that the 2014 NAS report found that "related risk-assessment results may differ substantially" with the choices scientists make about which studies to use. The commenter states that different risk conclusions can be reached because of "differences arising from different emphases placed on datasets and how uncertainty and variability" were considered. The commenter adds that the 2014 NAS report recommended that the EPA identify and include variability in human exposures and vulnerability into risk assessments in order to better protect health. The commenter states that epidemiology studies would be vital to this recommended approach and that while risk assessment conclusions may vary due to different points of departure or emphases on different datasets, the decisions made are not "arbitrary," as the docket language claims.
Response: This rule concerns the availability of dose-response data underlying pivotal science. These comments address risk assessment not availability of dose-response data or even other data.
Uncertainties in Environmental Regulations
Comment: Commenter (0560) recommends consideration of uncertainties in environmental regulations. The commenter suggests EPA emphasize using national and international guidance on the measurement of uncertainties, including Taylor, NIST 1994, and JCGM 100:2008. The commenter suggests EPA give especial attention to Type B errors. The commenter states that climate science publications have particularly ignored these national and international guidelines on the measurement of uncertainty.
Response: This rule applies to independent validation of dose-response data underlying pivotal science. Considerations of uncertainties in environmental regulations is outside the scope of this regulation.
Incorporation of Model Uncertainty
Comment: Commenter (0062) states that it is appropriate to consider model uncertainty in a new field of study, since data would not be available to enable development of reliable, verified models. The commenter states that in fields that have been studied for a long time, plenty of data would be available to compare with model predictions enabling identification of the most appropriate model and rejection of models that are inconsistent with data. 
Commenter (0062) states that a possible approach EPA can take to resolve the issue of model ambiguity/uncertainty is to set up an open, online debate on the subject inviting key participants on both sides to provide evidence and arguments supporting their own side and provide reasons why the opposing side's arguments and evidence are not valid. The commenter notes that the back and forth debate on individual points should be allowed to continue until a conclusion is reached, as determined by a neutral observer. The commenter provides that with such an open process, the decision on the correct model can be reached in a transparent, unbiased manner. 
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Comment: Commenter (6046) states that the proposed rule should not propose scientific methods, approaches or policy, such as when certain dose-response assumptions should be used or whether EPA should incorporate the concept of model uncertainty. The commenter asserts that science is a method which should be left to scientists who best understand its use. 
Response: This rule does not apply to models or dictate what scientific methods should be used.
Comment: Commenter (6448) agrees that as our understanding of the evidence of non-linearity in the concentration response function for certain pollutants and health effects improves, the corresponding risk models should be appropriately refined with appropriate model uncertainty to reflect the improved understanding. 
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Comment: Commenter (6881) states that, while we are supportive of the concept of incorporating model uncertainty, simply fitting a prescribed list of models to a small number of observations just provides a list of varying risk numbers without any strong scientific basis or ability to move forward. The commenter notes that this reflects a fundamental misunderstanding of the concept of model uncertainty, ignoring the fact that some models are better justified than others, either based on scientific theory or empirical evidence related to curve-fitting. The commenter contends that the text is confusingly written and not well thought through and would leave the Agency ill-prepared to conduct risk assessments. 
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Comment: Commenters (0559, 5170) state that EPA needs to have precautionary policies for risk assessment and management of hazardous chemicals with low-dose effects. The commenters argue that because human health is so important and illness so costly, evidence of health risk should take precedence over model uncertainty. The commenters express concern that the introduction of model uncertainty based on competing models at low-dose ranges will result in a lack of appropriate regulatory activity, even where there is strong evidence for low-dose adversity in peer-reviewed publications. The commenters argue that, where there is evidence for adversity, particularly at low-dose ranges, it cannot be assumed that there exists a threshold below which no risk exists. Commenter (5170) states that regulation must proceed even without perfect information in order to do our best to protect public health. 
Commenter (6956) asks that the agency avoid the precautionary principle approach to regulatory action. The commenter recommends that the agency be very transparent with those datasets that seek to establish the precautionary approach for a product or for all regulations within the U.S. The commenter provides that models should reflect real-world data when that data is available for comparison. The commenter states that the Agency should not accept model results that contradict actual sample data. The commenter adds that different offices within the Agency or different agencies should not have models that suggest wildly different result-scenarios for the same test subject. The commenter asserts that when you have the introduction of such model uncertainty and discrepancy based on competing models then some investigation must occur. The commenter states that, for model uncertainty at low-dose ranges, if one model shows a possible adverse condition while other models do not, that is no justification for simply accepting outliers. 
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
4.1.1.2.7 Model Assumptions
Scientific Basis of Model Assumptions
Comment: Commenters (0068, 0555, 1011, 6363) agree that the scientific basis for model assumptions should be clearly described and documented. Commenter (6363) states that models need to have a foundation that is replicable and reflective of relevant exposure scenarios. The commenter adds that there can be a significant gap in modeling assumptions and what occurs in relevant environmental conditions. According to the commenter, disclosure of the underlying assumptions and methodology will allow affected stakeholders to provide input that could enhance the validity (reproducibility) of modeled outcomes. 
Commenter (0555) supports the requirement that EPA describe and document its assumptions and methods and show how sensitive modeled results are to those and alternative assumptions. The commenter states that assumptions and judgments can bias the ultimate advice provided to decision-makers and the public. The commenter states that documenting those assumptions and estimating how sensitive predicted outcomes are to them and alternative assumptions and judgments could greatly improve the transparency and quality of EPA's decisions. Commenter states that policymakers will face more difficult choices when faced with a range of reasonable estimates but argue "that is how it should be." Commenter states that this requirement comports with recommendations from various sources, including the following: 
 Analyses that do not provide information on how sensitive the primary estimate is to assumptions, data, and models, and the range of outcomes possible under reasonable alternative analytic assumptions should raise questions. Sensitivity analysis examines different "what if" scenarios to see how changes in key assumptions (or combinations of assumptions) influence estimated outcomes. Because many uncertain factors determine the impact of any regulation, one should look for a convincing justification regarding which uncertain parameters have the most consequential effects on outcomes, and a sensitivity analysis that varies these factors over a reasonable range to gauge their effects on the rule's net benefits.
 In 2010, OSTP directed agencies to communicate scientific and technological findings to the public "by including a clear explication of underlying assumptions; accurate contextualization of uncertainties, and a description of the probabilities associated with both optimistic and pessimistic projections, including best-case and worst-case scenarios where appropriate."
 This is important, because, as Dudley and Peacock explain, "scientists will never have complete information to predict outcomes with certainty, so analysts rely on what the [National Research Council] calls `risk assessment policy' -- assumptions, judgments, and rules of thumb -- to guide the use of scientific information in analyses that inform policy in the face of uncertainty." 
 The Institute of Medicine observed in 2013:
 Uncertainty is inherent in the scientific information upon which health risk estimates are based. Uncertainties enter the health risk assessment process at The George Washington University Regulatory Studies Center every step and can be caused by the potential confounders in observational studies, by extrapolation from animal studies to human studies, by extrapolation from high to low dose exposures, by inter-individual variability, and by modeling the relationships between concentrations, human exposures, and human health responses and evaluating the effect of interventions or risk control options on public health risk.
 Fundamentally, the EPA should replace risk values that are built on science-policy assumptions with risk estimates that acknowledge underlying uncertainties. For instance, the agency could follow the example of the Intergovernmental Panel on Climate Change and report a range of risks that correspond to different models. Users would then be able to see whether a value is sufficiently precise to support a particular course of action.
 The EPA's definitive values are illusions: they conceal uncertainty that cannot be resolved scientifically. Bringing conflicting value judgements into the open will enable honest debate and improve public health.
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Disagree with Limiting Assumptions Made in Models
Comment: Commenter (2023) does not agree with the proposal to limit the number of assumptions made in models for dose response data. The commenter states that assumptions are necessary unless the EPA is now willing to test dosages on humans. The commenter asserts that models are necessary after testing is done on animals, and for models to work, assumptions have to be made with the best available information. The commenter asks who will decide on whether a model is making too many assumptions? 
Commenters (2023, 6373) express concern that the proposal may inappropriately be used to discredit studies that assess the impacts of exposures to pollutants. Commenter (2023) asserts that the language about impact assessment and economic impact assessment implies that valuable models will be discounted because they make assumptions based on other scientific studies. Commenter (2023) states that, if those models and estimates went away (because they are deemed "not transparent"), then the absence of guidance will cause more human health and environmental damage. Commenter (6373) states that, to require EPA to assess modeling and assumptions independently, and in addition to the assessment contained in the study itself, provides EPA with unwarranted discretion to inappropriately discount certain studies because there may be multiple dose response curves, or no dose response curve, that best or "correctly" fits a data set, despite a demonstration within the data that exposure has impacts to human health.
Response: EPA has not finalized proposed 40 CFR 30.6 so there are no requirements specific to dose-response models. 
Sensitivity Analysis Requirement for Alternative Modeling Assumptions
Comment: Commenter (6446) states that the requirement for sensitivity analysis for all alternative-modeling assumptions does not require an evaluation of the scientific validity of those models. The commenter notes that the proposed rule's requirement that dose-response modeling include a sensitivity analysis including all alternative modeling assumptions does not make any mention of how those assumptions would be scientifically justified. According to the commenter, this lack of scientific justification criteria would invite the submission of models that would not have biological relevance for inclusion in the sensitivity analysis. The commenter contends that this would result in a process that would make EPA risk assessment documents unwieldy, unmanageable and unreliable.
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Use of Default Assumptions
Support the Use of Default Assumptions
Comment: Commenters (6446, 6881, 6924, 8771) support the use of default assumptions and express opposition to the proposed requirements in §30.6 regarding assumptions. 
Commenters (6446, 6881) assert that the need for default approaches in risk assessments has long been recognized as necessary. The commenters note that the National Academies of Sciences (NAS) has reviewed this approach to defaults throughout the years, and the need for default approaches to perform assessments absent compelling science to the contrary has been established. Commenter (6924) states that the NAS highlights that health protective defaults are important because they incorporate factors that reflect the range of variability and susceptibility in the population to ensure risks are not underestimated. Commenter (6446) states that EPA has therefore developed a series of practices, handbooks and guidance documents over the years that guide the development of risk assessments. 
Commenter (6924) recommends that the default should always be used for factors that are known to influence risk unless there is chemical-specific data that support increasing or decreasing it. The commenter suggests that EPA's defaults should include:
 Inter-human variability, general
 Inter-human susceptibility to carcinogens, adult
 Inter-human susceptibility to carcinogens, early life (including prenatal)
 Inter-human susceptibility to non-carcinogens, early life (including prenatal)
 Animal findings are relevant to humans
 Findings from one route of exposure are considered representative unless data show otherwise.
Commenter (6881) asserts that the statement that "the use of default models, without consideration of alternatives or model uncertainty, can obscure the scientific justification for EPA actions" is incomplete, poorly defined, and does not reflect the necessity to make decisions based on uncertain information. The commenter states that it is important to recognize at the outset that default models are intrinsic to risk assessment. The commenter states that, starting with the 1983 Red Book, which established the foundations of risk assessment, federal agencies were recommended to develop uniform inference guidelines to avoid the possibility of manipulation of risk assessment outcomes and to help to standardize risk assessment across chemicals, sites, and scenarios. The commenter asserts that these so-called "defaults" were intended to be the best choice in the absence of data to the contrary, based on a strong scientific foundation but ultimately requiring some science policy judgments. The commenter states that "default" should therefore not be used as a pejorative term  -  it simply reflects the best current scientific understanding at a time when a decision needs to be made, for an Agency facing many similar decisions on a regular basis. The commenter provides that new scientific understanding will lead to methods that deviate from the defaults and may, in fact, lead to new defaults. According to the commenter, default models are based on current scientific understanding  -  far from obscuring the scientific justification for decisions, they illuminate them. 
Commenter (6446) states that requiring justification for all default assumptions ignores establishment of defaults through prior public processes and external public peer reviews. The commenter notes that requiring EPA to justify the use of default assumptions for every toxicity health factor derivation ignores the detailed development process of those default assumptions. The commenter provides that those default assumptions were developed in several EPA documents (including the 2005 Guidelines for Carcinogen Risk Assessment), which received public comment and extensive external peer-review. The commenter contends that requiring EPA to re-justify those default assumptions every time they are used, as the proposed rule suggests, would serve no purpose other than to add unnecessary delay to risk assessments and regulatory action.
Commenter (8771) states that "reducing reliance on defaults" sounds reasonable for a brief second, but why on earth would an Agency want to reduce reliance on sound science? The commenter states that defaults should be discarded when they are wrong, either across-the-board or correct in general but wrong for a specific chemical or application (NAS 1994; Finkel 2003; NAS 2008). The commenter asserts that it is arbitrary for EPA to penalize research that relies on reasonable and time-tested models merely because they are parsimonious and have been used as "defaults." According to the commenter, replacing evidence-based "default" models and assumptions with half-baked exculpatory theories is a recipe for the erosion of science -- and has no place in this proposal. The commenter states that defaults are often simple and were arrived at decades ago -- so were Newton's laws and Maxwell`s equations --  but that has no bearing on whether they are also theoretically correct, logical, and predict risk better than fancier models that look more sophisticated but may be completely unwarranted.
Commenter (6924) supports updating established defaults to account for newer science demonstrating that EPA's typical safety factor of 10 is insufficient to account for variability due to life stage, genetics, underlying disease status, and external stressors that may be due to poverty or other difficult life conditions.
Response: EPA has not finalized proposed 40 CFR 30.6 so there are no requirements specific to dose-response models.
Do Not Support the Use of Default Assumptions
Comment: Commenter (2426) supports the proposed rule language relating to consideration of dose response models and uncertainty, rather than relying on default assumptions.
Commenter (6911) concurs that this proposal should include EPA's actions on exposure-response relationships and model selection. The commenter asserts that, despite the fact that significant progress has been made in the U.S. and internationally to define scientific frameworks and evaluate toxicity datasets to determine biologically plausible modes of action and relevance to humans, the EPA, in practice, has embraced conservative default assumptions, and any movement to modify these approaches has been slow. 
Response: EPA has not finalized proposed 40 CFR 30.6 so there are no requirements specific to dose-response models.
4.1.1.2.8 Other Comments on Models
Comment: Commenter (4541) asserts that the EPA's new policy of scientific integrity and transparency should be applied to computer climate models that currently prevail in EPA's funded, published, and cited climate science. The commenter contends that continued use of default models, without consideration of alternatives or model uncertainty, creates a false scientific justification for EPA actions, policies, and regulatory burdens. The commenter states that arbitrary choices made by EPA officials have resulted in assumptions about model accuracy that have been proven incorrect, unable to recreate past climate history or project future climate changes. According to the commenter, all EPA-adopted models have predicted a much higher rate of temperature sensitivity to CO2 than has been recorded by temperature observations. The commenter adds that such direct proof of model inadequacies cannot be ignored and must be addressed before policy and regulatory actions are proposed.
Response: This rule does not apply to models. See the preamble of the final rule which explains the decision to limit the scope of the rule to dose-response data.
Comment: Commenter (9224) agrees that any data, methodology, or models produced by EPA itself should be transparent and available to the public in a reproducible manner. 
Response: EPA agrees. The purpose of this rule is to increase transparency of dose-response data underlying pivotal science.
Comment: Commenter (6891) states that, when appropriate, EPA should apply transparency principles to data and models that EPA has considered but rejected, particularly when EPA relies on data or models that do not meet stated acceptance criteria for their use. The commenter states that adopting a policy of explaining acceptance and rejection of data or models would be well-supported by EPA's long-held policies of transparency, validity, and public participation. The commenter suggests that EPA make rejected data and models available for public review and evaluation. According to the commenter, by allowing comments on data and models that EPA evaluated but excluded, EPA will further its goals and provide a check on its own assumptions and analyses.
Response: This rule applies narrowly to pivotal science. Applying this rule to all information considered by EPA, which may include thousands of studies, would be unfeasible.
Comment: Commenter (8281-PH20) states that the proposed rule's requirements for specific types of defaults, test methods, dose response models, and/or analysis are not supported by current science. 
Response: This rule does not apply to models or set requirements for test methods of the use of defaults.
Comment: Commenter (6373) is concerned that §30.6 imposes unsustainable burdens on EPA that will inhibit it from timely considering scientific data. The commenter suggests that the requirements of §30.6 will quickly overwhelm EPA staff and result in a backlog of relevant scientific studies. 
Response: EPA has not finalized proposed 40 CFR 30.6. See the preamble of the final rule which explains the decision not to finalize this proposed provision.
4.1.1.3 Publicly Available in a Manner Sufficient for Independent Validation 
4.1.1.3.1 Support/Opposition to Requirement 
Comment: Several comments were received that generally supported or opposed EPA's proposal that data and models underlying scientific studies pivotal to EPA regulations be "publicly available in a manner sufficient for independent validation."
Commenters (0555, 1974, 6129, 6148, 6175, 6176, 6177, 6179, 6366, 6449, 6867, 8281-PH83) express support for EPA's proposal that the data and models underlying scientific studies pivotal to EPA regulations be publicly available in a manner sufficient for independent validation. Reasons provided for their support include:
 Commenter (6176) believes that public access to underlying data will benefit the EPA. The commenter expresses that the public participation in the rulemaking process helps to inform and shape the decisions made by agencies, and EPA staff who may have access to underlying data and methodology will benefit from hearing different views on this data and methodology, including views from other scientists. The contends that even the "experts" could use the public's feedback and the expertise of other researchers and scientists.
 Commenter (6867) asserts that promises to provide other researchers with access to data, methodology, or computational code are not an adequate substitute for public availability. According to the commenters' experience, such promises often go unfulfilled (example provided). Significantly, the commenter states that many uncredentialed members of the public are capable of reviewing and identifying errors in data, methodology, and computational code (provides examples). 
 Commenter (6449) states that increased data and model availability will facilitate and expedite corrections, for example, where changes were made in a model or to data and accompanying text was not updated. The commenter adds that data and model availability will also allow interested parties to determine if alternative data and model choices would substantially change results. 
 Commenter (6175) believes that increased transparency around the assumptions and methods employed in underlying scientific studies will enhance meaningful review and public comment on proposed regulatory actions and decisions. Similarly, commenter (6366) expresses that a more-informed stakeholder community, which has access to such data, will be better suited to participate in the notice and comment process. 
 Commenter (1974) asserts that the Proposal would result in the American public having greater confidence in the policies advance it its (EPA's) name. According to a recent National Academy of Sciences report titled "The Irreproducibility Crisis in Modern Science," the commenter states that the EPA is contrasted unfavorably with agencies like the NIH for lax science standards: "We recommend that other government agencies, especially the Environmental Protection Agency and the Department of Energy, adopt the NIH's new standards." (https://www.nas.org/storage/app/media/Reports/Irreproducibility%20Crisis%20Report/NAS_irreproducibilityReport.pdf, page 50)
 Commenter (6129) notes that their experience with EPA's physiologically based pharmacokinetic (PBPK) model for trichloroethylene (TCE) indicates that replication of this model poses difficulties for even sophisticated users, based on the level of documentation that is currently available to the public in the IRIS Toxicological Review for TCE. Thus, the commenter encourages the EPA to develop a system by which the same models that EPA uses in its risk assessments can also be used by the public for independent validation efforts.
 Commenter (6148) notes that, in some cases, concerns have been raised that data have been "cherry-picked" to achieve a pre-determined outcome. According to the commenter, unless data is publicly available for independent assessment, these concerns are likely to continue. The commenter states that data transparency not only allows interested parties to know what data was used in an analysis, but also what data has been omitted. The commenter contends that the most practical means to alleviate such concerns is to enable interested parties the ability to review and comment on the raw data and resulting analysis that form the basis of EPA decisions. 
 Commenter (6177) asserts that the EPA regulations based on dose response data and models to predict health or environmental impacts that are opaque end up creating unnecessary costs and competitive disadvantages for U.S. companies. The issue for many within industry has been the inability to understand the Agency's underlying analyses and conclusions, as data sets and models often are incomplete or otherwise unavailable for review. According to the commenter, the proposed rule would provide access to the information and data needed for those parties not part of EPA's SAB or peer-review panels to comment more thoroughly on proposed environmental regulations. 
 Commenter (6179) asserts that the Transparency Proposal, when finalized, can enhance public input into the rulemaking process, resulting in better air regulations. The commenter adds that the Transparency Proposal will lead to increases in replicability and verification in the scientific process, thereby allowing the testing of critical methodological assumptions and mitigating biases in key studies that inform significant regulations. The commenter believes that the Proposal recognizes that transparency can be more than simply maximizing disclosure, but rather a means by which studies are contextualized to enhance public participation in the notice-and-comment process critical to effective rulemaking.
 Commenter (0555) asserts that the EPA's proposal to clearly identify available studies and make them available for public review is not only important for ensuring decisions are supported by the best information, but also consistent with policies on scientific integrity espoused by previous administrations (cites the Office of Management and Budget's (OMB's) 2002 Information Quality Guidelines, and a 2009 Memorandum issued by President Obama that encouraged transparency) and scholarly journals (cites Science and Nature). 
 Commenter (0555) states that the selection of the model used to estimate responses to different exposures to contaminants can have significant impacts on estimated regulatory benefits and notes that, in 2007, the Office of Information and Regulatory Affairs (OIRA) and the Office of Science and Technology Policy (OSTP) observed in a memorandum to agency heads on risk assessment that a "high degree of transparency with respect to data, assumptions, and methods will increase the credibility of the risk analysis, and will allow interested individuals, internal and external to the agency, to understand better the technical basis of the analysis."
Response: The EPA agrees with commenters that the opportunity to independently validate the pivotal science that the EPA relies upon is important in furthering scientific understanding and the Agency's mission. The EPA also believes that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety.

Through this rulemaking, the EPA is pursuing an incremental approach to maximizing transparency in the pivotal science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. As described in the preamble of the final rule, the EPA is focusing on the underlying dose-response data for this rulemaking because of the influence these data have on particularly impactful decisions at the Agency. 

Comment: Commenters (0044, 0671, 0980, 1973, 2337, 5017, 6133, 6135, 6168, 6772, 6868, 6939, 9227) express opposition for EPA's proposal that the data and models underlying scientific studies pivotal to EPA regulations be publicly available in a manner sufficient for independent validation. Reasons provided for their opposition include:
 Commenter (0671) contends that researchers are always free to confirm a study by performing their own study, and they often do, and doing so is a much better use of limited scientific resources than reanalyzing old data. The decision to make data public (and which data to make public) should stay between the researcher, the IRB, and journal editors. The best available science should be adjudicated via the peer review process, not by a applying a restrictive and unreasonable data access standard.
 Commenter (6868) notes that federal courts have already considered requests for revealing scientific data used by EPA. The commenter states that the court found these requests lacking in merit. According to the commenter, in the 2002 American Trucking Associations v Environmental Protection Agency heard by the D.C. District Court of Appeals, the court ruled that the CAA imposes no obligation to obtain and make public the data underlying certain "key studies" and similarly rejected arguments that a general requirement be imposed for EPA to obtain and publicize the data underlying published studies on which the agency relies. The court further stated, "requiring agencies to obtain and publicize the data underlying all studies on which they rely would be impractical and unnecessary." The commenter asserts that the court's findings call into question EPA's justification for the proposed rule.
 Commenter (0980) asserts that requiring all data and methodology to be published is completely impractical. The commenter contends that published studies will not be updated to reflect this new requirement, and new studies will only meet these standards if the authors have this requirement in mind. According to the commenter, scientists do not publish their data reduction pipelines, not because they are hiding something, but because there are standard procedures, or because they are easily reproducible by other competent scientists. The commenter states that the use of scientific studies by regulatory agencies needs to align with how scientific studies are used by others in the scientific community. According to the commenter, if a scientist wants to see more data or ask a question about some methodology, they email the author of a study. 
 Commenter (1973) asserts that making data and models publicly available in a manner sufficient for independent validation is not necessary to assure people that the studies have been reasonably conducted because the study protocols, recruitment criteria, measurement techniques, and statistical modeling methods including lists of the adjustment variables have all been made publicly available, and peer reviewed. The commenter adds that tables of descriptive statistics are also already public. Moreover, the commenter asserts that the peer review process of journals means that a group of experts unrelated to the study have already commented on, and requested changes in, methods that they find wanting. Moreover, the commenter states that the scientific community has already adopted several protocols and guidelines specifically designed to improve reporting and evaluation of studies and improve the quality and transparency of the interpretation of findings.
 Commenter (6133) asserts that neither the Proposal nor docket contains any factual, scientific, technical, logical, or legal support for the suggestion that science and data that are "publicly available in a manner sufficient for independent validation" are necessary elements for the "validity," "reliability," or "transparency" of scientific information. The commenter adds that the EPA provides no basis for its assumption that science or studies for which data are publicly available yield more valid or reliable results than the best available, peer-reviewed, independent, credible science, for which the underlying data are not publicly available. Similarly, the commenter notes that the Proposal arbitrarily fails to address, much less explain, why prior EPA regulatory actions that relied upon studies, data, or other information did not reflect the "best available science" or why they were otherwise unreliable, despite failing to meet the Proposal's standards. 
 Commenters (2337, 6914, 6939, 9227) assert that there are many reasons (legal/ethical/practical) why data cannot, or cannot easily, be made public, including data no longer being available (older studies), the size of the study and volume and format of the underlying data, and privacy and individual/business confidentiality concerns. Commenter (9227) asserts that the agency's failure to consider or examine any of the legitimate reasons for not making data publicly available is arbitrary and capricious.
 Commenter (6135) opposes asking scientific researchers to make publicly available administrative claims datasets (e.g., hospital admission and emergency department records, Medicare records, etc.) or other population datasets (e.g., National Health and Nutritional Examination Survey, NIH-AARP Diet and Health Study, etc.) which would not only violate data use and IRB agreements but is wholly unnecessary given that these records are already openly available to any individual or group that adheres to data privacy requirements.
 Commenter (6939) asserts that the Proposal would necessitate the sharing of data protected under health privacy laws and otherwise barred from disclosure and that its main impact would be to undermine science in regulatory decision-making by making it impossible for EPA to consider the best available science when regulating by prohibiting, except in limited circumstances, the consideration of relevant science from the decision-making process if the underlying data is not publicly available. The commenter notes that, in the absence of substantial new resources, efforts to replicate studies and prepare data for public release would reduce the resources available for new research and delay the regulatory process. 
 Commenter (9227) notes that, with respect to studies that have already been completed, there are formidable barriers to public release of underlying data and models, particularly older studies where data is no longer available for a multitude of reasons.
 Commenter (6168) states that requiring full data availability and broad application of default models encourages analyses aiming to sow false uncertainty and put off protective regulation. According to the commenter, in addition to eliminating evidence from the regulatory process, the proposed measures on data and model availability facilitate not transparency but the culture of science denialism, which fits with the anti-protection agendas of fossil fuel and chemical industry lobbyists. The commenter contends that scientific uncertainty is used not only to deny risks, but to make potential benefits of regulation uncertain and to give weight to the costs; such cost-benefit analysis justifies failure to regulate. In particular, the commenter notes that external "validation" -- a central component of this proposed rule -- can instead take the form of "deconstruction of evidence" through "data dredging".
 Commenter (9227) notes that there are legal and ethical requirements that restrict making public the data underlying studies, including rules to shield private personal information, requirements to maintain CBI, situations where obtaining the necessary permissions to release data are logistically difficult or impossible (e.g., historic studies), and situations in which researchers have made significant investments in developing datasets that they intend to continue to work with for future studies. According to the commenter, not all these barriers can be overcome, nor can they be overcome in every case. The commenter notes that, while there are ways to potentially address some of them, they can be extremely costly and burdensome, and/or may harm the prospects for further research. Accordingly, the commenter asserts that, while the scientific community has made efforts to make more data publicly available, to the best of our knowledge all of the policies adopted by government and academic journals recognize that data is not, and need not be, publicly available to evaluate their quality. The agency's failure to consider or examine any of these legitimate reasons for not making data publicly available is arbitrary and capricious.
 Commenter (9227) notes that, even with respect to prospective application of EPA's proposal, providing for public release of underlying data and models is costly and resource intensive, creating a serious disincentive for researchers to meet EPA's proposed requirements. The commenter provides that investigators willing to make their study underlying data publicly available would still face the logistical hurdle of making the data and models available in a manner sufficient for independent validation by the public. In addition to the cost of thoughtful and effective deidentification or redaction of sensitive information, the commenter notes that the proposed text would likely require researchers to prepare annotated manuals including precise detail as to what variables were collected, how information was collected, and the rationale for each step taken. According to the commenter, some manuals alone run into hundreds of pages. The commenter provides that one press account noted the example of publicly available datasets from the National Center for Health Statistics, which can come with 100-page manuals; where researchers would need to hire additional staff to meet such requirements. 
 Commenter (5017) notes that some studies cover populations that are so limited in size or specialized in their characteristics that those data should not be posted on the web for all the world to see.
Response: Based on a consideration of public comments received in response to the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. 
As described in the preamble to the final rule, the transparency provisions of the final rule apply to dose-response data underlying pivotal science. Under this rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule under 40 CFR 30.7.
The requirements included in the final rule provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the EPA maintains a strong scientific basis for its decision-making. The incremental progress made possible by the final rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments.
4.1.1.3.2 Making Data and Models Available Does Not Make Them Understandable to the Public
Comment: Commenters (2169, 6046, 6168) assert that the Proposal to make data and models underlying scientific studies pivotal to EPA regulations be "publicly available in a manner sufficient for independent validation" does not make the studies understandable to the public.
Commenter (6168) states that the requirements of this rule fail to support significant public participation, because data and models are meaningless without explanation. According to the commenter, simply making them available does nothing to make them understandable or usable to the public. The commenter notes that there is already a huge gap at EPA between public availability of information and meaningful accessibility and transparency. The commenter states that this is true not only regarding data availability but also when analysis is made public without adequate explanation. The commenter notes, as an example, the extensive analyses underlying the NAAQS, which regulate the six criteria air pollutants. For each of these pollutants, the commenter states that the EPA synthesizes all relevant scientific information into an Integrated Science Assessment and a Risk or Exposure Assessment and then develops a Policy Assessment based on these reviews and recommendations from an advisory council. The commenter states that these thorough assessments are massive documents -- together running into the thousands of pages for each of the six pollutants -- that are publicly available. The commenter provides that the webpage for each pollutant, however, contains shockingly little information to help users understand and use this highly technical information. For example, according to the commenter, the page on particulate matter contains a single sentence describing "What Scientific and Technical Information Supports Review of the Standards" and a handful of sentences explaining the health effects of particulate matter. The commenter asserts that the EPA provides nothing between this handful of sentences and the thousands of pages in the assessments that would make this sufficient for public education and engagement. The commenter contends that this form of "transparency" falsely equates availability with accessibility. The commenter concludes that, if making analyses available without context fails at transparency and public participation, then making raw data and dose-response models available without context is an even greater failure.
Commenter (6046) notes that very few members of the general public have the statistical expertise or time to properly evaluate epi or pesticide study data and very few probably have any interest. Thus, the commenter asserts that disallowing the use of non-publicized data would not impact the general public. The commenter provides that some scientific researchers may be able to and have interest in evaluating such data. The commenter provides that currently, if EPA researchers use non-publicized data, researchers can request the raw data from the study author. According to the commenter, if the researcher cannot release the data to the requester due to confidential reasons, there is frequently sufficient summary statistics and study methodology reported by EPA that a researcher can replicate and evaluate the study results (e.g. using Monte Carlo methods). 
Commenter (2169) provides that releases of information/models necessary to allow the "public to understand" is not achievable. The commenter notes that public educational attainments are not at a level needed to insure understanding. Alternatively, the commenter asserts that the releases necessary to allow competent scientists, with access to statistical and other tools of scientific assessment, are possible. According to the commenter, the problem presented by the proposed standard of understanding is that simple public confusion becomes a basis for setting aside a regulation. The commenter asks: How is the public confusion to be ascertained? By public surveys? By assessing comments received from the public?
Response: In response to these comments, the EPA has clarified the purpose of the rulemaking in the preamble of the final rule. The purpose of this rulemaking is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. The EPA believes that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety.
Comment: Commenter (6046) notes that the proposed rule will not meet its stated goal of better informing the public. According to the commenter, by disallowing EPA researchers and analysts the use of study data which are not published, the non-published study data, regardless of their quality and applicability, will not be used or discussed by EPA, thus hiding their existence from the public. Thus, the commenter contends that the public would be less informed about the spectrum of data. Further, the commenter adds that disallowing study data would mean that a thorough critical weight of evidence evaluation cannot be conducted. 
Response: Based on a consideration of these comments and other, similar comments received in response to the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. 
As described in the preamble to the final rule, the transparency provisions of the final rule apply to dose-response data underlying pivotal science. Under this rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule under 40 CFR 30.7.
Implementation of this rule will more effectively share pivotal science for external consideration and, thus, increase the opportunity for independent subject matter experts to validate pivotal science should they choose to do so. As the dose-response data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
4.1.1.3.3 What Degree of Public Availability is Sufficient
Proposal Unclear Regarding the Level of Access and Disclosure/Explanation is Needed to be Publicly Available
Comment: Commenters (2426, 6133, 6194, 6362, 6875, 6915) express that the Proposal is not clear regarding what EPA considers to be sufficient public availability and the level of access and disclosure/explanation to the public that is intended. Commenter (6362) suggests that, another important aspect relevant to "public availability" is the level of data refinement EPA will require. 
Commenter (6362) expresses that the EPA does not define what it means by its use of the term, "publicly available." The commenter explains that there is more than one definition of the term currently in use by federal agencies. Commenter (2426) states that, perhaps one of the more frequently used terms that has caused the most confusion is "publicly available." The commenter provides that, outside the context of this proposed rule, "publicly available" generally means that information is available without restriction to anyone who wants it. Commenter (6194) explains that the "public" referenced in the Proposal could be scientists trained in the relevant field or members of the general public. Commenter (6875) notes that a significant number of journals require a subscription or payment in order to read their articles. The commenter asks: Does EPA consider these studies publicly available? Commenter (6362) adds, that if EPA intends to determine this on a case-by-case basis, that also be made clear. 
Commenter (2426) notes that, while making the level of accessibility to information available without restriction to anyone who wants it is acceptable for most information, the commenter notes that it becomes problematic for sensitive personal information, personal health information, or CBI. In order to meet existing privacy and confidentiality laws that protect this type of information, the commenter states that increased accessibility tends to come with reduced refinement (i.e., less detailed information) in the data. 
Commenter (6133) asserts that, although the phrase, "in a manner sufficient for independent validation," is repeated frequently throughout the Proposal, and is integral to its very operation, the phrase is not defined in the proposed definitional section (§30.2). The commenter notes that, later in proposed regulatory text, the Proposal does specify that "[i]nformation is considered `publicly available in a manner sufficient for independent validation' when it includes the information necessary for the public to understand, assess, and replicate findings." 83 Fed. Reg. at 18,773 - 74 (proposed §30.5). The commenter adds that proposed §30.5 goes on to list categories of information that "may" be included in this concept. The commenter notes that the explanation provided by proposed §30.5 is a non-definition; it provides no additional clarification. The commenter asks: How much information is sufficient for the public to understand, assess and replicate findings? Can this standard sometimes be met by releasing methodology but not raw data? 
Commenter (6133) expresses that the proposal fails to explain how the term, "in a manner sufficient for independent validation," and the proposed §30.5 definition will increase transparency in science or why it is necessary to ensure that EPA will consider the best available science. To the contrary, the commenter contends that the Proposal's approach would preclude EPA from considering the best available science that is relevant to EPA's responsibilities. The commenter adds that the EPA also fails to explain why data underlying peer-reviewed studies must be publicly available "in a manner sufficient for independent validation" when independent researchers can verify science without making the underlying data, which is often confidential, publicly available. 
Commenter (6194) states that the Proposal provides that, "where necessary, data would be made available subject to access and use restrictions." 83 Fed. Reg. at 18,773-74. However, the commenter states that what this process would look like is not addressed in the Proposal or discussed at all in the preamble. According to the commenter, this leaves significant open questions, such as who will pay for and implement these restrictions and who will decide which people can access the data and under what standards.
Commenter (6915) states that the "information necessary for the public to understand, assess, and replicate findings" examples provided in the Proposal include "(a) Data (where necessary, data would be made available subject to access and use restrictions)[;] (b) Associated protocols necessary to understand, assess, and extend conclusions; (c) Computer codes and models involved in the creation and analysis of such information; (d) Recorded factual materials; and (e) Detailed descriptions of how to access and use such information." 83 Fed. Reg. at 18,774. The commenter states that this could be interpreted to require maintenance of data down to the most basic level, verging on the absurd, and could impose unreasonable and impractical requirements that go well beyond those already included in current protocols. For example, the commenter notes that it could require maintenance of records that are not routinely archived by academic research labs, such as printouts of data from all calibration curves and analyses from instruments that measure clinical parameters in blood or other similar endpoints in animal and human studies, or photos of each individual organ as it is evaluated for gross pathology in toxicology studies. According to the commenter, even if such data are maintained for a period after a study is completed, it is not feasible for such records to be maintained indefinitely by research laboratories, which would then make the study that the data supports unavailable for use in future regulations. According to the commenter, for toxicology studies, such data availability requirements would result in favoring studies performed under Good Laboratory Practice (GLP) protocols, which typically retain more raw data than research studies. The commenter provides, however, that GLP studies may not evaluate the most sensitive and relevant toxicological effects of the chemical being studied and are not inherently of higher quality than studies conducted under other protocols.
Commenter (0671) cites the Proposal preamble as stating "[t]he benefits of EPA ensuring that dose response data and models underlying pivotal regulatory science are publicly available in a manner sufficient for independent validation are that it will improve the data and scientific quality of the Agency's actions and facilitate expanded data sharing and exploration of key data sets; this is consistent with the conclusions of the National Academies." The commenter states that nothing in that report suggests the EPA should require publication of PII on human subjects. The commenter adds that there is also a problem here with the definition of "data... underlying pivotal regulatory science". The commenter asks: Does this mean the raw data, or the data used to make the final figures and tables in a paper or the data as it exists at some processing step in between? If the raw data, how raw? Some raw genomic data is so large it can't be efficiently stored. The commenter contends that there is no one unique thing called "the data" for a study.
Response: In response to these comments and other, similar comments, the EPA issued a supplemental notice of proposed rulemaking (SNPRM) in 2020 that included definitions for key terms in the 2018 proposal and clarified definitions included in the 2018 proposal. The EPA also proposed a modified approach to the public availability provisions for data and models that would underly significant regulatory decisions and an alternate approach.
Data Availability Usability Certification
Comment: Commenter (0556) states that the EPA needs to specify the requirements of proper data availability as needed for usability. According to the commenter, this might be a simple checklist of questions, or perhaps something more complex. The commenter provides that there is already considerable discussion and action within the open science community, as to just what availability for the purpose of replication requires. The commenter recommends that EPA draw on this existing work and expertise. The commenter notes that the basic requirements are not that mysterious: (1) The data and codes need to be available and properly documented. (2) The procedures followed, and decisions made along the way need to be properly explained. (There is already relevant literature on this topic in some fields, especially biomedical.) (3) Research that cannot be certified is excluded.
Commenter (0556) suggests that the EPA might need to certify that each piece of research it wants to use is properly available for attempted replication. The commenter provides that they believe that the best way to do this is for EPA to contact the researchers to determine that the research is properly available. According to the commenter, another possibility is to establish a practice of self-certification among those researchers who desire that their research be usable by EPA. The commenter adds that certification of EPA usability might also be done by third parties, such as scholarly societies. The commenter states that such certification could even become a condition of funding with some funders. 
Response: As described in the preamble to the final rule, scientific information is considered available in a manner sufficient for independent validation when it includes the information necessary to understand, assess, and reanalyze findings. The EPA has identified the information that may need to be available for dose-response data to be considered publicly available in a manner sufficient for independent validation, including: 
 Data (data would be made available subject to access and use restrictions);
 Associated protocols necessary to understand, assess, and extend conclusions;
 Computer codes and models involved in the creation and analysis of such information;
 Recorded factual materials; and
 Detailed descriptions of how to access and use such information.
As described in the final rule, the EPA intends to clearly identify the studies considered pivotal in the documentation at the proposed rule stage for significant regulatory actions and when influential scientific information is disseminated for peer review. The Agency will also identify the pivotal science that was given lesser consideration and provide a description of why lesser consideration was given.
The EPA would like to emphasize that this final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedure and does not direct or require any outside entity to make dose-response data available or determine that dose-response data are available. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science. 
Necessary Associated Protocols for Making Data and Models Available
Comment: Commenter (2426) recommends that the EPA strongly consider implementing a formal request and approval process for qualified researchers to obtain access to data that is protected by confidentiality laws, rather than aiming for all data to have unrestricted access. The commenter contends that this process would still allow identifiable or refined de-identified data to be re-evaluated by experts external to the EPA or the original study authors, while protecting the privacy of the study participants or businesses who created the model or dataset.
Commenter (6362) states that the National Academies of Science, Engineering, and Medicine (NAS) held a workshop in 2016 to discuss obstacles for sharing data. The commenter provides that the NAS defined several key terms to ensure clarity at the workshop. The commenter recommends that the EPA consider adopting a similar lexicon to increase the clarity of its regulation and refers the EPA to Table 1 of Appendix B of their comment letter for these terms (Table 1. Definitions in NAS Principles and obstacles for sharing data from environmental health research: Workshop summary). In addition, the commenter provides that the NAS Report suggests a "cleaned dataset" would be acceptable to use for all routine analyses and verification. (See Table 2 in Appendix B of their comment letter (Table 2. Data flow from NAS Report)). The commenter recommends that the EPA establish clear standards on the acceptability of "cleaned datasets." The commenter contends that this will help to standardize data reporting and formatting. It will also prevent over- and under-reporting.
Response: Based on a consideration of these comments and other, similar comments received in response to the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. Under this rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule under 40 CFR 30.7.
In the 2020 SNPRM, the EPA presented the different stages of data described in the National Academies of Science, Engineering, and Medicine (NAS) 2016 workshop reported cited by the commenter and proposed a definition for data consistent with cleaned-up data (i.e., the set of recorded factual material commonly accepted in the scientific community as necessary to validate research findings in which obvious errors, such as keystroke or coding errors, have been removed and that is capable of being analyzed by either the original researcher or an independent party).
Computer Codes and Models
Comment: Commenter (6175) that they support the broader proposed requirements to describe and document any modeling assumptions and methods used; evaluate default assumptions; explain the scientific basis for each model assumption used; conduct sensitivity analyses of the modeled results to alternative assumptions; and give consideration to high-quality studies that explore other factors, including confounding variables. The commenter recommends that the EPA also evaluate the effect of variability and uncertainty on results, giving preference to studies that evaluate a range of models.
Commenter (6175) asserts that, while scientific modeling plays an important role in informing policy and regulatory decisions, models need to have a foundation in actual real-world data. The commenter notes that often there can be a significant gap in modeling assumptions and what is really occurring on the ground. The commenter states that disclosure of the underlying data, assumptions and methodology will allow affected stakeholders to provide input that will improve the credibility and reproducibility of modeled outcomes.
Response: As explained in the preamble to the final rule, the EPA is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. Thus, the EPA is focusing on the underlying dose-response data for this rulemaking because of the influence these data have on particularly impactful decisions at the Agency.
While the scope of the final rule is focused on dose-response data underlying pivotal science, the final rule requires the EPA to describe critical assumptions and methods used in the Agency's dose-response assessments and to characterize the variability and uncertainty of such assessments. 
Access to Data
Comment: Commenter (6194) states that the Proposal provides that, "where necessary, data would be made available subject to access and use restrictions." 83 Fed. Reg. at 18,773-74. However, the commenter states that what this process would look like is not addressed in the Proposal or discussed at all in the preamble. According to the commenter, this leaves significant open questions, such as who will pay for and implement these restrictions and who will decide which people can access the data and under what standards.
Response: Based on a consideration of these and other similar comments, the EPA proposed, through a 2020 SNPRM, a modified version of the 2018 proposed rulemaking regulatory text at 40 CFR 30.5. In the 2020 SNPRM, the EPA proposed Agency consideration of studies where there is tiered access to data and models that have CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects. For all other studies, data and models should be publicly available if the studies are to be used as pivotal regulatory science or pivotal science. In the 2020 SNPRM, the EPA also proposed an alternative. Under alternative proposed 40 CFR 30.5, when promulgating significant regulatory decisions or finalizing influential scientific information, the Agency would, other things equal, give greater consideration to studies where the underlying data and models are available in a manner sufficient for independent validation either because the information is publicly available or available through tiered access when the data include CBI, proprietary data, or PII and appropriate techniques have been used to reduce the risk of re-identification.
Based on a consideration of public comments received in response to the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. The EPA would like to emphasize that this final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedure and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenter (6362) provides that several agencies and organizations, in addition to NIH, have successfully addressed the issue of data access while maintaining confidentiality that should be considered by EPA. For example:
 The existing rule requiring federally funded research to be made available to other researchers. This standard could be adopted and applied to third-party funded researchers.
 Health care claims and related data are now being made available to researchers in de-identified form by some health insurance companies, such as Optum, which offers a "proprietary research database of health care and administrative data that links patient, physician, and treatment attributes from millions of geographically diverse individuals in the U.S." Optum appears to have developed methods and procedures to appropriately address confidentiality concerns.
 Medicare claims data are already available to researchers in de-identified form. Algorithms and methods developed by the Center for Medical Services should be examined by the EPA.
 Several professional societies have guidance on the protection of health data and de-identification, such as the Institute of Electrical and Electronic Engineers and the International Association of Privacy Professionals.
Response: Based on a consideration of these and other similar comments, the EPA proposed, through a 2020 SNPRM, a modified version of the 2018 proposed rulemaking regulatory text at 40 CFR 30.5. In the 2020 SNPRM, the EPA proposed Agency consideration of studies where there is tiered access to data and models that have CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects. For all other studies, data and models should be publicly available if the studies are to be used as pivotal regulatory science or pivotal science. In the 2020 SNPRM, the EPA also proposed an alternative. Under alternative proposed 40 CFR 30.5, when promulgating significant regulatory decisions or finalizing influential scientific information, the Agency would, other things equal, give greater consideration to studies where the underlying data and models are available in a manner sufficient for independent validation either because the information is publicly available or available through tiered access when the data include CBI, proprietary data, or PII and appropriate techniques have been used to reduce the risk of re-identification.
Based on a consideration of public comments received in response to the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. The EPA would like to emphasize that this final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedure and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenter (2171) supports open data, and that their state is a national leader in scientific and regulatory transparency. The commenter notes that their agencies are at the forefront of making environmental and health surveillance data publicly available, providing technical assistance for using that data, and engaging partners across communities and research institutions around effective data dissemination and utilization. The commenter adds that their agencies host multiple platforms for accessing high-quality health surveillance and environmental monitoring data, while protecting privacy and providing essential risk communication and prevention strategies. According to the commenter, detailed data are similarly available for research uses, under the approval and guidance of state IRBs.
Response: Based on a consideration of public comments received in response to the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. The EPA would like to emphasize that this final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedure and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenter (6726) states that one way to ensure transparency in the regulatory science aimed at solving a problem affecting our civilization is to fund two or more independent studies of the problem if questions arise about a single peer-reviewed study. According to the commenter, such questions do not often arise, but if they do it is an excellent opportunity for the authors of the original study to submit additions and corrections for peer review, as well as for others to independently study the problem. The commenter notes that everybody makes mistakes, but scientists and engineers are trained to recognize, avoid, and correct systematic errors in their experiments. The commenter provides that a peer-reviewed study usually contains a list of assumptions and approximations and an analysis of random and systematic errors to aid others in repeating the experiment and analyzing their own data. 
Response: The EPA agrees with commenters that the opportunity to independently validate the pivotal science that the EPA relies upon is important in furthering scientific understanding and the Agency's mission. Through this rulemaking, the EPA is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. 
As described in the preamble of the final rule, the EPA is focusing on the underlying dose-response data for this rulemaking because of the influence these data have on particularly impactful decisions at the Agency. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information.
Comment: Commenter (5419) states that the proposed rule mentions data research centers that facilitate secondary data usage by the public. The commenter suggests that, if the Centers for Medicare & Medicaid Services (CMS) data are considered to fall in this category, they would encourage EPA to work with CMS to streamline and simplify the data acquisition process and make the data available to the public free of charge. Otherwise, according to the commenter, the cost and security requirements of obtaining these data may place an undue burden on stakeholders seeking to replicate analyses, since the use of this data for a project they conducted were acquired at a significant cost (> $100,000) and a comprehensive data management plan that outlined rigorous security protocols was required. The commenter asserts that, clearly, these requirements would make replication of the work untenable for many researchers. The commenter expresses concerns that the challenges of making these data available in an equitable fashion may result in the data and insights from this study being removed from consideration in standard‐setting activities for PM2.5, ozone, and NO2.
Response: The EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. The EPA would like to emphasize that this final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedure and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Commitment to Development of Permanent Long-Term Storage
Comment: Commenter (0045) asserts that a foundation for open and repeatable science is the availability of original data sets. The commenter provides that the availability of original data is often retained in government repositories, both in the USA and internationally. The commenter suggests that it should be a basic principle of the proposed rule that the U.S. government commit to the permanent long term storage of all data collected under government funding and the establishing of open access to such data, with the proper protections for data which may be subject to legal controls such as commercial confidentiality or patient HIPAA data. The commenter recommends that effective access be implemented to permit peer review, reproducible tests, or new work, based upon enabling persons to be authorized as broadly as possible, and best practices of anonymizing data. The commenter suggests that the USA work with governments around the world to obtain similar safeguarding of fundamental data which may be of universal benefits, with reciprocal access arrangements. The commenter adds that where EPA receives data, studies, analysis, and projections from sources beyond government funded science (e.g., industry, private citizens, or foreign entities), such data, studies, and projections should be subject to the same regime of durable storage, access for peer review, independent reproducibility, and safeguards, which are accorded to government funded sources and studies.
Response: The EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. The EPA would like to emphasize that this final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedure and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
4.1.1.3.4 "Independent Validation" 
Independent Validation is Not Described/Defined
Comment: Commenters (6168, 6185, 6449, 6955) assert that "independent validation" is not described/defined. Commenter (6373) notes that the impacts of the proposed Rule are difficult to predict unless EPA better defines what data the Rule applied to, and terms such as "independent validation."
Commenter (6955) states that validation usually requires a comparison to new data; it is rarely used in scientific studies to mean a check to see that data leads to the stated results using the stated methods and/or software. The commenter notes that new studies can validate a model or numerical finding, but reproducing the analysis only validates the math. The commenter adds that neither can sensitivity analysis validate a study. The commenter provides that the meaning of the phrase, "sufficient for independent validation" is unclear and likely to raise great confusion, as different stakeholders define it differently. What would invalidate a study? We are not told. Furthermore, the commenter adds that no details are given as to what would constitute sufficiency. The commenter contends that, by not explicitly identifying its definition of validation, the agency has skipped over the statistical and regulatory implications of post-hoc analyses, losing the public comments that would otherwise have been forthcoming.
Commenter (6185) states that the approach for "independent validation" is not described. The commenter asks: Would EPA's staff scientists conduct these validations, or would EPA contract this out to third parties? How would EPA determine that third parties have the necessary qualifications and resources to perform these validations? According to the commenter, without these listed information elements, commenters are left to guess at the scope and potential impact of EPA's proposal as it applies to "independent validation." For example, the commenter provides that a recent study that is likely to be considered "pivotal regulatory science" for the current particulate matter (PM) NAAQS review is by Di, et al., "Air pollution and mortality in the Medicare population." The commenter notes that this study of chronic effects, along with a complementary study on acute effects, uses 460,310,521 person years of follow-up and has billions of data points. According to the commenter, the computational resources necessary to replicate or validate the analysis are available only at large institutions like the Harvard-MIT Data Center. The commenter asks: How does EPA's proposal on independent validation apply to a study like this? 
Commenter (6168) states that, while the proposed rule emphasizes that data and models should be available in a manner sufficient for independent validation, the rule fails to clearly define independent validation. As a result, the commenter suggests that it is unclear if the rule applies only to replication of analysis or also to replication of entire studies. According to the commenter, by emphasizing availability of data and models, the rule seems to be narrowly focused on replicating analysis, particularly as related to dose-response relationships. Yet, the commenter notes that wording such as "using scientific information that can be independently validated" [FR 18770] can be interpreted expansively so that the rule can appear to require the ability to replicate entire studies, as was the intent of the failed HONEST Act. The commenter asserts that there are several problems with this. First, the commenter provides that if the intention is to require the ability to replicate entire studies, then this proposed rule is misleading; as written, it seems to be only about replicating analysis. As a result, the commenter states that this proposed rule is a stealth-- not at all transparent-- way to eliminate the use of a wide swath of studies that are currently used for regulatory decision-making. For example, the commenter suggests that studies based on tragic accidents involving toxic releases provide important public health knowledge, but cannot be replicated (for more on this, see EDGI's white paper). Second and more broadly, the commenter provides that the lack of clarity on this matter would allow the EPA to pick and choose, in an arbitrary manner, which studies to include or exclude in risk analysis. The commenter contends that this misuse of power would easily influence the findings of risk analysis. 
Similarly, commenter (6449) states that the definition of "independent validation" is not clear. The commenter adds that it is not clear if the intent is to reproduce study conclusions using underlying data, replicate entire studies, or validate the use of data extracted from published studies in a dose response model. The commenter also notes that it is not clear whether this means all the information required to replicate the entire study should be included or whether this is limited to data once generated. The commenter requests that EPA present a clear definition.
Response: In response to these comments and other similar comments, the EPA issued a supplemental notice of proposed rulemaking (SNPRM) in 2020 that included definitions for key terms in the 2018 proposal, such as "independent validation," and clarified definitions included in the 2018 proposal.
Based on comments received on the 2018 proposed rule and the 2020 SNPRM, the EPA finalized the definition of "independent validation" as "the reanalysis of study dose-response data by subject matter experts who have not contributed to the development of the study to evaluate whether results similar to those reported in the study are produced."
Comment: Commenters (6114, 6116, 6131) state that the proposed rule states that "... EPA should ensure that the data underlying those (studies) are publicly available in a manner sufficient for independent validation," but the proposal does not say by whom. The commenters assert that scientists who are familiar with these types of studies are also familiar with the protocols under which these studies are carried out and peer reviewed, ensuring that an independent, critical assessment is performed. The commenters provide that scientists of this caliber and experience understand the assumptions and limitations that have been made during the study and are qualified to make judgments of whether the studies are valid or not. The commenters contend that, if the EPA seeks "independent validation"' by those who are not familiar with these protocols or with how these studies are normally conducted, that type of validation may be meaningless.
Response: In response to these comments and other similar comments, the EPA issued a supplemental notice of proposed rulemaking (SNPRM) in 2020 that included definitions for key terms in the 2018 proposal, such as "independent validation," and clarified definitions included in the 2018 proposal.
Based on comments received on the 2018 proposed rule and the 2020 SNPRM, the EPA finalized the definition of "independent validation" as "the reanalysis of study dose-response data by subject matter experts who have not contributed to the development of the study to evaluate whether results similar to those reported in the study are produced."
Independent Validation of Studies/Data Submitted to EPA by the Public
Comment: Commenter (6133) asserts that it is possible (even likely) that studies or data submitted by the public during comment periods would need to be independently assessed/validated before consideration by EPA. According to the commenter, against the backdrop of EPA rulemakings with public comment periods and open rulemaking time periods and the voluminous amounts of data that would need to be de-identified, shared, and re-analyzed, it would be impossible to achieve independent verification of relevant dose-response information.
Response: The EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. The EPA would like to emphasize that this final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedure and does not direct or require any outside entity or the EPA to independently validate dose-response data underlying pivotal science. 
Independent Validation by Inspection
Comment: Commenter (0044) states that, if the Agency is so concerned about transparency and validity of studies, they may well copy the Food and Drug Administration (FDA) approach to validating data. The commenter provides that, in human research and marketing applications, manufacturers must submit all clinical (i.e. human) data, even for studies that don't meet the statutory requirement of "adequate and well-controlled." The commenter notes that all studies and data are used in the evaluation and decision-making processes. The commenter adds that data are validated by inspection. The commenter further adds that the FDA has an inspectorate that inspects not only clinical data (especially so-called pivotal data), but also preclinical data and manufacturing data. According to the commenter, the results of these inspections are used to determine the quality and validity of the data and thereby the studies utilizing these data.
Response: The EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. The EPA would like to emphasize that this final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedure and does not direct or require any outside entity or the EPA to independently validate dose-response data underlying pivotal science.
Independent Verification Where Laws and Regulations Prohibit the Public Release of Data
Comment: Commenter (6831) states that, throughout their professional career, they dealt with laws and regulations that prohibited the public release of data that could identify an individual or individual business. The commenter provides that they appreciate the need for independent verifications, but there should never be a restriction on the science used to support regulations, especially those involving human or environmental health. The commenter asserts that there are work arounds that can achieve independent verifications, as needed: release of aggregate data, for example, or individuals needing access can agree to and sign confidentiality agreements to access the data, still with the same restrictions on them for publishing individual data. The commenter states that the EPA must rely on relevant science and work with researchers for independent verification of data/results as needed. According to the commenter, the proposed rule is NOT needed and should be abandoned.
Response: Based on a consideration of these and other similar comments, the EPA proposed, through a 2020 SNPRM, a modified version of the 2018 proposed rulemaking regulatory text at 40 CFR 30.5. In the 2020 SNPRM, the EPA proposed Agency consideration of studies where there is tiered access to data and models that have CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects. For all other studies, data and models should be publicly available if the studies are to be used as pivotal regulatory science or pivotal science. In the 2020 SNPRM, the EPA also proposed an alternative. Under alternative proposed 40 CFR 30.5, when promulgating significant regulatory decisions or finalizing influential scientific information, the Agency would, other things equal, give greater consideration to studies where the underlying data and models are available in a manner sufficient for independent validation either because the information is publicly available or available through tiered access when the data include CBI, proprietary data, or PII and appropriate techniques have been used to reduce the risk of re-identification.
Based on a consideration of public comments received in response to the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. 
As described in the preamble to the final rule, the transparency provisions of the final rule apply to dose-response data underlying pivotal science. Under this rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule under 40 CFR 30.7.
The requirements included in the final rule provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the EPA maintains a strong scientific basis for its decision-making. The incremental progress made possible by the final rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments.
No Requirement that Data Be Independently Validated or Replicated Before EPA Can Consider It
Comment: Commenter (6133) states that there is nothing in the proposed regulatory text or preambular language that requires information, science or data to be independently validated or replicated before EPA may consider it. The commenter contends that the EPA does not base the Proposal upon any requirement or expectation that the information, science or data be shown to be accurate, trustworthy, reliable or correct before EPA may consider it. The commenter asserts that this portion of the Proposal reveals EPA's unlawful agenda to be one concerned with prohibiting EPA from considering relevant, peer-reviewed, quality science, not one concerned with actual replication or validation. The commenter adds that the Proposal's condition that science and information be "publicly available in a manner sufficient for independent validation and replication" is revealed to be mere smokescreen for an EPA enterprise to censor the best available science that would support adoption of more protective health and environmental safeguards. 
Response: The EPA believes that the opportunity to independently validate the pivotal science that the EPA relies upon is important in furthering scientific understanding and the Agency's mission. Through this rulemaking, the EPA is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. 
As described in the preamble to the final rule, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. The EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information.
The requirements included in the final rule provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the EPA maintains a strong scientific basis for its decision-making. The incremental progress made possible by the final rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments.
4.1.1.3.5 Assertion that EPA's Proposal is Consistent with the Conclusions of the National Academies (NAS)
Comment: Commenter (6133) cites the proposal preamble as stating that "[t]he benefits of EPA ensuring that dose response data and models underlying pivotal regulatory science are publicly available in a manner sufficient for independent validation are that it will improve the data and scientific quality of the Agency's actions and facilitate expanded data sharing and exploration of key data sets; this is consistent with the conclusions of the National Academies." 83 Fed. Reg. at 18,772 (footnote omitted). The commenter provides that the last statement links to a 120 page document titled "Expanding Access to Research Data Reconciling Risks and Opportunities," by the Panel on Data Access for Research Purposes, Committee on National Statistics, Division of Behavioral and Social Sciences and Education of the National Research Council of the National Academies Press. (https://www.nap.edu/catalog/11434/expanding-access-toresearch-data-reconciling-risks-and-opportunities). The commenter provides that the Proposal does not suggest that its plan to preclude the use of scientific studies from regulatory decision making is consistent or supported by the National Academies. Rather, the commenter notes that the Proposal generally states that benefits of data availability the Proposal seeks is consistent with conclusions of the National Academies. The commenter further states that the Proposal does not say what the conclusions of the National Academies are or how they support the Proposal. According to the commenter, the charge to the Panel in the cited document was "to assess competing approaches to promoting exploitation of the research potential of microdata -- particularly linked longitudinal microdata -- while preserving respondent confidentiality." According to the commenter, the Panel was asked to consider the tradeoffs between the benefits and risks of data access and to make recommendations about "how microdata should optimally (from a societal standpoint) be made available to researchers." The commenter explains that the Panel offered various recommendations, focused on agencies that have data-collection responsibilities providing data to researchers. The commenter adds that this is a different context than EPA's proposal to preclude the consideration scientific studies when undertaking its statutorily required decision making to protect human health and the environment. The commenter asserts that the EPA's general citation to this 120-page document for consistent conclusions does not support the Proposal. 
Response: The EPA agrees that this was an incorrect citation and has removed the reference to the document titled "Expanding Access to Research Data Reconciling Risks and Opportunities."
4.1.1.3.6 Other
EPA Should Work with Entities Where Scientific Data are not Publicly Available in a Manner Sufficient for Independent Evaluation
Comment: Commenter (6362) recommends that where data are not available in a manner sufficient for independent evaluation, EPA attempt to work with data owners to reach an agreement to make the information available to the public to the greatest extent practicable without jeopardizing the privacy, confidentiality, or the proprietary interests that deserve protection. In circumstances where there is significant difficulty making data available in a meaningful way, the commenter suggests that the EPA consider contracting with external experts in the scientific discipline at issue, have them sign confidentiality agreements, analyze the data, and prepare a confidential report with a non-confidential summary for EPA to share publicly.
Response: The EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. The EPA would like to emphasize that this final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedure and does not direct or require any outside entity or the EPA to independently validate dose-response data underlying pivotal science.
Proposal Not as Bad as Either Side Perceives 
Comment: Commenter (0844) provides that the George Washington University Director of Regulatory Studies Susan Dudley wrote in a May 21, 2017 Forbes.com article, "Critics and Supporters of EPA's Transparent Science Proposal: File Comments On It, But Read It First," that the EPA transparency proposal "is not as dramatic as either side [of the political divide] would lead one to believe, however. Essentially, it requires that the studies, models, and data underlying significant regulatory decisions are `publicly available in a manner sufficient for independent validation.' Independent validation is the essence of the scientific method, which depends on falsifiable hypotheses, data gathering, replication, dissent, and challenge."
Response: The EPA agrees that the opportunity to independently validate the pivotal science that the EPA relies upon is important in furthering scientific understanding and the Agency's mission. The EPA also believes that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety.
Through this rulemaking, the EPA is pursuing an incremental approach to maximizing transparency in the pivotal science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. As described in the preamble of the final rule, the EPA is focusing on the underlying dose-response data for this rulemaking because of the influence these data have on particularly impactful decisions at the Agency.
Comment: Commenter (11393) expresses concern regarding the rule's potential impacts on health risk assessments. The commenter specifically expresses opposition to language that removes or dilutes the linear no-threshold (LNT) dose response in the evaluation or establishment of radiation health risk assessments, rule-making or establishment of radioactive recommended action levels or benchmarks (influential scientific information) made available to the public.
Response: After considering comments on both the 2018 proposed rule and the 2020 SNPRM, the EPA has determined that this rule should apply to dose-response data rather than dose-response data and models. Therefore, the rule does not include discussion of any particular dose-response model. Given the specificity of 40 CFR 30.6 to dose-response data and models, and in particular dose-response models, the EPA is not finalizing 40 CFR 30.6. The EPA is adapting one provision of 40 CFR 30.6 as a factor in 40 CFR 30.5 in determining the consideration to afford pivotal science for which the dose-response data are not available for independent validation. Specifically, the EPA is finalizing as a factor in 40 CFR 30.5 the consideration that the EPA would give to high quality studies that explore a broad class of parametric dose-response models, non-parametric models that incorporate fewer assumptions, various threshold models, and models that investigate factors that might account for spatial heterogeneity.
Comment: Commenter (11492) indicates that the EPA continues to assign greater value to dose-response studies, which by definition reduce the use/value of epidemiological studies. Section 30.6 of the supplemental proposal has been revised but continues to give "explicit consideration to ... parametric dose-response or concentration-response models." As the commenter suggests they stated in its 2018 comments, the EPA itself admits that, "there is frequently a lack of dose-response data available for human subjects. When data are available, they often cover only a portion of the possible range of the dose-response relationship." The commenter states that this gives more weight to toxicological studies as opposed to epidemiological studies, which form the basis of many human health exposure studies. The commenter asserts that the EPA must continue to use epidemiological studies; to do otherwise is to compromise the health of all Americans. Commenter (12403) concurs, indicating that the SNPRM's identification of specific dose-response modeling techniques that would be given "explicit consideration" is inappropriate and ignores the improvements and refinements that are the hallmark of good science.
Response: Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science is unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that are unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator, as detailed under 40 CFR 30.7.
The EPA disagrees that the rule implies a bias against the use of epidemiology studies or data. The final rule includes sufficient flexibility in terms of the allowance for restricted access, considerations in 40 CFR 30.5, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data to minimize such potential unintended consequences. The EPA will continue to consider and evaluate all relevant and appropriate science, including epidemiology studies, in its significant regulatory actions and influential scientific information.
In addition, after considering comments on both the 2018 proposed rule and the 2020 SNPRM, the EPA has determined that this rule should apply to dose-response data rather than dose-response data and models. Therefore, the rule does not include discussion of any particular dose-response model.
Comment: In support of existing model availability, commenter (11894) provides the specific example of the trichloroethylene (TCE) case study. The commenter notes that the TCE assessment relied on approximately 15 models. The commenter notes that this is likely to be the norm with the same ones being used repeatedly. The commenter adds that most of the models EPA employs in the Toxic Substances Control Act (TSCA) program have been developed by the Agency and are publicly-available. The commenter provides that access can be gained through a number of EPA webpages and/or searches on the internet, and others are described in the peer-reviewed literature, access to which is controlled by the authors.
Response: After considering comments on both the 2018 proposed rule and the 2020 SNPRM, the EPA has determined that this rule should apply to dose-response data rather than dose-response data and models. Therefore, the rule does not include discussion of any particular dose-response model.
Comment: Commenter (12465) states that the EPA relies upon many types of data and models to support its regulatory decisions and it is unclear that the current proposal has adequately considered the numerous types. The commenter expresses concern that the treatment of existing data or models under the proposals disincentivize the generation of new data under the new requirements while reducing the importance of consideration of existing data, regardless of the scientific validity or merit of that data. 
The commenter (12465) provides the example of the Westat Survey. The commenter notes that the report entitled "Household Solvent Products: A National Usage Survey" (or the Westat Survey) is a survey of consumer usage for a wide variety of products to determine various attributes and inform exposure assessments. According to the commenter, while the report was completed in 1987, it remains in use today informing the EPA's Consumer Exposure Model and the Westat Survey informed several of the first 10 chemicals currently being evaluated under reformed TSCA. The commenter states that it is unclear in the proposal how the Westat Survey would be treated, let alone the multitude of exposure models that rely upon it.
Response: As further described in the preamble to the final rule and the overarching response to this chapter, the EPA is not finalizing the definition of "model." The rule only applies to dose-response data and will not impact the Agency's use or reliance on models, including exposure models. Since the Westat Survey is a consumer behavior survey for assessing exposure, it would also not be applicable to this rule.
The EPA disagrees that the rule disincentivize the generation of data. The final rule includes sufficient flexibility in terms of the allowance for restricted access, considerations in 40 CFR 30.5, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data to minimize such potential unintended consequences. The EPA will continue to consider and evaluate all relevant and appropriate science in its significant regulatory actions and influential scientific information. 
4.1.1.4 Applicability of Requirements to Other Types of Data and Information
4.1.1.4.1 Arbitrary Rationale for Limiting Applicability to Dose Response Data and Models
Comment: Commenter (9227) questions why EPA has singled out dose response studies to be excluded if their underlying data and models are not publicly available and has not similarly targeted other types of studies commonly used by EPA. According to the commenter, EPA has provided no justification for targeting dose response studies and not other types of studies or scientific information. The commenter asserts that, absent any explanation from the Agency, it is impossible to comment on the factual predicates for EPA's proposed decision, or the reasonableness of EPA's justification, except to state that it appears to be completely arbitrary in the absence of any rationale. See, e.g., Transactive Corp. v. United States, 91 F.3d 232, 237 (D.C. Cir. 1996) ("A long line of precedent has established that an agency action is arbitrary when the agency offered insufficient reasons for treating similar situations differently.").
Response: In this final rule, the EPA will be considering the availability of dose-response data underlying pivotal science, not dose-response data and models. The EPA is focusing on the underlying dose-response data for this rulemaking because of the influence these data have on particularly impactful decisions at the Agency. Risk assessments and regulations that target emissions and risk reduction of one or more pollutants, contaminants, or substances are integral to the Agency's mission and the underlying dose-response data that inform the quantitative value used to evaluate and mitigate potential risk are critical to understanding the assessment or regulatory action. In addition, the data underlying the dose-response assessment are more distinct than the broad range of data informing an entire risk assessment. Therefore, the EPA is concentrating its current efforts to increase transparency on a well-defined step in the quantitative assessment of risk supporting specific Agency actions. This final rule provides an important step in furthering the progress already being made toward maximizing transparency and will provide important insight for developing future statute-specific requirements.
4.1.1.4.2 Support Applicability to Other Data and Models
Clarification on Whether or How Proposed Rule Would Apply to Cost-Benefit Determinations
Comment: Commenter (6915) states that the EPA seeks comment as to "whether the disclosure requirements . . . should be expanded to cover other types of data and information, such as, for example, economic and environmental impact data and models that are designed to predict the costs, benefits, market impacts and/or environmental impacts of specific regulatory interventions." 83 Fed. Reg. at 18,772 (emphasis added). However, the commenter adds that the EPA also states that the "pivotal regulatory science" to which the Proposed Rule would already apply includes "studies, models, and analyses that drive the magnitude of the benefit-cost calculation." Id. at 18,770. According to the commenter, it is thus unclear whether and how EPA intends the Proposed Rule to apply to the cost-benefit determinations that it performs.
Response: The EPA acknowledges the lack of clarity in the proposed rule and emphasizes that this final rule only applies to does-response data. Rather than having this final rule apply to all the studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact, the EPA is focusing on those studies that describe the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect. Specifically, the scope of dose-response data in this final rule is those studies consisting of the data integral to characterizing dose-response relationships. These dose-response studies that constitute pivotal science may be used to identify candidate PODs that are then employed in the benefit-cost analysis, but not all studies informing the benefit-cost calculation would be considered pivotal science.
General Support to Applying the Rule to Other Data and Models
Comment: Commenters (0844, 4541, 5185, 6141, 6150, 6161, 6182, 6360, 6375, 6867, 6913, 6915, 6917, 6921, 8771) suggest that the Agency uses other data and models (in addition to dose response data and models) to support regulatory decisions that would benefit from an improved regulatory framework that enhances transparency with respect to underlying pivotal data and assumptions.
Commenters (0844, 4541, 6150, 6182, 6375, 6921) state that the Proposal disclosure requirements applicable to dose response data and models should be expanded to cover other types of data, such as economic and environmental impact data and models that are designed to predict the costs, benefits, market impacts and/or environmental effects of specific regulatory interventions on complex economic or environmental systems. Commenter (0844) believes it is imperative that this type of information should be applicable to EPA's proposed transparency requirements, considering the enormous economic effect EPA regulations can have on the U.S. economy. 
Response: The EPA has considered applying this rule to all data and models and proposed to do so in the 2020 supplemental notice of proposed rulemaking. In this final rule, however, rather than considering all studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact as "pivotal science," the EPA is balancing transparency and feasibility by focusing on those studies that inform the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect. Thus, "pivotal science" includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). These are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. In the context of this rule, studies informing the dose-response modeling (e.g., benchmark response selection), studies identifying data for toxicokinetic adjustments, or studies informing the selection of uncertainty factors do not drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. Future statute-specific rulemakings may interpret "pivotal science" more broadly.
Support Application to Economic and Environmental Data
Comment: Commenter (6921) notes that economic and environmental data are traditionally more available and generally present less of a transparency problem. However, the commenter states that, to the extent that data and models used to estimate economic or environmental impacts are not generally available, they support including them in this rulemaking if it can be done without delaying the promulgation and implementation of the rule.
Commenter (6141) notes that when the Agency employs models, such as those used in the Social Cost of Carbon (SCC) calculation (especially with respect to climate damages), to support its regulatory decisions, the EPA should at a minimum provide documentation that facilitates peer review of its analysis as well as the assumptions used to support that analysis. According to the commenter, data that models rely upon to project damages may suffer from the same peer review problems as with dose response data, leading to false or dubious conclusions that are then carried forward into regulatory requirements. The commenter adds that, in other cases, the underlying data may be out of date and unreliable for generalizations and decision-making purposes. The commenter requests that the EPA ensure that modeled projections are appropriately characterized. As an example, the commenter notes that mathematical modelling often gives the impression of precision and tempered complexity, in spite of significant and potentially problematic uncertainties involved in the calculations. According to the commenter, not disclosing the uncertainties, or performing robust sensitivity analyses on them, may obfuscate shortcomings of the modelled results. The commenter suggests that an improved regulatory framework that enhances transparency and deliberation with respect to underlying pivotal data and assumptions will improve the regulatory process by increasing confidence that the rules accurately reflect and appropriately address risks to human health and the environment.
Commenter (5185) states that EPA's benefit monetization is highly subjective and controversial and that they hoped that EPA concurrently examines transparency associated with the development of regulatory cost-benefit analyses. The commenter provides that data compiled by the U.S. Chamber found that, between 2000 and 2016, EPA issued 62 rules claiming a total of $923 billion in regulatory benefits. Incredibly, according to the commenter, $898 billion of these benefits -- or 97.2 percent  -- were monetized based on non-public data associated with fine particles (PM2.5). The commenter states that these benefits comprise nearly 80 percent of all regulatory benefits across the entire federal government. The commenter notes that, even though the vast majority of these rules were not intended to address PM2.5, and even though the vast majority of their corresponding claimed benefits came from areas of the country that are already deemed safe and in compliance with EPA's PM2.5 standard, the Agency repeatedly touted these figures to build public support for its regulations. The commenter contends that federal rules that impact millions of people and billions of dollars should be held to a higher standard.
Commenter (8771) asserts that the economists' analysis of the costs of regulation and the values of longevity and other benefits underlying EPA regulations are flawed, opaque, and in need of reanalysis. The commenter believes that every criticism leveled in this proposal against risk science (epidemiology, toxicology, exposure assessment) ought to be first applied to regulatory economics. The commenter notes that the EPA defines "pivotal regulatory science" as results that determine the final regulatory standard, and that costs are as "pivotal" or more "pivotal" as estimates of risk since EPA uses informal "bright lines" to reject regulatory controls if they are "too expensive" or "economically infeasible". The commenter states that regulatory cost estimates are notoriously biased high for reasons of self-interest but also due to inevitable factors such as economies of scale and technological learning. The commenter adds that cost estimates are often surrounded by more uncertainty than that surrounding risk estimates but are rarely if ever presented with error bars. According to the commenter, nothing could be more uncertain than a prediction whose very sign is incorrect (e.g., a prediction of job losses when in fact jobs will be created, or vice versa; a prediction of dire costs when in truth there will be cost savings) -- and yet, the commenter complains that not a word in the Proposal requires any additional transparency, disclosure, reanalysis, or sensitivity analysis be done by economists or by the industry or Agency consultants who prepare such estimates.
Commenter (8771) offers two economic study examples that they believe illustrate the need of repair: 
      First, in a new paper, Branden Johnson and I [the commenter] analyzed the data reported from more than 1,000 estimates of the VSL (value of a statistical life) derived from hundreds of stated-preference studies worldwide. Only 42 percent of the estimates were presented with any information about the standard deviations or ranges of the individual VSL values given by subjects questioned in the study --  so in the majority of cases, EPA has had to derive a value for the VSL (certainly the single most "pivotal" quantity in all of risk regulation!) with no information other than the average VSL among dozens or hundreds of persons surveyed. No one can reanalyze most of these studies to see what higher or lower values of the VSL are also completely compatible with the data, because the economists fail to provide even a summary of the variability in the individual data.
      Perhaps the most well-known "study" of the aggregate costs of regulation is the (series of) reports from Mark and Nicole Crain suggesting that regulations "cost" the U.S. nearly $2 trillion per year. This work is of no real value, since it relies on comparing the GDP [gross domestic product] growth rates of different countries to a subjective index of "regulatory quality," which of course has no relationship to regulatory burden. And I hope needless to say, even a robust correlation between GDP and actual burden is not useful for estimating "cost." But a recent article details how a researcher asked the Crains for their data and data sources so he could see if their results, such as they were, would hold up after controlling for important differences among nations. The requester received no information: "Crain and Crain have refused to publish or share the data supporting their 2014 study, despite numerous requests from this author. In so doing they follow the course they adopted with their 2010 study, in which they refused to divulge their data even to the Government Accountability Office".
The commenter (8771) concludes that EPA must require all significant economic analyses to comply with whatever strictures remain in the final rule (assuming it is not withdrawn); otherwise, the commenter believes that the final rule cannot withstand judicial review, because the essence of arbitrariness is putting hurdles in front of only half of each analysis and ignoring the other half. 
Commenters (6161, 6360, 6867) support EPA expanding the Proposed Rule to encompass all science. Commenter (6360) specifically points out that the EPA commonly employs fate-and-transport models, route of exposure models, cost estimation models, statistical techniques to project analytic data beyond method detection limits, and numerous other specific applications of science that involve judgement by EPA and by the application developers. According to the commenter, they see no logical principle to require transparency for one application and not another. As an example, the commenter notes that a fate and transport model could have as much economic impact as a dose-response model. The commenter notes that the importance of this principle is highlighted by EPA's current proceedings within the U.S. Court of Appeals for the D.C. Circuit where the judges are unable to follow EPA's statistical methodology for calculating the best performing sources for a MACT standard. The commenter provides that this case is instructive since it shows (1) the lack of transparency in other scientific applications besides dose-response models; (2) the cost in social resources when EPA fails to be transparent; and, (3) the risks that the lack of transparency may delay measures to protect human health and the environment. Commenter (6867) asserts that the replication crisis and public confidence rationales that underlie the Proposed Rule suggest that data, methodology, and computational code should be disclosed for all science relied upon to support Agency actions. The commenter recommends, as a first step, the phrase "dose response data and models" should be replaced with "data and models, including dose response data and models" throughout the Proposed Rule.
Commenter (6913) provides that the Proposed Rule places emphasis on dose-response data and models that provide the foundation for "pivotal regulatory science". The commenter expresses that this implies that the transparency rule may only apply to human health effects under the environmental statutes. According to the commenter, EPA's criteria, whether developed for protection of human health, aquatic health, agriculture, livestock health, or some other reason, has the same enforceable impact on industry, states, and the public. According to the commenter, there is no reason that a standard of scientific transparency should not apply to all EPA actions, including criteria developed for any environmental media (water, air, soil, etc.) and protection of all endpoints (human health, aquatic life, etc.). The commenter asserts that the EPA must implement the same transparency standard towards all their criteria derivation actions, regardless of whether they are protective of human health or specific aspects of the environment.
Commenter (6366) asserts that recent EPA action on the retrospective review of the Lead; Renovation, Repair and Painting (RRP) Program under Section 610 of the Regulatory Flexibility Act (EPA-HQ-OPPT-2016-0126) reinforces the need for the proposal. The commenter expresses that they have repeatedly called on EPA to perform an updated economic analysis of the program. According to the commenter, in finalizing the 610 review, EPA claimed a new economic analysis had been completed, yet the data provided cannot be reproduced and is insufficient to fully evaluate the decisions EPA made. The commenter notes that this is extremely disappointing given that the "new" information had a direct bearing on the Agency's findings which concluded that despite the lack of an accurate lead paint test kit, the rule's requirements should remain unchanged. The commenter states that, since the data used to conduct EPA's analysis has not been transparently presented in the associated docket, stakeholders have been unable to fully evaluate the report's underlying economic claims or their resulting impact on the review's conclusion. The commenter contends that since data is not transparently presented, it creates continuing challenges for stakeholders as they seek to work with EPA to ensure the program is effective and robust and that all impacts on small entities have accurately been taken into consideration.
Response: The EPA has considered applying this rule to all data and models and proposed to do so in the 2020 supplemental notice of proposed rulemaking. In this final rule, however, rather than considering all studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact as "pivotal science," the EPA is balancing transparency and feasibility by focusing on those studies that inform the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect. Thus, "pivotal science" includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). These are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. In the context of this rule, studies informing the dose-response modeling (e.g., benchmark response selection), studies identifying data for toxicokinetic adjustments, or studies informing the selection of uncertainty factors do not drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. Future statute-specific rulemakings may interpret "pivotal science" more broadly.
The EPA is addressing the manner in which it develops cost-benefit analyses for its regulations in other rulemakings, starting with rulemakings under the CAA, and as such the issue of what economic data the EPA considers in its cost-benefit analyses is outside the scope of this rulemaking. Pivotal science relying on dose-response data may be used to identify candidate PODs, which are then employed in the benefit-cost analysis for a significant regulatory action, but not all studies informing the benefit-cost calculation would be considered pivotal science.
Support Application to Pesticide Registrant Data
Comment: Commenters (0196 (mass campaign representing thousands of commenters), 0320, 6046) support EPA increasing availability of pesticide registrant data. 
Commenter (6046) notes that pesticide registrants submit toxicology study data as well as the identity of inert ingredients to EPA in order to register a pesticide product. The commenter provides that these data are considered CBI and are thus not publicly available. According to the commenter, the public must rely on EPA scientists to appropriately evaluate the study data and the toxicology of the inert ingredients, just as the public does when EPA uses unpublished study data for other regulatory decisions. The commenter states that non-publicly available pesticide registrant toxicology data are routinely evaluated by EPA scientists. The commenter states that there are often many unpublished pesticide registrant studies for each active ingredient pesticide. The commenter asserts that not requiring that pesticide registrant study data used by EPA be publicly available, when there is a need for confidentiality for both pesticide and non-pesticide data at the same time as requiring dose response data and models be publicly available illustrates a bias and if EPA is interested in transparency to the public, then methods to make pesticide registrant data available to the public should be evaluated and put into practice. The commenter notes that pesticide registrant data meets the definition of pivotal regulatory scientific data. The commenter recommends that the EPA take the lead to identify means to make inert ingredient hazard information available to the public. The commenter adds that the current list of approved inert ingredients is a start but does not provide consumers with sufficient information to determine if they may, for example, have an asthmatic allergic reaction to un-disclosed inert ingredients.
Commenters (0196, mass campaign representing thousands of commenters) note that, under the proposed "transparency" plan, the public will still lack access to key data about the effects and efficacy of commercial poisons approved for sale and application in their communities and homes. The commenters state that the proposed policy declares that it will "help ensure that EPA is pursuing its mission of public health and the environment in a manner that the public can trust and understand," yet it applies only to a very limited set of studies used to support certain EPA regulations. According to the commenters, the pesticide registration and review processes are particularly lacking in transparency, opportunity for public review, and access to data. The commenters add that, because pesticides are toxic chemicals broadcast into the environment, nowhere is transparency more important than pesticide registration.
Commenters (0196, mass campaign representing thousands of commenters) state that, because the pesticide manufacturer produces the underlying data for these EPA approvals and controls access to them, no transparency will be provided in pesticide registration:
 The public does not have access to the underlying data provided by the manufacturer to justify registering a new pesticide for commercial distribution;
 Industry will not have to provide the data used to assess health and environmental effects and farmworker impacts to set allowable dietary and non-dietary exposure limits; and
 Neither the public nor EPA has access to the underlying efficacy data, except for very limited purposes.
The commenters assert that, to ensure full review, it is critical that the public and independent scientific community have complete access to safety testing data before pesticide products enter the marketplace. According to the commenters, if EPA applies the proposed policy to pesticides, the failure to allow consideration of independent study results without disclosing confidential patient information will cripple the validity of EPA safety reviews. 
Commenters (0196, mass campaign representing thousands of commenters) contend that EPA pesticide regulation is based on secret science and is particularly lacking in transparency. The commenters provide that nonpublic data submitted by the registrant are used by EPA to assess health and environmental effects of the pesticide and impacts on farmworkers, and to set allowable human exposures through dietary and nondietary routes  - all without any opportunity for public review. The commenters assert that full disclosure of known and unknown adverse effects is needed in the registration process and on the label. The commenters assert that conditional registration is missing crucial data and allows market entry for a product in the absence of certain data normally required for registration and that efficacy data on pesticide products are lacking. According to the commenters, (1) the public do not have access to, and EPA does not review, manufacturer data on pesticide efficacy, even though the statutory registration standard requires weighing the risks of pesticides against their benefits; (2) secret ("inert") ingredients" in pesticide products are not disclosed, although they have been classified as hazardous under a variety of environmental laws; and (3) only active ingredients, not formulations, are tested, so data on the human health and environmental effects of the actual product on the market are entirely lacking.
Commenter (0320) provides that, to ensure full review, it is critical that the public and independent scientific community have complete access to safety testing data before pesticide products enter the marketplace. The commenter contends that, if EPA applies the proposed policy to pesticides, the failure to allow consideration of independent study results without disclosing confidential patient information will cripple the validity of EPA safety reviews. 
Response: Other commenters have noted that the 2018 proposed rule appeared to conflict with FIFRA pesticide registration requirements and associated implementing regulations, which require registrants to submit data and information to the EPA to enable the Agency to make its unreasonable adverse effects determinations. These commenters argued that, under the 2018 proposed rule, the EPA would not be able to consider these data, which are often claimed as CBI, when evaluating the pesticide registrations because the data could not legally be made publicly available. In this final rule, the EPA has codified in 40 CFR 30.3 that in the event the procedures outlined in this part conflict with statutes the EPA administers, or their implementing regulations, the statutes and regulations will control. This would include FIFRA and the Agency's pesticide registration activities. This final rule does not require any dose-response data to be made publicly available in a manner that would violate law, and rather protects privacy, confidentiality, CBI, and is sensitive to national security.
Support Application to EPA Integrated Planning Model (IPM)
Comment: Commenter (6917) provides that the EPA Integrated Planning Model (IPM) is frequently used in rulemaking to estimate cost impacts of regulations on the electric utility industry. The commenter notes that the IPM data output might be considered to be in the class of "pivotal regulatory science" since the model output data is "... critical to the calculation of a final regulatory standard or level, or to the quantified costs, benefits, risks and other impacts on which a final regulation is based." The commenter recommends that the Agency's use of IPM be made more transparent. The commenter notes that, although inputs into the model may be listed in one or more documents located somewhere on the rulemaking webpage or in the docket, finding all the input data is like looking for a needle in a haystack. In addition, according to the commenter, even if the input data can be found, it's not always clear how the model processes the data to produce the output. As a result, the commenter contends that, if the model were to produce erroneous output results, it may not be detected or the reason for a detected error may not be determined.
As an example, commenter (6917) states that, in year 2014 in the proposal for the Emission Guidelines for Existing Stationary Sources: Electric Utility Generating Units (Clean Power Plan or CPP), the supporting IPM output data erroneously indicated the retirement of Cleco's Brame Energy Center Unit 2 by year 2016. However, the commenter notes that, as written in the 2014 Cleco comments on the CPP proposal, Cleco's own predictive modeling at the time indicated there would be normal 2016 operation. In addition, the commenter states that, at the time of the commenting on the proposed CPP, the reason for the erroneous IPM output regarding Unit 2 could not be determined. The commenter states that, as a result, confidence in the IPM model results for the CPP proposal was reduced. The commenter recommends that, to increase transparency in the application of the IPM, clear information should be shared with the regulated community regarding the data-input to the IPM model and how the model's output is arrived at.
Response: Rather than pursuing the expansive scope of pivotal science outlined in the 2018 proposed rule, which may include IPM data, the EPA is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. The EPA is focusing on the underlying dose-response data for this rulemaking because of the influence these data have on particularly impactful decisions at the Agency. Risk assessments and regulations that target emissions and risk reduction of one or more pollutants, contaminants, or substances are integral to the Agency's mission and the underlying dose-response data that inform the quantitative value used to evaluate and mitigate potential risk are critical to understanding the assessment or regulatory action. In addition, the data underlying the dose-response assessment are more distinct than the broad range of data informing an entire risk assessment. Therefore, the EPA is concentrating its current efforts to increase transparency on a well-defined step in the quantitative assessment of risk supporting specific Agency actions. This final rule provides an important step in furthering the progress already being made toward maximizing transparency and will provide important insight for developing future statute-specific requirements.
Support Application to Other Data and Models in a Subsequent Rulemaking
Comment: Commenter (6151) suggests, in response to EPA's request for input as to whether the transparency rule should apply to other data, such as economic analyses or environmental impact models, that EPA initially focus on data related to pivotal scientific regulatory decisions as outlined in the notice.
Commenter (6375) notes that, although they support expanding the Proposed Rule to cover other types of data, they understand that this is a focused rulemaking effort and that the current rulemaking may not be the appropriate mechanism to address other types of data and information. The commenter encourages that, if EPA decides not to address other types of data and information in this rulemaking, EPA address in a subsequent rulemaking. 
Response: The EPA agrees with the commenters, and notes that this final rule provides an important step in furthering the progress already being made toward maximizing transparency and will provide important insight for developing future requirements in statute-specific rulemakings.
Support Application to Economic and Environmental Data and Models but Protect CBI and PII
Comment: Commenter (1011) suggests that EPA should provide access to economic and environmental impact data and models that are designed to predict the costs, benefits, market impacts and/or environmental effects of specific regulatory interventions on complex economic or environmental systems, so long as CBI, private personal information, and other potentially sensitive data sources are not made public in a manner that could violate confidentiality (i.e., EPA should aim to make aggregated data available). The commenter adds, however, that EPA must be cognizant of the increased burden any such requirement would place on researchers, and the agency should avoid making extensive or repetitive requests for information and explanation.
Response: The EPA clarifies that this final rule only applies to dose-response data. Rather than having this final rule apply to all the studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact, the EPA is focusing on those studies that describe the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect. Specifically, the scope of dose-response data in this final rule is those studies consisting of the data integral to characterizing dose-response relationships. These dose-response studies that constitute pivotal science may be used to identify candidate PODs that are then employed in the benefit-cost analysis, but not all studies informing the benefit-cost calculation would be considered pivotal science. The EPA further clarifies that this rule will not place any burden on researchers, as this rule exclusively pertains to the internal practices of the EPA and does not regulate the conduct or determine the rights or obligations of any entity outside the federal government.
Support Application to Modeling Used to Develop TMDLs
Comment: Commenter (6180) supports application of the Proposed Rule to modeling used to develop total maximum daily loads (TMDLs).
Response: The EPA clarifies that this final rule only applies to dose-response data. Rather than having this final rule apply to all the studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact, the EPA is focusing on those studies that describe the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect. Specifically, the scope of dose-response data in this final rule is those studies consisting of the data integral to characterizing dose-response relationships. These dose-response studies that constitute pivotal science may be used to identify candidate PODs that are then employed in TMDLs, but not all studies considered in the TMDL would be considered pivotal science. Further, the EPA notes that a future statute-specific rulemaking made under the CWA may interpret "pivotal science" more broadly. 

4.1.1.4.3 Oppose Applicability to Other Data and Models
Comment: Commenters (0671, 6191, 6915), in addition to opposing the applicability of the Proposal to dose response data and models underlying pivotal regulatory science, oppose expanding the applicability of the rule requirements to other data and models used by EPA to support regulatory decisions.
Commenter (6191) states that if economic and environmental impact data and models used to support a rule are developed by EPA scientists, they would seem to be subject to the federal Open Data Policy requirements. In general, the commenter notes that economic and environmental impact data do not necessarily include data from individual humans and therefore do not pose the same challenges as for dose response data and models. The commenter, adds, however, there are issues with mandating sharing when: (1) the research was conducted without intent to inform a rule; (2) was conducted without a pre-existing mandate to share data; iii) was conducted a long time ago and accessing the data for the research is impossible. According to the commenter, there will always be reasons why not all data can be shared and a strict mandate for public access will needlessly remove important research from consideration.
Commenter (6915) notes that, in developing regulations EPA uses other types of models in addition to dose response models. The commenter explains that these include toxicokinetic models that predict a chemical's absorption, distribution, metabolism, and excretion, as well as fate and transport models that predict a chemical's movement in the environment and distribution to environmental media. The commenter asserts that the Proposed Rule's provisions that could decrease protectiveness of dose response analysis (e.g., requiring justification of default assumptions and precluding consideration of relevant studies due to data disclosure requirements) could similarly result in decreased protectiveness of these other types of models. The commenter contends that, in regulations based on dose response analysis combined with toxicokinetic and/or fate and transport analyses, the overall decrease in protectiveness would be magnified.
Response: The EPA has not expanded the applicability of this final rule, but rather has narrowed it to only dose-response data underlying pivotal science. As codified in 40 CFR 30.5 of the final rule, the EPA will rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies --  based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements or quantitative analyses of an EPA significant regulatory action or influential scientific information will be identified as pivotal science. The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
4.1.1.5 Impact on EPA Decision-Making 
4.1.1.5.1 Need for Improvement in the Use of Scientific Evidence in Support of Regulatory Decision Making
Comment: Commenter (0068) expresses that they believe there are aspects of the proposal that, when properly revised and qualified, would represent an improvement in the use of scientific evidence in support of regulatory decision making that relies on dose-response data and models. 
Commenter (0068) specifically endorses the requirement that EPA:
      ...describe and document any assumptions and methods used and should describe variability and uncertainty. EPA shall evaluate the appropriateness of using default assumptions, including assumptions of a linear, no-threshold dose response, on a case-by-case basis. EPA shall clearly explain the scientific basis for each model assumption used and present analyses showing the sensitivity of the modeled results to alternative assumptions. When available, EPA shall give explicit consideration to high quality studies that explore: A broad class of parametric dose-response or concentration-response models; a robust set of potential confounding variables; nonparametric models that incorporate fewer assumptions; various threshold models across the dose or exposure range; and models that investigate factors that might account for spatial heterogeneity.
The commenter asserts, however, in requiring such discussion of assumptions, and parametric analysis, EPA should simultaneously make clear that its statutory responsibility is to protect public health, and hence in making decisions based on such evidence it will adopt the public health decision rule, "when in doubt, act to protect public health and wellbeing," as opposed to the classic decision rule used in scientific publishing, "place greatest emphasis on avoiding false positives."
Commenter (6855) suggests that the Agency modernize its entire conceptual framework for making regulatory decisions, including how the Agency manages its risk assessment methods, assumptions, models, uncertainty, and non-risk factors at the risk management stage.
Response: Consistent with the requirements in the 2018 proposed rule, this final rule requires the EPA to describe critical assumptions and methods used in dose-response assessments conducted by the Agency that drive the requirements, quantitative analyses, or both of a significant regulatory action or influential scientific information. The final rule also requires the EPA to characterize the variability and uncertainty of such dose-response assessments. 
The EPA does not agree that the Agency should codify the suggested public health decision rule or changes to the conceptual risk-based framework for making regulatory decisions in this final rule, as these activities are outside of the scope of the current rule. The purpose of this final rule is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. The EPA believes the approach described in the final rule will result in significant regulatory actions and influential scientific information that are based on the highest quality, most relevant studies and ensure adequate human health and environmental protections, while maximizing transparency and maintaining consistency with the statutes the EPA administers.
4.1.1.5.2 Limit Amount of Quality Data for Use in Support of Regulatory Decision-making
General Comments on Impacts of Limiting Data in Support of Regulatory Decision-making
Comment: Commenters (0501, 0945, 1077, 1203, 1281, 1306, 1310, 1311, 1331, 1355, 1363, 1369, 1373, 1375, 1385, 1388, 1395, 1400, 1445, 1797, 1879, 1883, 1946, 2003, 2107, 2157, 2171, 2361, 2430, 2907, 4595, 5011, 5022, 5165, 5180, 6111, 6121, 6125, 6153, 6181, 6355, 6373, 6376, 6378, 6446, 6801, 6880, 6894, 6896, 6915, 6919, 8275, 8281-PH38, 9227) oppose the rule and contend that the proposed rule will limit the use of science and quality data that can be used in EPA decision making and is contrary to statutory mandates/federal law. Science and quality data and studies that they note that EPA would not be able to consider in regulatory decision-making included, but were not limited to epidemiological studies, data/studies that rely on CBI and personal health information, and studies where the underlying data was no longer available.
Commenters (0501, 0569, 0844, 0945, 1077, 1203, 1272, 1281, 1306, 1310, 1311, 1331, 1355, 1363, 1369, 1373, 1375, 1385, 1388, 1395, 1400, 1445, 1797, 1883, 1946, 2003, 2107, 2157, 2361, 2428, 2431, 2479, 2746, 2899, 4848, 5164, 5165, 5180, 6121, 6133, 6153, 6157, 6185, 6355, 6373, 6378, 6446, 6874, 6884, 6895, 6896, 6909, 6915, 6939, 6955, 8275, 8276, 8281-PH38, 9171, 9227) note that the limitation of EPA's ability to use science and quality data will adversely affect policies/regulatory decisions (including, but not limited to, studies used as a basis for programs like the NAAQS Program, studies that link elevated lead levels to adverse health effects, and research into neurotoxicants and endocrine disruptors associated with adverse reproductive health outcomes) and the mission of the Agency to protect the environment and human health. 
Some specific comments regarding the proposed rule regarding the inability to use quality data/studies and the adverse impacts that would have on regulatory decision-making include:
 The rule could lead to ignoring critical evidence and could only lead to adverse consequences as it weakens decision making processes and delays important action. (1373)
 The proposed push for all studies used in government regulations to be made publicly available is not applied consistently, e.g. companies are not required to make publicly available all information and studies on pesticides. The commenter asserts that, the proposed regulation will cater to smaller, less relevant studies and will result in less evidence-based regulations to the detriment of all US citizens. (2107)
 The implementation of this rule, and censoring studies from use in EPA work because they could not be "independently validated" because of privacy issues could result in regulatory records supporting unreliable, pre-determined outcomes or decisions. (2171)
 Rather than increasing the transparency of decision-making, this proposed rule would disallow and thus hide non-publicly available studies, regardless of their scientific quality. (2361)
 Regulatory agencies are most effective when they have the entirety of the best scientific literature at their disposal for rulemaking to set new standards and policies. (2431)
 EPA's failure to consider individual health data -- because it cannot be made public -- would miss important data regarding particularly vulnerable populations that the NAAQS were developed to protect. (6446)
 The Proposed Rule would have an effect opposite to its claimed purpose; it would suppress important and relevant science conducted in large part by the best minds in academia and government, thereby unduly restricting the evidence available to EPA and potentially favoring data developed by industry. (5022)
 Along with the changes to the NAAQS review process outlined in EPA's May 9, 2018, "Back to Basics" memo and the recent requirement that members of the Clean Air Science Advisory Committee and related panel members not receive any current EPA funding, the Proposed Rule would constrain both the range of expertise and body of scientific literature that is available to be considered in these reviews, undermining the NAAQS. (6185)
 Limiting the ability of the EPA to the use of only those studies that can make their data public will drastically reduce the accuracy of rulemaking and will not allow a for a full accounting of costs and benefits. The commenter states that this will lead the EPA to be blind to much of the information needed and widely concurred as valid by the scientific community. (9171)
 Commenter opposes the proposed rule because it would require discarding valid studies that have passed rigorous peer review, even if published in top journals like Nature or Science, and even if confirmed by independent research. The commenter states that a decision to not regulate is also a decision, just like a decision to regulate. The commenter asks why can a decision to not regulate be made with no supporting studies, but a decision to regulate must be supported solely by unrestricted-distribution studies? (2899)
 Sound science is critical to policy decision-making and provides a process that explores how to best answer questions through a carefully structured approach. The commenter states that it cannot replace good decision-making, but science does enable us to make well-informed decisions and to predict the consequences and outcomes of our choices. (6157)
 The proposed rule would limit consideration of science in decision making in a manner that is unjustified and arbitrary, and would lead to inadequate regulatory protections, inconsistent with federal law. The commenter states that existing law and precedent dictates that the EPA take action based on the weight of scientific evidence even in the face of some uncertainty. The commenter states that this requires that the agency consider all available scientific evidence in its decision making and not make arbitrary exclusions of research. (8275)
 The purpose and effect of the proposal would be to degrade the quality of science in EPA's decision making. The commenter contends that there is no reason to think that excluding relevant science merely because the underlying data are not publicly available would increase the net benefits of a regulation. The commenter states that, for example, it appears that under the proposed rule EPA would exclude a peer-reviewed, published study whose conclusion had been reproduced based upon numerous different datasets (and whose underlying data, though not publicly available, had been reevaluated by outside experts), while including a study that had had no peer review, was not published, had no corroborating studies, and had not actually been replicated or reproduced, merely because the underlying data was made publicly available. The commenter states that this is simply not a recipe for more accurate decision making. (9227)
 Commenter does not agree with the agency's assertion that the best available science is that for which underlying datasets are fully open to public inspection. The commenter states that, if the EPA restricts the science it relies upon to only those studies that publish open datasets, the outcomes and data from key areas of scientific inquiry related to EPA's decision-making will be put beyond the reach of agency staff. (6181)
 Commenter urges EPA to withdraw the proposed rule as it limits EPA's ability to use the best available science, such as studies produced by leading research institutions to protect construction workers and the public in future rulemaking proceedings. (6896)
 The proposed rule would directly contradict the mandates of many Acts that require that the scientific review draw upon the best available science. (6378)
 The proposed rule would violate the agency's common sense and statutory duty to consider the sound scientific data. The commenter states that the proposed regulation turns the statutory scheme of Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA) upside down by forcing the Agency to ignore data that EPA can use to prevent Love Canal type disasters in the future. (6801)
 The proposed rule would prohibit the continued and future use of studies by EPA thereby obstructing EPA's statutory duty to consider the "best," "reasonably" available information in its decision-making processes. The commenter states that the resulting information vacuum would occur for no other reason than that the underlying human subject data are private and cannot be publicly disseminated. (6111)
 The proposed rule would in many instances prohibit EPA from relying on the best available science relevant to many of the regulatory issues that the agency faces. The commenter adds that this proposed requirement contravenes five decades of EPA practice in which EPA has repeatedly affirmed that its mission requires it to rely on the best available scientific evidence, without ever asserting that it should exclude from consideration studies for which the underlying data were not publicly available. (6111)
 When considering the likely consequences of EPA's proposal to base its regulations solely on studies for which data are publicly available or for which exceptions are made, it is instructive to consider examples of research into neurotoxicants and endocrine disruptors, which are associated with adverse reproductive health outcomes. The commenter notes that the EPA efforts to reduce lead and methylmercury exposure for pregnant women and children might not have been so successful if the agency had used the overly stringent definition of transparency it now proposes. The commenter adds that programs that might not exist, or exist in less-effective forms, include those to reduce lead exposure from paint, gasoline, and drinking water; limit mercury emissions; and advise pregnant women to steer fish consumption away from species that often contain high levels of methylmercury. (6939)
Response: Based on a consideration of these and other similar comments, the EPA proposed, through a 2020 SNPRM, a modified version of the 2018 proposed rulemaking regulatory text at 40 CFR 30.5. In the 2020 SNPRM, the EPA proposed Agency consideration of studies where there is tiered access to data and models that have CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects. For all other studies, data and models should be publicly available if the studies are to be used as pivotal regulatory science or pivotal science. In the 2020 SNPRM, the EPA also proposed an alternative. Under alternative proposed 40 CFR 30.5, when promulgating significant regulatory decisions or finalizing influential scientific information, the Agency would, other things equal, give greater consideration to studies where the underlying data and models are available in a manner sufficient for independent validation either because the information is publicly available or available through tiered access when the data include CBI, proprietary data, or PII and appropriate techniques have been used to reduce the risk of re-identification.
Based on a consideration of public comments received in response to the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. 
As described in the preamble to the final rule, the transparency provisions of the final rule apply to dose-response data underlying pivotal science. Under this rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule under 40 CFR 30.7.
The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information. 
Limiting Consideration of Epidemiological Data/Studies in Support of Regulatory Decision-making
Comment: Commenters (0039, 0501, 1331, 1388, 1424, 2064, 2171, 2495, 4537, 4828, 5981, 6113, 6125, 6133, 6144, 6153, 6174, 6181, 6874, 6884, 6895, 6915, 6919, 6932, 6939, 8272, 8281-PH38, 9227) express concern that this regulation would dramatically limit the consideration of epidemiological research data in future EPA regulations because patients' data must be kept confidential. Specific comments include:
 Commenters (1375, 1395, 2171, 5165, 6111, 6373, 6915, 6919, 8281-PH38) express specific concern that the rule would bar, due to confidential data, EPA's consideration of relevant scientific literature in the establishment of regulations designed to protect human health and the environment, including the Six Cities Study and related studies (e.g., American Cancer Society Study (ACS). Specific comments include:
 From a risk assessment perspective, not including epidemiology studies in regulatory science is not sound or prudent. According to the commenter, laboratory, toxicology, and epidemiology are complementary and necessary pieces of understanding and quantifying effects of a pollutant on human health and that excluding evidence from one of these three essential disciplines threatens the science basis for regulatory decisions and actions. The commenter contends that the proposed rule would put regulators tasked with protecting human health in an impossible situation of relying primarily on animal models or in-vivo models that cannot be directly extrapolated to human dose response estimates. (2171)
 Most of the cost-benefit analyses for air regulations are based on Six Cities and ACS studies because they present the best epidemiological data available for use in such analyses. According to the commenter, if the EPA determines that it cannot use the Six Cities and ACS studies in future regulatory action, this would also prevent their use for the EPA's cost-benefit analyses. The commenter argues that exposure and epidemiology studies have been fundamental to the policy through which public health agencies protect public health. The commenter points out that the EPA's proposal states that, "[f]or regulatory programs, like the National Ambient Air Quality Standards program, in which future significant regulatory actions may be based on the administrative record from previous reviews  -  particularly where the governing statute requires repeated review on a fixed, date-certain cycle  -  the EPA seeks comment on the manner in which this proposed rule should apply to that previous record." According to the commenter, this request for comment suggests that previous regulatory actions, like those using the Six Cities and ACS studies, are in peril because they were properly conducted in accord with health privacy regulations to protect sensitive personal data from the health studies on which these regulations are based. (6919)
 Commenter (1395) estimates that EPA cites approximately 50,000 studies each year, many of which are based on analyses of confidential medical records. The commenter states that, if this rule is put into effect, these studies, including the Six Cities Study, would not be able to be used and the rule could be used as a basis to undo rules protecting public health. 
 Commenter (1375) states that this proposed regulation is an attempt to exclude the most impactful and important public health studies in history. The commenter states that public health research requires the collection of personal data from study participants in order to understand their lifestyle and environmental exposures. The commenter states that this data must be extensive and detailed to capture the many possible variables impacting health outcomes, precisely so that the actual cause of the observed health impact can be assessed while accounting for confounding variables. The commenter states that the personal data are often so extensive that even if the participants were anonymized, it would still be possible to determine from the information collected who the actual participant was. 
 Commenter (5165) asserts that EPA should publicly confirm that it would consider existing literature such as the Six Cities Study in future rulemakings, should the proposed rule be enacted. The commenter notes that EPA suggests in footnote 3 of the proposal that it would exclude such studies from consideration. 
 Commenter (6111) states that, contrary to EPA's stated goal of improving the basis for its regulatory decisions, requiring the public availability of all raw data will instead undermine EPA's ability to make reasonable decisions. The commenter states that this requirement will effectively prohibit EPA from considering studies that by design are based on data that cannot be made publicly available due to laws and contracts designed to protect patient and human subject privacy and ensure willingness of people to participate in research by sharing their private information with researchers. The commenter states that the proposed rule precludes consideration of studies based on confidential data, even when those results have been confirmed by other studies.
 Commenters (6373, 6915) assert that, if EPA excludes relevant studies because the underlying data cannot be made public, the efficacy of EPA regulations themselves will be threatened. The commenters suggest that, if EPA had disregarded the Six Cities and ACS Cohort Studies based on the lack of publicly released data, air quality standards would not have been tightened; they might have been relaxed to allow increased levels of pollution because relevant and reliable scientific evidence would not have been before EPA. The commenters state that this could have had significant negative public health implications, including greater excess morbidity and mortality attributable to air pollution. Commenter (6915) states that EPA estimates that reductions in ambient particulate matter under the 1990 CAA Amendments will prevent 230,000 adult deaths by 2020.
 Commenter (9227) expresses concern that EPA's proposed regulation would preclude the Agency from using key studies to support the proposed rule regarding methylene chloride in paint and coating removers. The commenter asserts that the effect would be to severely jeopardize the finalization of this life-saving rule. The commenter notes that EPA has proposed a ban on the use of methylene chloride in paint and coating removers and that methylene chloride is associated with a number of hazardous health effects, including impaired visual and motor functions, respiratory irritation, headaches, nausea, and death. The commenter provides that the scientific basis for the proposed regulation is provided in the Agency's 2014 risk assessment. According to the commenter, the raw data underlying key studies used to derive the benchmark margin of exposure for chronic exposure and acute exposures to methylene chloride are not publicly available.
 Commenters (6153, 6874, 6939, 8272) state that programs that might not exist, or exist in less-effective forms, include those to limit mercury emissions; and advise pregnant women to steer fish consumption away from species that often contain high levels of methylmercury.15 Commenters state that EPA efforts to reduce methylmercury exposure for pregnant women and children might not have been so successful if the Agency had used the overly stringent definition of transparency it now proposes. 
 The lead rules are based on risk analyses conducted by EPA using epidemiology studies published in the 1990s that correlate childhood blood lead levels with impaired brain function and adverse behavioral effects. The commenter states that many of the published studies are longitudinal cohort studies that include measurements of lead in blood from children decades ago, and then follow them out over time to observe lasting effects. The commenter notes that it makes it impossible to replicate the exposure conditions at the time the original children in the study cohort had their blood lead levels measured, such as the Port Pirie cohort study population living near a lead smelter in the 1980s. The commenter asserts that longitudinal cohort studies, particularly those that capture exposures that may no longer occur -- are not reproducible. (6133). Some specific regulations cited by commenters that have been developed as a result of lead studies include:
 Regulations governing lead in paint. (1424, 5981, 6125, 6144, 6153, 6174, 6874, 6939, 8272) 
 Regulations governing lead in gasoline. (1424, 5981, 6153, 6874, 6895, 6939, 8272)
 Regulations governing lead in dust and/or soil. (6125, 6144, 6174)
 EPA's drinking water standards for lead. (4828, 6125, 6144, 6153, 6174, 6874, 6939, 8272)
 EPA's Lead and Copper Rule (LCR) of 1991. (6133)
 EPA's 2006-2008 lead air standards. (6125)
 Commenters (6125, 9227) express concern that, if the proposed rule were applied retroactively or if EPA re-evaluates the maximum contaminant level (MCL) for arsenic, the proposed rule would likely mean that EPA could not rely on two key studies because the data are four to five decades old and include confidential individual health information. According to the commenter (9227), the MCL finalized by EPA was based on the two epidemiological studies, both of which were peer reviewed, published in prestigious health and environmental journals, and have been cited numerous times by other researchers.
 Commenters (6133, 6932) specifically express concern that regulation of chlorpyrifos could be impacted by the proposed rule because the epidemiologic studies of chlorpyrifos can no longer be reproduced. Commenters (6153, 6874, 8272) state that agricultural workers, the majority of whom are Hispanic, bear the greatest burden of current chlorpyrifos exposure. Failing to take regulatory action to reduce these exposures because individual-level data sets from high-quality research are not publicly available would allow the persistence of environmental injustice. Commenter (6113) states that a striking example of the risks posed by the proposed regulation were highlighted when the Ninth Circuit USCA released its decision in League of United Latin American Citizens v. Wheeler. The commenter provides that, in that case, the Ninth Circuit ruled that EPA had to ban the use of chloropyrifos on food products because of the studies indicating harm to humans. According to the commenter, in the event the EPA's regulation were in effect, chloropyrifos could not be banned despite the pesticide's documented risks to the brains of children. 
 Commenters (0039, 4537, 6133) states that the reassessment of the LNT model for radiation could profoundly impact many regulations and non-regulatory guidance from multiple entities, including the EPA, the Nuclear Regulatory Commission, and others. A list of regulations and non-regulatory guidance that would be impacted is provided by commenter 6133 in their comments. Commenter (6133) notes that the epidemiologic science and associated studies that are the basis of adherence to the LNT and decades of protective radiation standards are likely to be expressly excluded from consideration by EPA by the terms of this Proposal. The commenter adds that NAS and other studies that EPA has long relied upon in the radiation standards setting process are epidemiological human cohort studies. The commenter states that the EPA's Proposal, if implemented, would limit EPA staff from basing regulatory actions on precisely these types of studies by requiring that the underlying data of these studies be publicly shared. Data for some of the radiation epidemiological studies are accessible to users,with a detailed description of how a user can access the information. However, the commenter notes that public sharing of PII is restricted because the studies rely on confidential health data. The commenter contends that these studies are profoundly important studies that have been peer reviewed for decades and the science that has emerged from them has been validated multiple times. (6133)
 Commenters state that EPA announced that the Agency will begin the process of developing, under the SDWA, maximum contaminant levels (MCLs) for perfluorooctanoic acid (PFOA) and perfluorooctane sulfonate (PFOS), in addition to designating these chemicals as "hazardous substances," possibly under the CERCLA. The commenters assert that EPA's proposed rule would limit EPA's ability to protect the public from the health hazards associated with PFOA and PFOS. (6111, 6137, 6144, 9227)
 Researchers conducted exposure and health epidemiological studies consisting of nearly 70,000 participants to examine the health impacts of exposure to these chemicals. (6137) 
 Commenters provide that when EPA issued health advisories for these two chemicals in 2016, the Health Effects Support Documents relied extensively on epidemiological studies generated by the C8 Health Project. According to commenters, through this work, the researchers identified probable links between exposure to these chemicals and six specific diseases: diagnosed high cholesterol, ulcerative colitis, thyroid disease, testicular cancer, kidney cancer, and pregnancy-induced hypertension. (6111, 6137, 6144, 9227) 
 Commenter provides that these studies looked at "demographic data, medical diagnoses (both self-report and medical records review), clinical laboratory testing, and determination of serum concentrations of 10 perfluorocarbons (PFCs)," information that is both sensitive and confidential. The commenter states that, although the results of these studies have been published in numerous articles in scientific journals, under EPA's Proposed Rule this research would be excluded from consideration despite the clear evidence of harm these chemicals pose. (6137)
 Commenter (6137) states that turning a blind eye to this data would have dramatic public health consequences, as it would preclude evaluation of valuable evidence of harm from exposure that could and should form the foundation of protections under the statutes EPA is charged with executing for the benefit of the public and the environment. Commenter (9227) states that, in failing to consider such crucial case studies, EPA would be ignoring best available science, thereby undermining its own attempt to protect Americans from emerging health threats such as PFOA and PFOS.
 Commenter states that pyrethroid pesticides are classified by EPA as a "likely human carcinogen," and are linked in published studies to Parkinson's Disease and adverse behavioral problems in prenatally exposed children., The commenter provides that EPA convened a FIFRA Scientific Advisory Panel in October 2017 to assess its use of a Physiologically Based Pharmacokinetic Model (PBPK) used in its risk assessment for the pyrethroid pesticides. The commenter notes that, despite the central role of the pyrethroid PBPK model in EPA's regulatory approval for pyrethroid pesticides, scientific peer reviewers on the FIFRA Scientific Advisory Panel were unable to obtain the raw data necessary to provide a robust peer review of the model. According to the commenter, at this point, the EPA Scientific Advisory Panel meeting is postponed indefinitely. The commenter states that the stated reason is "due to the unavailability of experts," but that they believe that the more likely reason is to bias the panel with the addition of industry experts, as EPA has done recently with its Scientific Advisory Boards. According to the commenter, a model that underestimates exposures and health risks will lead to regulations that fail to protect Americans from harmful exposures to pyrethroid pesticides.(6133)
 Commenter (2064) notes that the EPA is currently reviewing the registration of an herbicide called paraquat, which, through peer-reviewed and validated research, is strongly associated with an increased risk of Parkinson's disease. The commenter provides that much of the research relies on large, population-based studies, which is the type of research most impacted by this rule. The commenter states that, without the ability to consider this research, the EPA will miss vital information needed to protect public health, thus placing Americans at risk. The commenter contends that many of the most important studies involving environmental exposures and Parkinson's disease would be excluded from future determinations for one or more of the following reasons:
 The authors did not previously obtain the needed consents to release the data and would thus be subject to criminal penalty if they did so.
 The data cannot be deidentified appropriately while complying with federal privacy protection laws. 
 The cost to deidentify would be very high and many research institutions do not have people on staff who can perform the work needed to do so.
 Commenter (0501) states that epidemiological and observational studies that involve personal data are essential for identifying the links between pollution and health effects. The commenter states that it is difficult to make these data public because researchers need to protect the privacy of study participants and their data. The commenter states that the fact that the data will not be made public does not diminish the quality of these studies. The commenter states that this rule will limit the amount of quality data that can be used by the Agency and will adversely affect the mission of the Agency to protect the environment and human health.
 Commenter (1331) states that the proposed regulation underestimates the usefulness of environmental epidemiology in highlighting the areas of the environment that need remediation due to their negative effects on public health. The commenter states that, by implementing the proposed regulation, public transparency of data from studies looking at the connections between these two integral components of public policy would be illegal and ethically wrong.
 Commenter (6884) states that epidemiological studies produce vital and valid scientific information that is necessary to determine impacts on humans of various types of occupational and environmental exposures. The commenter states that, without these studies, real world impacts on individuals and communities would not be captured, and important data that could affect hundreds and thousands of people would be lost.
 Commenter (6915) states that the provisions of the proposed rule would essentially prohibit the use of such epidemiology data in human health risk-based assessment despite their clear superiority over animal data for use in risk assessment. The commenter states that, in general, and specifically in EPA's 2005 Guidelines for Carcinogen Risk Assessment, human (i.e., epidemiology) data are preferred to animal data as the basis for risk assessment toxicity factors when they are of sufficient quality and are amenable to dose response modeling. The commenter states that this is because animal data always carry inherent uncertainties in regard to their relevance to humans. The commenter notes that epidemiology data collected over at least the last 40 years, however, have been generated under the auspices of IRBs working to protect the patient or participant information obtained by academic institutions, government entities, hospitals, and other organizations, and thus disclosure of that data would be difficult, if not impossible. The commenter states that these factors would all preclude EPA's or researchers' ability to provide raw, unpublished data for re-analysis as required under the proposed EPA rule. 
 Commenter (6915) states that, for older epidemiology data, such as data from studies on occupational exposures to workers in factories before the advent of strict IRB requirements, raw data are seldom if ever still available. The commenter states that such data, including high quality data generated by major corporations in conjunction with academic institutions, would not be available to EPA under the proposed rule. The commenter states that, effectively, the proposed rule would restrict the epidemiology data available for use by EPA, even where the weight of the evidence clearly supports a finding of causality and risk.
Response: Based on a consideration of these and other similar comments, the EPA proposed, through a 2020 SNPRM, a modified version of the 2018 proposed rulemaking regulatory text at 40 CFR 30.5. In the 2020 SNPRM, the EPA proposed Agency consideration of studies where there is tiered access to data and models that have CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects. For all other studies, data and models should be publicly available if the studies are to be used as pivotal regulatory science or pivotal science. In the 2020 SNPRM, the EPA also proposed an alternative. Under alternative proposed 40 CFR 30.5, when promulgating significant regulatory decisions or finalizing influential scientific information, the Agency would, other things equal, give greater consideration to studies where the underlying data and models are available in a manner sufficient for independent validation either because the information is publicly available or available through tiered access when the data include CBI, proprietary data, or PII and appropriate techniques have been used to reduce the risk of re-identification.
Based on a consideration of public comments received in response to the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. 
As described in the preamble to the final rule, the transparency provisions of the final rule apply to dose-response data underlying pivotal science. Under this rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule under 40 CFR 30.7.
The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
Limiting Consideration of Studies that Use Confidential Data (Personal and Business Confidential Data) and Studies Where Underlying Data is No Longer Available in Support of Regulatory Decision-making
Comment: Commenters (1281, 1306, 1445, 5011, 6111, 6181, 6378, 6895, 6909, 6915, 6919, 8281-PH38) express concern that, in developing future regulations to protect human health and the environment, EPA would be precluded from considering relevant information that has been validated through peer review, on the sole basis that the underlying data are not publicly available. Specific comments include:
 The proposed rule fails to recognize or acknowledge the existence of many studies already designed and published with terms that make complete transparency difficult or impossible because of IRB requirements and other important confidentiality protections. (6915)
 Commenters (1310, 1311, 6865) express concern that, in developing future regulations, EPA would be precluded from considering relevant studies on the sole basis that the underlying data are not publicly available. Commenter (1311) states that, given that much key research relies on confidential health data from study subjects, this proposal risks severely restricting the number and quality of studies available for use when making critical decisions. Commenter (6865) states that, discarding the opportunity to engage important studies based on their use of private data would be a costly mistake given their potential to save lives and advance scientific understanding in policy decisions.
 Commenter states that, by prohibiting the usage of information derived from anonymous sources, this largely excludes many useful scientific sources of knowledge. The commenter states that, in order to receive IRB approval to conduct work with human subjects, assuring the anonymity of participants is an essential step. The commenter states that guaranteeing anonymity of subjects is considered the top standard for conducting research with human subjects when investigating sensitive topics such as negative health outcomes. The commenter states that excluding studies because they are upholding scientific standards is a very unwise decision. (1310)
 Commenter expresses concern that, under the misleading veil of "transparency," the proposed rule could force investigators to invade the confidentiality of research participants and make confidential and private data open to all. (5113)
 Commenter states that requiring a study's underlying datasets be fully open to public inspection would drastically limit EPA's ability to use the most up-to-date, peer-reviewed science. The commenter states that this standard conflicts with privacy protections and the confidentiality of personally identifying information. The commenter states that it is not possible for a host of epidemiological studies and long-term health studies to open their datasets without publishing sensitive information that cannot, by law, be made public. The commenter states that the cost of sanitizing such data would be immense and would be borne by researchers who do not necessarily have the resources or institutional support to complete the process. (6181)
 Commenter provides two examples of important studies that would not be available to the EPA review if the proposed rule were put in place: Kaufman et al 2016, based on the MESA Air study, and Miller et al 2007, based on the Women's Health Initiative study. The commenter states that both studies rely on human subject data that is protected. (6378)
 Concerns about protecting privacy may deter researchers from providing studies to the EPA. The commenter states that researchers have a responsibility to assure the confidentiality of data they collect from human subjects. The commenter states that combining or merging datasets can increase risk to confidentiality by increasing the likelihood of identifying individuals. The commenter states that researchers may therefore be reluctant to submit their research to the EPA because producing the required data may violate informed consent provisions. (6877)
 The proposal would effectively prevent the EPA from using many kinds of scientific studies vital to its decision-making, including studies that rely on personal health data, CBI, intellectual property, or older studies where the authors or data sources may not be accessible. The commenter asserts that EPA's proposed rule would do nothing to improve transparency for scientists, policymakers or the public. The commenter provides that scientists do not review the raw data for studies since that would reveal little; rather the scientist reviews the research questions, methods, summarized data, results and conclusions in order to assess the quality of the work. (5011)
 The proposed rule precludes consideration of studies based on confidential data, even when those results have been confirmed by other studies. The commenter provides that, in the professional scientific and medical research community, "transparency" means clear and detailed disclosure of all methods, data, assumptions, and uncertainties. The commenter states that studies are considered "transparent" when the study design and methodology are clear enough to allow other scientists to challenge assumptions, test hypotheses, and either reproduce or replicate the study to determine whether the results obtained are consistent with the original study. According to the commenter, the concept of transparency promoted by the draft rule is harmful to good decision-making, to implementation of the statutes, and, most of all, to protection of public health and safety. (6111)
 The Proposal could cull the use of research that includes CBI or older studies that has data stored on older technology that can't be recovered. (8281-PH38)
 The Proposal would affect protections currently in place, such as limits on certain toxic air emissions from tail pipes and smoke stacks, and information on the health effects of many of these; where more than 150 chemicals come from older studies built on confidential patient or private business data that cannot be made public. (8281-PH38)
Response: Based on a consideration of these and other similar comments, the EPA proposed, through a 2020 SNPRM, a modified version of the 2018 proposed rulemaking regulatory text at 40 CFR 30.5. In the 2020 SNPRM, the EPA proposed Agency consideration of studies where there is tiered access to data and models that have CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects. For all other studies, data and models should be publicly available if the studies are to be used as pivotal regulatory science or pivotal science. In the 2020 SNPRM, the EPA also proposed an alternative. Under alternative proposed 40 CFR 30.5, when promulgating significant regulatory decisions or finalizing influential scientific information, the Agency would, other things equal, give greater consideration to studies where the underlying data and models are available in a manner sufficient for independent validation either because the information is publicly available or available through tiered access when the data include CBI, proprietary data, or PII and appropriate techniques have been used to reduce the risk of re-identification.
Based on a consideration of public comments received in response to the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. 
As described in the preamble to the final rule, the transparency provisions of the final rule apply to dose-response data underlying pivotal science. Under this rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule under 40 CFR 30.7.
Additionally, the final rule does not require the EPA or other parties to make dose-response data underlying pivotal science available (publicly or through restricted access) in a manner sufficient for independent validation. 
The EPA believes that the final rule will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of significant regulatory actions and influential scientific information.
4.1.1.5.3 Skew Regulatory Decision-making Toward Studies That are Less Rigorous and Trustworthy/Transparent
Decision-making Will be Less Scientific and More Arbitrary
Comment: Commenters (0569, 1203, 1355, 1879, 4595, 6895, 6909, 6916) assert that the proposed rule will deprive EPA of great amounts of useful information and will make EPA decision making less scientific and more arbitrary, by just that measure. Specific comments include:
Commenters (0569, 1203, 4595, 6880, 6895) express concern that an incomplete pool of scientific evidence could skew regulatory decision-making toward studies that are less rigorous and trustworthy even if data were available. Commenter (1203) states that a broad-brush rule that eliminates a lot of evidence is a recipe for weak, poorly informed policy that may exaggerate some public health threats and minimize others. 
Specific comments include:
 By limiting the science EPA can use in policies and regulations, EPA will constrain itself to smaller groups of studies and risk biased outcomes. The commenter states that, if the pool of research is smaller--which is what this proposed rule will lead to--that smaller pool utilized creates an inherent bias and thus the rulemaking process will yield distorted results. (4595)
 Should EPA adopt this rule, established knowledge that has been through the scientific validation process will be ignored. The commenter states that acting as though established knowledge does not exist will lead to flawed decisions and threaten environmental and public health. (6895)
 Not only would that render the future substantive rules unlawful under the Act's requirements, but it would also make them arbitrary and capricious, as "failure to consider the evidence proffered renders [a decision] arbitrary and capricious" and the Agency may not "hastily discount[]" relevant studies. The commenter asserts that the Agency seems unaware that its new policy, if finalized, could lead to future rulemaking actions that are unlawful, arbitrary and capricious. (6916)
 The new rule is not an attempt to make environmental regulations more transparent, it is a rule to limit research on the impacts of deadly pollution and give free reign to deregulate industries in the direct contradiction of science. The commenter states that not including research that includes confidential information translates into not including health science. (1355)
 The provision of the rule allowing EPA to arbitrarily bypass the requirement for studies that they like is completely anti-scientific at its heart. The commenter states that the whole rigor of the enterprise is based on taking all available information into account, independent of prior preferences or ideas. The commenter states that, if to the exclusion is added the ability to include without justification some studies that would otherwise be excluded, the danger of circular self-confirmation in views already held becomes extreme. (1879)
 The proposed rule's attempt to exclude perfectly reliable scientific studies from EPA's consideration is a thinly veiled attempt to allow the EPA to ignore sound science at its discretion if that science does not conform with the current Administration's political agenda. (6909)
 The proposed rule would freeze the science in procedures that may or may not make sense today but will certainly not be scientifically defensible in the future. The commenter states that issuing regulations on risk assessment methodology is a slippery slope that potentially subjects the process to at best, control by risk managers and attorneys, and at worst, politicization. (3135)
 Commenter expresses concern that the rule could replace evidence-based decision-making with arbitrary determinations based on political considerations. (5113)
 While the Proposal would eliminate many studies from EPA consideration due to confidential data, this limitation is not equally imposed by the EPA on industry and academia. The commenter states that industry-funded studies could keep their data secret by claiming it is "CBI." The commenter asks what is the EPA justification for this unequal treatment of corporate and academic research? (2907)
 While many commenters have pointed out the potential harm from chemical exposures that could result by excluding studies from consideration in rulemaking, the proposal could also unfairly skew rulemaking against use of chemicals that would not result in an expectation of unreasonable harm. The commenter states that a recent paper based on findings of the NIH Agricultural Health Study (https://aghealth.nih.gov/) showed no association between exposure to glyphosate and any cancer. The commenter states that, in deciding how and whether to regulate glyphosate (e.g., setting tolerances), EPA would have to ignore this real-world study, thus giving disproportionate weight to other studies (some lab-based as opposed to conducted under real-world conditions) that suggest glyphosate could cause cancer under some circumstances. The commenter states that this could result in an unfair burden on the regulated community in terms of stricter regulation or the discontinued use of products. (6880)
Response: Based on a consideration of these and other similar comments, the EPA proposed, through a 2020 SNPRM, a modified version of the 2018 proposed rulemaking regulatory text at 40 CFR 30.5. In the 2020 SNPRM, the EPA proposed Agency consideration of studies where there is tiered access to data and models that have CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects. For all other studies, data and models should be publicly available if the studies are to be used as pivotal regulatory science or pivotal science. In the 2020 SNPRM, the EPA also proposed an alternative. Under alternative proposed 40 CFR 30.5, when promulgating significant regulatory decisions or finalizing influential scientific information, the Agency would, other things equal, give greater consideration to studies where the underlying data and models are available in a manner sufficient for independent validation either because the information is publicly available or available through tiered access when the data include CBI, proprietary data, or PII and appropriate techniques have been used to reduce the risk of re-identification.
Based on a consideration of public comments received in response to the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. 
As described in the preamble to the final rule, the transparency provisions of the final rule apply to dose-response data underlying pivotal science. Under this rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule under 40 CFR 30.7.
The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
Comment: Commenter (1798) asserts that GLP compliant studies are not transparent. The commenter suggests that, while industry's GLP test raw data may be available to regulators, the industry does not allow regulators/agencies or academic scientists to access any data beyond what they make available in the GLP study. The commenter states that, in contrast, toxicity studies by academics will usually allow 3rd parties that they trust to access all raw data. The commenter states that the instances of secret industry occupational studies are too numerous to deny. According to the commenter, EPA evaluators should have the flexibility to make case by case reliability & relevance decisions.
Response: As described in the preamble to the final rule, the transparency provisions of the final rule apply to dose-response data underlying pivotal science. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption, on a case-by-case basis, from the requirements of the final rule under 40 CFR 30.7.
Joint Statement by Editors
Comment: Commenters (1306, 1797, 1883, 6111, 9227) note that the editors in chief of the world's top scientific journals have notified EPA that "[i]t does not strengthen policies based on scientific evidence to limit the scientific evidence that can inform them; rather, it is paramount that the full suite of relevant science vetted through peer review, which includes ever more rigorous features, inform the landscape of decision making." The commenters state that the editors conclude that EPA's proposal to exclude relevant studies from EPA's consideration based solely on the fact that underlying data or methods cannot be made available to the public "will adversely affect decision-making processes." Specific comments include:
 The stringent criteria of data availability proposed by the EPA fail to enhance currently accepted standards for quality of evidence and transparency, and most certainly would exclude significant research from consideration in policy decisions. The commenter states that, while they believe in the ideal of data transparency, they also believe that where such transparency is not justifiably possible, peer-reviewed science should not be excluded from informing policy decisions. (1797)
 The proposed rule fails to take into account the fact that studies are reliable and constitute the best available science when they comply with professionally-established best practices for describing the methodology, sampling size, sampling procedure and assumptions utilized and the results are consistent with those of other studies. (6111)
 Publication in a peer-reviewed scientific journal is the way that scientists communicate their findings to other scientists and is considered the hallmark of scientific quality. (9227)
 In response to EPA's Proposal, the editors-in-chief of Science and Nature, and other leading scientists explained that though "[d]ata sharing is a feature that contributes to the robustness of published scientific results. . . in not every case can all data be fully shared." For example, full sharing is not possible when data sets include "personal identifiers." The commenter states that the editors confirm that even under circumstances where underlying data cannot be made generally available, it is possible to evaluate the merits of a study, explaining: "Importantly, the merits of studies relying on data that cannot be made publicly available can still be judged. Reviewers can have confidential access to key data and as a core skill, scientists are trained in assessing research publications by judging the articulation and logic of the research design, the clarity of the description of the methods used for data collection and analysis, and appropriate citation of previous results." (9227)
Response: Based on a consideration of these and other similar comments, the EPA proposed, through a 2020 SNPRM, a modified version of the 2018 proposed rulemaking regulatory text at 40 CFR 30.5. In the 2020 SNPRM, the EPA proposed Agency consideration of studies where there is tiered access to data and models that have CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects. For all other studies, data and models should be publicly available if the studies are to be used as pivotal regulatory science or pivotal science. In the 2020 SNPRM, the EPA also proposed an alternative. Under alternative proposed 40 CFR 30.5, when promulgating significant regulatory decisions or finalizing influential scientific information, the Agency would, other things equal, give greater consideration to studies where the underlying data and models are available in a manner sufficient for independent validation either because the information is publicly available or available through tiered access when the data include CBI, proprietary data, or PII and appropriate techniques have been used to reduce the risk of re-identification.
Based on a consideration of public comments received in response to the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. 
As described in the preamble to the final rule, the transparency provisions of the final rule apply to dose-response data underlying pivotal science. Under this rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule under 40 CFR 30.7.
The requirements included in the final rule provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the EPA maintains a strong scientific basis for its decision-making. The incremental progress made possible by the final rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments.
PLOS Medicine Editorial
Comment: Commenters (2906, 6133) note that in an editorial in PLOS Medicine, Professor John Ioannidis states "[i]f the proposed rule is approved, science will be practically eliminated from all decision-making processes. Regulation would then depend uniquely on opinion and whim." Commenter (2906) asserts that EPA is a non-partisan organization that needs the best available research & minimal political interference in order to create regulations that balance safety with vigorous commercial growth. Commenter (6133) suggests that, instead of restricting the pool of available science by instituting an unworkable requirement for a broad category of scientific inquiry, EPA should focus on identifying weaknesses in the available evidence and targeting future investigations towards addressing specific deficiencies. The commenter adds that, assessing whether any particular study is reliable is not contingent on whether the underlying data can be made public, a fundamental point made clear in a report that EPA itself cited in the Proposal. The commenter includes the following additional quotes from the PLOS Medicine editorial:
      Ioannidis explains that "we should recognize that most of the raw data from past studies are not publicly available," and "[s]ome deficiencies may be unavoidable. For example, researchers cannot ethically randomize people to harmful exposures in order to tackle confounding, nor violate informed consent agreements that prohibit open sharing of private data from past studies." Ioannidis goes on to say that "simply ignoring science that has not yet attained such standards, is a nightmare," and "we would see governments discarding science at massive scale because of perceived imperfections and impurities." Ioannidis also notes that "we have extremely strong evidence that the tobacco pandemic is devastating; that the MMR vaccine is generally safe; that climate change is happening; and that air pollution is a major health hazard," in contrast to "most dietary advice one might hope to give about specific nutrients." The subjects that Ioannidis explains have strong evidence are the issues EPA is responsible for addressing that the Proposal seeks to discredit. Ioannidis further notes:
            For example, the pivotal research on the health effects of air pollution is particularly strong. The Six Cities and American Cancer Society studies are exemplary large-scale investigations, with careful application of methods, detailed scrutiny of measurements, replication of findings, and, importantly, detailed re-analysis of results and assessment of their robustness by entirely independent investigators. The re-analysis and sensitivity analyses were conducted by the Health Effects Institute that was funded by stakeholders some of whom may have desired to see opposite conclusions. It would be wonderful, if in the future the same rigorous re-analysis and replication standards could become the standard for all important areas of research that can inform policy.
Response: Based on a consideration of these and other similar comments, the EPA proposed, through a 2020 SNPRM, a modified version of the 2018 proposed rulemaking regulatory text at 40 CFR 30.5. In the 2020 SNPRM, the EPA proposed Agency consideration of studies where there is tiered access to data and models that have CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects. For all other studies, data and models should be publicly available if the studies are to be used as pivotal regulatory science or pivotal science. In the 2020 SNPRM, the EPA also proposed an alternative. Under alternative proposed 40 CFR 30.5, when promulgating significant regulatory decisions or finalizing influential scientific information, the Agency would, other things equal, give greater consideration to studies where the underlying data and models are available in a manner sufficient for independent validation either because the information is publicly available or available through tiered access when the data include CBI, proprietary data, or PII and appropriate techniques have been used to reduce the risk of re-identification.
Based on a consideration of public comments received in response to the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. 
As described in the preamble to the final rule, the transparency provisions of the final rule apply to dose-response data underlying pivotal science. Under this rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule under 40 CFR 30.7.
The requirements included in the final rule provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the EPA maintains a strong scientific basis for its decision-making. The incremental progress made possible by the final rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments.
4.1.1.5.4 Impacts on Addressing New and Emerging Environmental and Public Health Concerns
Comment: Commenter (1363) states that limiting scientific studies that can be used in forming the EPA's regulations to only those in which all information is "publicly available" is unreasonable and will place unnecessary constraints on scientists and slow the speed in which studies can be conducted and findings presented. The commenter states that this hurts the agency's ability to consider and form regulations and policies quickly in response to the changing environment. The commenter states that, if testing cannot be done quick enough to prove new regulations must be put in place to protect environmental and individual health, then unwanted harm will come to the people and the planet. The commenter states that the government must ensure a fine balance between accountability and efficiency. The commenter contends that this proposed rule will ruin the current balance by placing an unwarranted emphasis on accountability, hurting the efficiency of scientists and EPA policy makers. The commenter states that, while Republicans are constantly calling for a smaller government with less interventions and paperwork, so that it functions more efficiently, this rule goes against these values and will only cause more problems and delays in the future policy-making process.
Response: The EPA disagrees that the requirements of the final rule will result in any meaningful delay in promulgating regulations. While the final rule requires the Agency to evaluate the availability of dose-response data for pivotal science, the incremental burden to the EPA to carry out these requirements is expected to be small given (1) the extensive scientific review the EPA already conducts regularly and (2) that the requirement is limited to pivotal science (i.e., typically a small, though highly important, subset of the studies the EPA would review).
Comment: Commenter (6111) states that the proposed rule would impede EPA's ability to address new and emerging public health risks in future rulemakings, including actions for two fluorochemicals, perfluorooctanoic acid ("PFOA") and perfluorooctane sulfonate ("PFOS")., The commenter states that, if finalized, the proposed rule would prevent these EPA actions. The commenter notes that when EPA issued health advisories for these two chemicals in 2016, the Health Effects Support Documents relied extensively on epidemiological studies generated by the C8 Health Project. The commenter states that a key component of the evidence for the harmfulness of these chemicals consists of epidemiological studies based on data that are not publicly available. The commenter states that researchers published more than three dozen papers based on these data, identifying probable links between PFOA exposure and "diagnosed high cholesterol, ulcerative colitis, thyroid disease, testicular cancer, kidney cancer, and pregnancy-induced hypertension." The commenter states that this situation underlines the arbitrariness and irrationality of the proposed rule: on the one hand, EPA is proposing to take regulatory action to protect the American people from emerging health threats and on the other -- through the proposed rule -- it is simultaneously undermining its own ability to follow through on those proposals.
Response: Based on a consideration of these and other similar comments, the EPA proposed, through a 2020 SNPRM, a modified version of the 2018 proposed rulemaking regulatory text at 40 CFR 30.5. In the 2020 SNPRM, the EPA proposed Agency consideration of studies where there is tiered access to data and models that have CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects. For all other studies, data and models should be publicly available if the studies are to be used as pivotal regulatory science or pivotal science. In the 2020 SNPRM, the EPA also proposed an alternative. Under alternative proposed 40 CFR 30.5, when promulgating significant regulatory decisions or finalizing influential scientific information, the Agency would, other things equal, give greater consideration to studies where the underlying data and models are available in a manner sufficient for independent validation either because the information is publicly available or available through tiered access when the data include CBI, proprietary data, or PII and appropriate techniques have been used to reduce the risk of re-identification.
Based on a consideration of public comments received in response to the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. 
As described in the preamble to the final rule, the transparency provisions of the final rule apply to dose-response data underlying pivotal science. Under this rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule under 40 CFR 30.7.
The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information. 
Comment: Commenter (6903) states that the data disclosure requirements in the Proposed Rule would make it impossible for the EPA to issue or update its rules and standards, preventing the EPA from fulfilling its statutory duties. The commenter states that EPA has an explicit mandate to protect the public's health from "known or anticipated" harm and must rely on science to establish the evidence or potential for harm. The commenter states that, by requiring that the EPA use only the science where data can be made publicly available, the Proposed Rule would create an undue burden on the Agency that would effectively prevent the Agency from passing new or updating existing standards and rules. 
Response: Based on a consideration of these and other similar comments, the EPA proposed, through a 2020 SNPRM, a modified version of the 2018 proposed rulemaking regulatory text at 40 CFR 30.5. In the 2020 SNPRM, the EPA proposed Agency consideration of studies where there is tiered access to data and models that have CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects. For all other studies, data and models should be publicly available if the studies are to be used as pivotal regulatory science or pivotal science. In the 2020 SNPRM, the EPA also proposed an alternative. Under alternative proposed 40 CFR 30.5, when promulgating significant regulatory decisions or finalizing influential scientific information, the Agency would, other things equal, give greater consideration to studies where the underlying data and models are available in a manner sufficient for independent validation either because the information is publicly available or available through tiered access when the data include CBI, proprietary data, or PII and appropriate techniques have been used to reduce the risk of re-identification.
Based on a consideration of public comments received in response to the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. 
As described in the preamble to the final rule, the transparency provisions of the final rule apply to dose-response data underlying pivotal science. Under this rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule under 40 CFR 30.7.
The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
4.1.1.6 Recommended Changes to and Clarifications of the Language in §30.5
4.1.1.6.1 Revise §30.5 to Require Documentation of "All Reasonable Efforts" of Making Data Available Before Concluding that it is Not Possible 
Comment: Commenter (6143) requests that EPA make the following revision to the proposed text under §30.5 (underlined text reflects suggested text additions): 
      The agency shall make and thoroughly document all reasonable efforts to explore methodologies, technologies, and institutional arrangements for making such data available before it concludes that doing so in a manner consistent with law and protection of privacy, confidentiality, national and homeland security is not possible.
Response: The EPA is not finalizing the commenter's suggested language in 40 CFR 30.5. The EPA acknowledges that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. In instances where the EPA gives the pivotal science lesser consideration, the rule requires the Agency to document why lesser consideration was given per 40 CFR 30.5(c).
4.1.1.6.2 Revise §30.5 to Clarify How EPA Will Work with Third Parties to Make Data Available
Comment: Commenter (6125) states that the Proposal fails to explain how it is meant to function for third party-conducted studies (for example, the text of the Proposal says, "[w]here data is controlled by third parties, EPA shall work with those parties to endeavor to make the data available in a manner that complies with this section" and does not explain "how" (§30.5)).
Response: The EPA has removed this language from the final rule.
4.1.1.6.3 Supports EPA Granting Greater Weight to Studies, Data, and/or Models Where Underlying Data are Publicly Available
Comment: Commenter (6918) agrees that, as a general principle, where practicable, EPA should grant greater weight to studies, data, and/or models for which the underlying data are publicly available. The commenter states that the EPA's preference for these studies is another way for EPA to increase transparency and enhance the value of public participation without violating data access limitations, such as privacy or confidentiality. The commenter adds that such a preference also encourages third parties, when they design their studies or data-collection efforts, to make all reasonable efforts to ensure their data and models can be made public, perhaps in masked or anonymized form. The commenter suggests that incentivizing third parties to do so also demonstrates EPA's commitment to transparency, public participation, and validity.
Response: The EPA agrees with the commenter and in this final rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation.
4.1.1.6.4 Revise §30.5 to Clarify What Constitutes "Independent Validation"
Comment: Commenters (6126, 6137, 6185, 6447, 6906) provide that §30.5 of the Proposal states: "the Agency shall ensure that dose response data and models underlying pivotal regulatory science are publicly available in a manner sufficient for independent validation," and that "independent validation" is defined as "information necessary for the public to understand, assess, and replicate findings." Commenter (6137) asserts that the EPA provides no indication what this means or how it will "ensure" or what EPA will do if it is not possible to ensure this. Similarly, commenter (6164) states that the term "validation" and is not defined and suggests that what EPA means be clarified. 
Commenters (6126, 6185, 6447) state that the approach for "independent validation" is not described and that the definition for "independent validation" needs to be further clarified. Commenters (6185, 6447) assert that EPA already has procedures in place for allowing independent validation that work and it is unclear what additional measures EPA is now proposing. Commenter (6126) requests that the EPA be clear about the intent and the goal for addressing independent validation.
Commenters (6125, 6126, 6162) also state that the Proposal does not define "replicate." Commenter (6126) notes that the Proposal uses both the terms "replicate" and "reproduce" and that, in scientific practice, replication and reproducibility can be interpreted as different practices where one relies on the precise data used in an initial analysis and another relies on similar methods and data. Commenter (6125) asserts that, not defining "replicate" in the rule is yet another instance of a failure to provide adequate notice. The commenter states that, in context, EPA is using replicate not in the sense of conducting a new study to determine if the original conclusions also are supported by analyses of new data, but whether the results are "reproducible" by other analysts using the original data. According to the commenter, read literally, requiring studies be reproducible as well as `transparent' would exclude the use of the following:
 Studies of the effects of natural or human-induced disasters and interventions on health and the environment 
 Studies of human exposures to historically high concentrations of environmental pollutants or to occupational exposures that could not ethically be reproduced.
Response: The EPA has defined "independent validation" in 40 CFR 30.2 as: "the reanalysis of study dose-response data by subject matter experts who have not contributed to the development of the study to evaluate whether results similar to those reported in the study are produced."
4.1.1.6.5 Revise §30.5 to Clarify Primary Data Record Needed for Independent Validation and Analysis 
Comment: Commenter (5419) states that §30.5 specifies the type of information that would be made public to allow independent validation. The commenter expresses concern that for complete independent validation and analysis, as stated in the Proposal, the primary data record would be required. According to the commenter, notwithstanding privacy concerns, which is covered under other sections of the Proposal, public posting of such large quantities of data would be unwieldly. The commenter contends that the interpretation of the term "data" has critical implications and requests that the EPA clarify its intent.
Response: The EPA has clarified in this final rule that 40 CFR 30.5 only concerns the availability of dose-response data underlying pivotal science in significant regulatory actions or influential scientific information. Where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies --  based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements or quantitative analyses of an EPA significant regulatory action or influential scientific information will be identified as pivotal science.
4.1.1.6.6 Revise §30.5 to Clarify Circumstances Where a Study Can be Used When Underlying Data Cannot be Made Publicly-Available
Comment: Commenter (6362) states that §30.5 refers specifically to the requirements that apply when "EPA uses dose response data and models underlying "pivotal" (which the commenter believes is more aptly expressed as "material") regulatory science." The commenter interprets this to mean that in these specific circumstances, the dose response data and models must be "publicly available in a manner sufficient for independent validation," which EPA defines as in a manner "consistent with law, protects privacy, confidentiality, confidential business information, and is sensitive to national and homeland security." The commenter states that information is considered "publicly available in a manner sufficient for independent validation" when it includes the information "necessary for the public to understand, assess, and replicate findings." The commenter asserts that, for environmental safety, exposure-response is the more appropriate relationship to evaluate because most of the environmental test guidelines require quantifying concentrations in media external to the organism for use as the exposure metric. The commenter requests that the EPA provide greater clarity regarding what it intends to do in circumstances where raw data cannot be made publicly available.
Commenter (6918) supports enhanced transparency in EPA rulemakings but recognizes the vital importance of privacy, confidentiality or personal and business information, and national and homeland security as limits on data access. The commenter recommends that EPA more-clearly detail an objective and consistent process that would determine when release of data underlying pivotal studies would be deemed inappropriate because it would impair privacy, confidentiality, and/or national security. The commenter also recommends that EPA specify the conditions under which it could rely on studies for which the underlying data cannot appropriately be released.
Commenter (0562) states that not all confidential or proprietary information is needed for qualified reviewers to make their weight of evidence (WOE) calls on the merits of the science, the study, or model results.
Response: The EPA disagrees with the commenter that the EPA should consider the availability of exposure-response data. The EPA is focusing on the underlying dose-response data for this rulemaking because of the influence these data have on particularly impactful decisions at the Agency. Risk assessments and regulations that target emissions and risk reduction of one or more pollutants, contaminants, or substances are integral to the Agency's mission and the underlying dose-response data that inform the quantitative value used to evaluate and mitigate potential risk are critical to understanding the assessment or regulatory action. In addition, the data underlying the dose-response assessment are more distinct than the broad range of data informing an entire risk assessment. Therefore, the EPA is concentrating its current efforts to increase transparency on a well-defined step in the quantitative assessment of risk supporting specific Agency actions. This final rule provides an important step in furthering the progress already being made toward maximizing transparency and will provide important insight for developing future statute-specific requirements.
In this final rule, the EPA has described how it will consider studies for which the underlying data cannot appropriately be released. The EPA will evaluate if the dose-response data underlying pivotal science are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. 
There may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
4.1.1.6.7 Revise §30.5 to Clarify How a Determination Will be Made on Information Sufficiency for the Public to Understand, Assess and Replicate Findings
Comment: Commenter (6126) states that it is not clear from the language in §30.5 on how a determination will be made on the information, or how much and what kinds of information satisfy the requirement that researchers must provide enough information for "the public to understand, assess, and replicate findings." The commenter notes that the list of types of information that may be required based on the language of §30.5 suggests an intent to be flexible, but the language needs to be clarified. 
Commenter (6125) states that proposed §30.5 is so awkwardly drafted that it is not clear if it is intended as an absolute requirement or is advisory. The commenter states that the language in footnote 3, and in §30.5 suggest that it is a requirement. The commenter states that other language does not, e.g, §30.5 (information can be used if it provides the information necessary to "replicate" findings).
Response: The types of information provided in 40 CFR 30.5(f), which are necessary for the public to understand, assess, and reanalyze findings, are provided as examples. The EPA intends to issue implementation guidelines and statute-specific rulemakings that will further describe the implementation of this rule. The EPA clarifies that the procedures codified in 40 CFR 30.5 of this final rule are requirements for the Agency.
4.1.1.6.8 Revise §30.5 to Add Requirement that Information be Sufficient for Verification/Change "Computer Codes" to "Computer Algorithms" 
Comment: Commenter (6362) makes the following suggested edits to §30.5 (underlined text is the suggested new text and crossed-out text is the suggested deletion):
      Section 30.5 What requirements apply to use of dose response data and models? 
      When promulgating significant regulatory actions, the Agency shall ensure that the dose response data and models underlying pivotal regulatory science are publicly available in a manner sufficient for independent validation, verification, and analysis. This may include: 
             Data (where necessary, could be subject to access and use restrictions)
             Associated protocols
             Computer algorithmscodes and models
             Recorded factual materials
             Detailed descriptions of how to access and use such information 
      But in a manner consistent with law, protects privacy, confidentiality, CBI, and is sensitive to national and homeland security. Information is "publicly available in a manner sufficient for independent evaluation" when it includes the information necessary for the public to "understand, assess, and replicate findings."
Response: The EPA is unsure what the commenter's rationale is for suggesting this change. The EPA has altered the text at 40 CFR 30.5(f)(3) to read, "Computer codes and models involved in the creation and analysis of such information."
4.1.1.6.9 Revise §30.5 to Minimize Exception to the Rule
Comment: Commenter (6867) recommends that EPA minimize exceptions to the rule and accordingly make the following suggested changes to §30.5 of the rule (crossed out regulatory text reflects the commenter's suggested deletions): 
      (a) EPA should amend the second sentence of §30.5 to provide that
      Where the Agency is making data or models publicly available, it shall do so in a fashion that is consistent with law, and protects privacy, confidentiality, confidential business information, is sensitive to national and homeland security.
      (b) EPA should amend the penultimate sentence of §30.5 to provide that
      The agency shall make all reasonable efforts to explore methodologies, technologies, and institutional arrangements for making such data available before it concludes that doing so in a manner consistent with law and protection of privacy, confidentiality, national and homeland security is not possible.
Response: The EPA disagrees with the commenter and has not made this change in the final rule. The EPA is committed to upholding legal protections for privacy, confidentiality, and CBICBI. This is a rule of internal agency procedure and does not conflict with laws governing privacy, confidentiality, CBI, or national security interests. 
4.1.1.6.10 Revise §30.5 to Provide for Disclosure of Dose Response Data and Models "When Available"
Comment: Commenter (6906) states that the EPA does not address how or if it can use studies where specified dose response data and models are not available. The commenter recommends that EPA amend the text of proposed §30.5 to provide for disclosure of dose response data and models "when available." 
Response: The EPA is clarifying in this final rule that the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. 
4.1.1.6.11 Clarify Provisions of §30.5 to Apply at an Interim Phase Prior to Issuance of the Final Rule
Comment: Commenter (6906) suggests that EPA amend §30.5 (disclosure of data and models) to require transparency steps at an interim phase prior to issuance of the final rule.
Response: The EPA agrees with the commenter and has altered the definition of "publicly available" at 40 CFR 30.2 to include "The public must be able to access the information on the date of publication of the proposed rule (or, as appropriate, a supplemental notice of proposed rulemaking, or notice of availability) for the significant regulatory action or on the date of dissemination of the draft influential scientific information for public review and comment."
4.1.1.7 Recommended Changes to and Clarifications of the Language in §30.6
4.1.1.7.1 §30.6 Does Not Provide the Necessary Details to Assure Meaningful Public Review and Comment
Comment: Commenter (6137) states that §30.6 addresses the "additional requirements" related to the use of dose response data and models underlying "pivotal regulatory science," but it is devoid of details necessary to assure meaningful public review and comment on how, when, or in what way EPA will implement this. The commenter also states that §30.6 cites no specific science and gives no indication of how or why EPA is attempting to redefine scientific information on dose response differently than it has done in the past to allow for informed consideration of why EPA is even attempting to address this issue.
Response: The EPA has removed proposed 40 CFR 30.6, and has transferred some of the requirements of that section to 40 CFR 30.5 of the final rule. However, the EPA disagrees with the commenter that there is insufficient information to review and comment. The EPA is establishing additional requirements for how it considers pivotal science because these studies are the most critical to the Agency's significant regulatory actions and influential scientific information. The EPA finds that limiting the scope of these requirements to pivotal science will still provide meaningful and impactful opportunity for reanalysis. For example, Lewandowsky et al. (2020) evaluated the cost-effectiveness of reanalysis studies under various scenarios and concluded that reanalysis studies are most cost-effective when they are focused on studies of the greatest interest to the scientific community (in this study, the number of citations was a surrogate for interest).
4.1.1.7.2 Clarify Case-by-Case Default Assumption Evaluations Under §30.6
Comment: Commenter (2169) requests that EPA clarify how it will decide, on a case-by-case basis, what default assumptions, including assumption of a linear, no-threshold dose response, are appropriate. The commenter asks:
 What if the sensitivity of the modeled results to alternative assumptions is itself not subject to scientific consensus? 
 Won't any regulated industry potentially subject to a proposed regulation be able to exaggerate the impacts of assumptions based on disputed sensitivity to effectively diminish in the public mind the cost/benefit ratio of a proposed regulation? 
 Will such an industry-inspired public confusion be sufficient to establish the procedural inadequacy of a proposed regulation?
Response: The EPA has removed proposed 40 CFR 30.6, and placed some of the requirements in 40 CFR 30.5(e). Of the language quoted by the commenter, "including assumptions of a linear, no threshold dose-response" has been removed. Therefore, under 40 CFR 30.5 of this final rule, the EPA shall evaluate the appropriateness of using default assumptions on a case-by-case basis used in its dose-response assessment. Further, the EPA shall clearly explain the scientific basis for critical assumptions used in the dose-response assessment that the EPA relied on for the significant regulatory action or influential scientific information. The EPA disagrees that these requirements, imposed solely on EPA, will introduce increased bias or arbitrariness, as the commenter suggests. Rather, the EPA finds that openly providing the scientific rationale for its critical assumptions used in dose-response assessments will improve the assessment's objectivity and clarity to the public.
4.1.1.7.3 Oppose §30.6 Requirement to Evaluate Appropriateness of Using Default Assumptions
Comment: Commenter (6447) states that the proposed rule, contrary to the scientific community practice, would have EPA "evaluate the appropriateness of using default assumptions, including assumptions of a linear, no-threshold dose response, on a case-by-case basis." (Proposed §30.6) According to the commenter, requiring EPA to justify default models every time it undertakes a risk assessment or rulemaking will slow the regulatory process, and increase its cost, with no benefit and the appropriate way to deal with dose-response models is to evaluate deviations from the default model on a case-by-case basis. Commenter (6133) contends that this "case-by-case" provision injects additional arbitrariness into the rule, in that it ensures that the Proposal may be applied unevenly -- for certain rulemakings the "rules" of the Proposal can be discarded or ignored where desired. 
Commenter (9227) provides that the National Academies' Science and Decisions document recommends that 1) "EPA should continue and expand use of the best, most current science to support or revise its default assumptions," 2) "work toward the development of explicitly stated defaults to take place of implicit or missing defaults," and 3) that "departure [from defaults] should occur only when the evidence of the plausibility of alternatives is clearly superior to the evidence of the value of the default." According to the commenter, these recommendations underscore and reaffirm the role of defaults, and make clear that deviations from defaults are to be considered carefully, on a case-by-case basis, and only when adequately justified. The commenter also provides that this deviates from EPA's Proposal's interest and intent to move away from "default models, without consideration of alternatives or model uncertainty" which purportedly "can obscure the scientific justification for EPA actions."
Commenter (6373) suggests that limiting researchers' ability to use modeling and assumptions as they deem appropriate will have human health consequences because scientists may be tempted to use models that EPA has deemed acceptable but that are not the best fit for their research needs.
Commenter (6446) suggests that requiring justification for all default assumptions ignores establishment of defaults through prior public processes and external public peer reviews. The commenter adds that requiring EPA to justify the use of default assumptions for every toxicity health factor derivation ignores the detailed development process of those default assumptions. The commenter states that those default assumptions were developed in several EPA documents (including the 2005 Guidelines for Carcinogen Risk Assessment), which received public comment and extensive external peer-review. Requiring EPA to rejustify those default assumptions every time they are used, as the proposed rule suggests, would serve no purpose other than to add unnecessary delay to risk assessments and regulatory action.
Response: The EPA has removed proposed 40 CFR 30.6, and placed some of the requirements in 40 CFR 30.5(e). The EPA has altered the language subject to these comments to read, "the EPA shall also describe critical assumptions and methods used in its dose-response assessment and shall characterize the variability and uncertainty of the assessment. The EPA shall evaluate the appropriateness of using default assumptions on a case-by-case basis. The EPA shall clearly explain the scientific basis for critical assumptions used in the dose-response assessment that the EPA relied on for the significant regulatory action or influential scientific information." The EPA agrees with the commenters that the EPA should not be required to describe all default assumptions and methods it uses, and thus has limited this requirement to "critical assumptions and methods."
In this final rule, the EPA is only required to describe critical assumptions and methods, and anticipates that the added time and cost of this practice to EPA will be small. The EPA agrees with the commenter that, "deviations from defaults are to be considered carefully, on a case-by-case basis, and only when adequately justified." This matches the requirements of 40 CFR 30.5(e).
The EPA disagrees with the commenter that this rule will prevent researchers from using modeling and assumptions. This final rule establishes procedural requirements for the EPA and does not regulate the conduct or determine the rights or obligations of any entity outside the federal government. Further, the requirements of 40 CFR 30.5(e) do not dissuade EPA scientists from using models or assumptions, but rather requires them to provide the scientific basis for critical assumptions (not all default assumptions, as proposed).
4.1.1.7.4. Requirement to "Describe and Document Any Assumptions and Methods Used" 
Comment: Commenter (6447) states that proposed §30.6 would require the Agency to "describe and document any assumptions and methods used and should describe variability and uncertainty." The commenter asserts that this is already standard Agency practice, and EPA has not provided any evidence to suggest that its staff has ever failed to follow this practice.
Response: The EPA has removed proposed 40 CFR 30.6, and placed some of the requirements in 40 CFR 30.5(e). The EPA has altered the language subject to these comments to read, "the EPA shall also describe critical assumptions and methods used in its dose-response assessment and shall characterize the variability and uncertainty of the assessment. The EPA shall evaluate the appropriateness of using default assumptions on a case-by-case basis. The EPA shall clearly explain the scientific basis for critical assumptions used in the dose-response assessment that the EPA relied on for the significant regulatory action or influential scientific information." The EPA agrees with the commenter that this is standard Agency practice and has determined to codify it in regulation.
Comment: Commenter (6449) suggests changing the regulatory text of §30.6 to read that: 
      In the main text of an analysis, EPA shall describe and document any assumptions and methods used, and should clearly present, in the main text, especially any procedure used for the first time in that type of evaluation (e.g., IRIS or TSCA) as well as presenting the results of using the standard procedure.
The commenter explains that deviations from standard procedures and guidance are expected, especially in cases where more than the usual amounts and types of data are available. According to the commenter, highlighting them, providing information on how the results differ from that which would result from "standard" procedures, and requesting specific review of such changes would save reviewers from each trying to discover and evaluate this information.
Response: The EPA does not understand the commenter's suggestion that the describing and documenting assumptions and methods must occur in the main text of an analysis and has not made these changes to the regulatory text. The EPA will fulfill the requirements of this final rule in a manner that is clear and accessible to the public.
Comment: Commenter (6906) requests that the EPA amend §30.6 to require EPA make available analysis of how it weighs and/or evaluates the conclusions drawn in a study based on criteria of "best available science."
Response: In addition to following this rule, the EPA will continue to follow its guidance and guidelines in assembling dose-response assessments. Consistent with existing Agency practice, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information. As discussed in the final rule, the EPA will continue to use the following, established factors to assess the quality of studies used to develop significant regulatory actions and influential scientific information: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review., When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Comment: Commenter (0068) suggests that, in addition to describing and documenting any assumptions and methods used, EPA simultaneously make clear that its statutory responsibility is to protect public health, and hence in making decisions based on such evidence it will adopt the public health decision rule, "when in doubt, act to protect public health and wellbeing," as opposed to the classic decision rule used in scientific publishing, "place greatest emphasis on avoiding false positives."
Response: In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers.
4.1.1.7.5 Revise §30.6 to Clarify that it Requires Documentation of Uncertainty and Variability 
Comment: Commenter (6143) requests that the EPA clarify that §30.6 sets forth a mandatory duty and requires documentation of uncertainty and variability commensurate with its requirement for other "assumptions and methods used" as described in this proposed provision. The commenter recommends that EPA replace the proposed text "and should describe variability and uncertainty . . ." with "and shall describe and thoroughly document variability and uncertainty . . ."
Response: The EPA agrees with the commenter and has revised the language (now in 40 CFR 30.5(e)) to read, "the EPA shall also describe critical assumptions and methods used in its dose-response assessment and shall characterize the variability and uncertainty of the assessment."
4.1.1.7.6 Revise §30.6 to Require, as a Default, the Application of Causal Analytics
Comment: Commenter (6143) requests that EPA require, as a default, the application of causal analytics in studies that include, but are not limited to, dose-response or concentration-response models or data, and models that investigate factors that might account for spatial heterogeneity. The commenter recommends that EPA revise the first sentence of §30.6 to read as follows:
      EPA shall describe and document any assumptions and methods used. It should describe variability, uncertainty, and treatment of missing data, exposure estimation errors, covariate estimation and classification errors, omitted explanatory variables, model specification errors and biases, data selection biases, missing data, assumptions made to support causal interpretations of dose-response or exposure-response data (e.g., unit homogeneity, no hidden confounders, etc.), and results of any tests performed to validate these assumptions and to rule out non-causal interpretations and threats to internal and external validity of causal inferences.
Response: The EPA does not understand the commenter's rationale for suggesting these revisions and has not made these changes in the final rule.
4.1.1.7.7 EPA Should Also Consider High-Quality Studies that Address Collider Biases and Latent as Well as Observed Confounders and Colliders
Comment: Commenter (6143) suggests that EPA also consider high-quality studies that address: (a) collider biases (e.g., selection, stratification, and Berkson's bias), and (b) latent as well as observed confounders and colliders.
Response: The EPA has removed proposed 40 CFR 30.6, and has removed the requirement that the EPA give explicit consideration to high quality studies that explore[...]. Instead, in 40 CFR 30.5(d) in determining the amount of consideration to grant pivotal science for which the underlying dose-response data are not available, the EPA may consider the study's consideration of a broad range of parametric dose-response or concentration-response models, a robust set of potential confounding variables, nonparametric models that incorporate fewer assumptions, various threshold models across the dose or exposure range, and models that investigate factors that might account for spatial heterogeneity. The EPA does not understand the commenter's rationale for suggesting this revision and has not made the suggested changes to the final rule.
4.1.1.7.8 Requirement to "Explain the Basis for Each Model Assumption and Present Analyses Showing the Modeled Results to Alternative Assumptions" 
Commenter: Commenter (6125) asserts that the most puzzling, and perhaps costly requirement in the rule appears in the wording of §30.6 of the Proposal: 
      EPA shall clearly explain the scientific basis for each model assumption used and present analyses showing the sensitivity of the modeled results to alternative assumptions. 
The commenter states that this language appears to require the Agency not only to explain the basis of each model-related assumption in the original study but also to provide an analysis of the original data using alternative assumptions. The commenter asserts that the EPA provides no reasons why this should be an automatic and invariable requirement for all studies identified as "pivotal."
Response: The EPA has eliminated the language cited, and has replaced it in 40 CFR 30.5(e) with: "The EPA shall clearly explain the scientific basis for critical assumptions used in the dose-response assessment that the EPA relied on for the significant regulatory action or influential scientific information." By limiting this to critical assumptions and removing the requirement for sensitivity analysis, the EPA has addressed the commenter's concern.
Comment: Commenter (6924) states that the proposal requires that "EPA shall clearly explain the scientific basis for each model assumption used and present analyses showing the sensitivity of the modeled results to alternative assumptions. When available, EPA shall give explicit consideration to high quality studies that explore: A broad class of parametric dose-response or concentration-response models; a robust set of potential confounding variables; nonparametric models that incorporate fewer assumptions; various threshold models across the dose or exposure range; and models that investigate factors that might account for spatial heterogeneity." The commenter asserts that these mandates inappropriately dictate how scientific studies should be conducted and are not supported by current science.
Response: The EPA has removed proposed 40 CFR 30.6, and has removed the language cited by the commenter. Instead, in 40 CFR 30.5(d) in determining the amount of consideration to grant pivotal science for which the underlying dose-response data are not available, the EPA may consider the study's consideration of a broad range of parametric dose-response or concentration-response models, a robust set of potential confounding variables, nonparametric models that incorporate fewer assumptions, various threshold models across the dose or exposure range, and models that investigate factors that might account for spatial heterogeneity. This requirement will not dictate how scientific studies should be conducted and therefore addresses the commenter's concern.
Comment: Commenter (6446) states that the requirement for sensitivity analysis for all alternative-modeling assumptions does not require an evaluation of the scientific validity of those models. The commenter adds that the proposed rule's requirement that dose-response modeling include a sensitivity analysis including all alternative modeling assumptions does not make any mention of how those assumptions would be scientifically justified. The commenter suggests that this lack of scientific justification criteria would invite the submission of models that would not have biological relevance for inclusion in the sensitivity analysis. According to the commenter, this would result in a process that would make EPA risk assessment documents unwieldy, unmanageable and unreliable.
Response: The EPA has removed the requirement for sensitivity analysis for all alternative-modeling assumptions in the final rule, and therefore has addressed the commenter's concern.
4.1.1.7.9 Use of Term "Uncertainty" in §30.6 of the Proposal is Vague
Comment: Commenter (6915) states that the proposed rule would require that EPA "describe and document any assumptions and methods used . . . and uncertainty." 83 Fed. Reg. at 18,774. However, the commenter notes that uncertainty is not defined in the proposed rule, and it is unclear what type of uncertainty is implied. The commenter provides that uncertainty could mean discussion of the magnitude of the statistically-based range of model predictions. The commenter adds that there could also be uncertainties unrelated to the model, such as qualitative uncertainty about the human relevance of the animal toxicity endpoint used as the basis for the risk assessment. According to the commenter, the EPA's failure to define the type of uncertainty at issue makes the proposed rule impermissibly vague and deprives the public of a meaningful opportunity to comment on its impacts.
Response: The EPA disagrees that the term "uncertainty" is vague or that there is significant ambiguity about what should be in the scope of a characterization of uncertainty. The characterization of uncertainty is a key factor in the assessments that the EPA conducts. It is a component of various EPA guidelines (e.g., Framework for Human Health Risk Assessment to Inform Decision Making) that the EPA relies upon in conducting its assessments. The scope of the uncertainty analyses that the EPA conducts necessarily varies across assessments and actions. The intent of this regulation is not to force uncertainty analyses into a one-size-fits-all approach, as that is not practical, good policy, or good science. Thus, a regulation governing internal procedures, such as this one, does not require a regulatory definition for a term that is already a key component of current EPA practices and guidelines and EPA's assessment process.
4.1.1.7.10 Recommend that EPA Quantify the Impact of Key Uncertainties Where Possible
Comment: In addition to the requirements provided in §30.6 of the Proposal, commenter (6921) recommends that EPA evaluate quantitatively the impact of key uncertainties and assumptions where possible in an integrated fashion, so that the public can understand the cumulative impact of these assumptions and uncertainties in assessing potential risks and hazards.
Response: The EPA disagrees with the commenter that how uncertainty is treated in dose-response assessments should be part of this final rule. The scope of the uncertainty analyses that the EPA conducts necessarily varies across assessments and actions. The intent of this regulation is not to force uncertainty analyses into a one-size-fits-all approach, as that is not practical, good policy, or good science.
4.1.1.7.11 Revise §30.6 to Emphasize the Use of National and International Guidance on the Measurement of Uncertainties
Comment: Commenter (0560) requests that the EPA emphasize the use of national and international guidance on the measurement of uncertainties, giving special attention to Type B errors. The commenter suggests that EPA amend §30.6 as follows (underlined text reflects suggested new regulatory text and crossed out text reflects suggested regulatory text deletions): 
      EPA shall describe and document any all assumptions and methods used, . and should EPA shall describe variability and address replicability,. and EPA shall systematically discuss uncertainty in measurement per national and international guidelines, particularly addressing Type B errors." Taylor NIST 1994, JCGM 100:2008.
The commenter also requests that EPA add the following language to §30.6 (underlined text reflects suggested new regulatory text):
      EPA shall review other relevant data and models found or brought to its attention, and discuss why those were not included.
Response: The EPA disagrees with the commenter that how uncertainty is treated in dose-response assessments should be part of this final rule. The characterization of uncertainty is a key factor in the assessments that the EPA conducts. It is a component of various EPA guidelines (e.g., Framework for Human Health Risk Assessment to Inform Decision Making) that the EPA relies upon in conducting its assessments. The scope of the uncertainty analyses that the EPA conducts necessarily varies across assessments and actions. The intent of this regulation is not to force uncertainty analyses into a one-size-fits-all approach, as that is not practical, good policy, or good science. 
4.1.1.7.12 Supports Use of Policy or Guidance Documents for Requirements Specified in §30.6
Comment: Commenter (6920) agrees with the first sentence of §30.6 ("EPA shall describe and document any assumptions and methods used, and should describe variability and uncertainty"), however, the commenter states that the remainder of the language in this section, if still desired after further evaluation, would be better placed in policy or guidance documents.
Commenter (6125) suggests that, if EPA has issues with its current risk assessment and benefit analysis procedures that currently follow NAS recommendations and its own existing guidelines, a more straightforward and fruitful approach would be to update the guidelines as needed, showing in far more detail than provided here the details of their reasoning, the conditions in which they should be employed, and the scientific bases for requiring consideration of alternative models in all risk assessments. The commenter adds that the draft guidelines should then undergo peer review by expert panels of the NAS and/or its own SAB and SAP. According to the commenter, while guideline development is arduous and time consuming, it has several advantages over rulemaking, including 1) guidance is developed by experienced risk assessors; 2) it is a science-based process that involves consensus building across the EPA as well as with the broader science community; and, 3) critically, it maintains the separation between risk assessment and risk management so that EPA's decision makers benefit from the availability of science that is produced in an independent and objective fashion. This has been the practice at EPA since 1983 when the NAS published Risk Assessment in the Federal Government: Managing the Process. 
Response: The EPA disagrees that the requirements of this rule are in conflict with or indicate issues with current Agency guidelines and guidance. Consistent with existing Agency practice, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information. The EPA will continue to use the following, established factors to assess the quality of studies used to develop significant regulatory actions and influential scientific information: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review., When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
4.1.1.7.13 Supports §30.6 Proposed Regulatory Language
Comment: Commenters (6150, 6918, 6921) support the principles and "additional requirements" reflected in the proposed §30.6 and believe that the proposed §30.6 language reasonably reflects what ought to be EPA's consistent practice in all rulemakings. 
Response: The EPA agrees with the commenter. The Agency has removed proposed 40 CFR 30.6, and placed some of the proposed requirements in 40 CFR 30.5(d)-(e).
4.1.1.7.14 Clarify Provisions of §30.6 to Apply at an Interim Phase Prior to Issuance of the Final Rule
Comment: Commenter (6906) suggests that EPA amend §30.6 (documentation and evaluation of studies) to require transparency steps at an interim phase prior to issuance of the final rule.
Response: The EPA agrees with the commenter and has altered the definition of "publicly available" at 40 CFR 30.2 to include "The public must be able to access the information on the date of publication of the proposed rule (or, as appropriate, a supplemental notice of proposed rulemaking, or notice of availability) for the significant regulatory action or on the date of dissemination of the draft influential scientific information for public review and comment."
4.1.1.7.15 EPA Should Not Base Any Regulatory Determination on a Single Study
Comment: Commenter (9224) encourages EPA to not base any regulatory determination on a single study, regardless of whether or not the data is publicly available. The commenter suggests that relying on a single study, no matter how robust it may be, can bring bias to a model or regulatory decision. If EPA must base a decision off of a single study, the commenter states that it should be imperative that the data be publicly available as well as the reasoning and methodology behind why the study was chosen, how it is being used and why no other studies were deemed sufficient to be included.
Response: The EPA agrees with the commenter and clarifies in this final rule that, consistent with existing Agency practice, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information. The EPA will continue to use the following, established factors to assess the quality of studies used to develop significant regulatory actions and influential scientific information: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review., When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. 
The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
4.1.1.8 Study Exceptions/Exemptions from Public Disclosure Requirements
4.1.1.8.1 EPA Should Minimize Study Exceptions to Public Disclosure Requirements 
Comment: Commenter (6143) states that EPA should ensure that any final action it takes on the proposed rule encourages transparency while making limited exceptions to accommodate legitimate, protected privacy interests such as CBI.
Commenter (6375) requests that EPA minimize the use of exceptions to the extent feasible within this rulemaking, but they understand that there will be some situations in which exceptions are needed. Regardless of the criteria used, the commenter contends that it is critical that EPA provide a robust and transparent explanation in the Federal Register that explains in detail why the data and models in question cannot be made publicly available. 
Response: The EPA generally agrees with the commenters and has finalized requirements that protect privacy, confidentiality, CBICBI, and national security interests. As described in the final rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. 
Additionally, under 40 CFR 30.7, the Administrator may grant an exemption to the requirements in 40 CFR part 30 for a study on a case-by-case basis if he or she determines that greater consideration is warranted because making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, CBICBI, or national security.
Comment: Commenter (6867) states that, in view of the widespread problems with replicating even peer-reviewed studies published in prominent journals, the EPA should minimize exceptions to the disclosure requirements set forth in the Proposed Rule. The commenter notes that Courts have frequently rejected attempts to withhold information based on claims to privacy, etc., unless protections are specifically provided by law. See, e.g., Johnson v. Dovey, 2011 U.S. Dist. LEXIS 128577, at *6-7 (E.D. Cal. Nov. 7, 2011) (rejecting vague assertion of privacy rights: "CDCR's objections are not specific in any way, and are thus insufficient to assert a privilege."); see also Donovan v. Nat'l Bank of Alaska, 696 F.2d 678 (9th Cir. 1983) ("The bank cannot refuse to comply with the subpoena as a whole on the basis of its vague allegations that it might be required . . . to produce records in violation of the Financial Privacy Act."). The commenter requests that EPA similarly reject such claims to "privacy, confidentiality, [and] confidential business information" vis-à-vis data and models, unless those claims are clearly supported by law or the Executive Branch's inherent powers. Accordingly:
      (a) EPA should amend the second sentence of Section 30.5 to provide that
      Where the Agency is making data or models publicly available, it shall do so in a fashion that is consistent with law and is sensitive to national and homeland security.
      (b) EPA should amend the penultimate sentence of Section 30.5 to provide that
      The agency shall make all reasonable efforts to explore methodologies, technologies, and institutional arrangements for making such data available before it concludes that doing so in a manner consistent with law and protection of national and homeland security is not possible.
Further, the commenter states that, in view of the evidence that even studies published in prominent journals are often false or overstate results, the EPA should carefully consider the weight given to studies, data, and models that are not disclosed for review by the public and other researchers, even where non-disclosure is required by law.
Response: The EPA agrees that the Agency should carefully consider the weight given to pivotal science where the underlying dose-response data are not available for independent validation. When determining the degree of consideration to afford pivotal science for which the underlying dose-response data are not available for independent validation, the EPA will consider the factors included in 40 CFR 30.5(d). These factors include, for example, the extent to which there are other studies for which the dose-response data are available and the sensitivity of the conclusions in the significant regulatory action or influential scientific information based on the use of the study. 
The EPA does not agree with the commenter's suggested changes to the relevant language in proposed 40 CFR 30.5. Given public comments expressing concerns related to the potential release of PII or confidential information, the Agency believes that it is important to acknowledge the need to protect privacy, confidentiality, and national security. 
4.1.1.8.2 Criteria for Excepting/Exempting Underlying Data from Public Disclosure Requirements
Comment: Commenters (6125, 6446) state that the notice of the proposed rule solicits comments on which criteria it should base exceptions to the rule, including whether case-by-case exceptions to the rule may be appropriate but has provided no suggestions of its own on that subject.
Commenter (6446) argues that, because the underlying premise of the rule is flawed, and the rule is unnecessary, unlawful, arbitrary, and harmful, they decline to provide suggestions for methods to determine exceptions to the rule. The commenter argues that there are no criteria or ad hoc methods for making exceptions to the rule that would not pose an unnecessary risk of constraining use of the best available science to make regulatory decisions, particularly where political appointees rather than expert scientific panels, such as the SAB, would apply the criteria or make the ad hoc determinations. 
Additionally, commenter (6446) asserts that there are no criteria or ad hoc methods that would prevent stalling vital decisions to protect public health and the environment by "analysis paralysis." The commenter states that the proposed rule, if adopted, would inevitably delay setting protective standards through prolonged evaluation of the sufficiency of the public availability of research data and methods, rather than evaluation of the actual quality and import of scientific research. The commenter adds that, a related likely consequence of the proposed rule, even if it contains ad hoc mechanisms for exceptions, is a reduction in EPA's ability to respond quickly to emerging challenges, when data and models take time to be made publicly available and/or redacted and otherwise prepared in a format appropriate for public review.
Response: In response to similar comments on the 2018 proposed rule and the 2020 SNPRM, the EPA is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements.
The EPA has finalized the Administrator's exemption provision at 40 CFR 30.7. As described in the preamble to the final rule, the Agency retained the provision because there are conditions under which the Administrator may determine that greater consideration of pivotal science for which the dose-response data are not available in a manner sufficient for independent validation is warranted (e.g., age of the study). To ensure that the Administrator's decision is appropriately transparent, the EPA will document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. The EPA is confident that the Administrator's exemption provision provides sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
The EPA finds that the requirements of the final rule will result in any meaningful delay in promulgating regulations. While the final rule requires the Agency to evaluate the availability of dose-response data for pivotal science, the incremental burden to the EPA to carry out these requirements is expected to be small given (1) the extensive scientific review the EPA already conducts regularly and (2) that the requirement is limited to pivotal science (i.e., typically a small, though highly important, subset of the studies the EPA would review). 
Comment: Commenter (6046) concludes that it would be a waste of resources to identify criteria for exceptions which would be much more objective than determining exceptions on a case-by-case basis. The commenter contends that it is likely that exceptions would be complex (e.g. an exception is to be made if 4 of 5 criteria are met if 3 of the 4 are considered strictly confidential info and the other two are highly confidential). 
Response: The EPA disagrees with this comment and finds that it is appropriate to identify conditions under which the Administrator may determine that greater consideration of pivotal science for which the dose-response data are not available in a manner sufficient for independent validation is warranted (e.g., age of the study). To ensure that the Administrator's decision is appropriately transparent, the EPA will document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. The EPA is confident that the Administrator's exemption provision provides sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7. 
Comment: Commenter (6921) recommends that specific research study exemption decisions should be based on the nature of the data, including whether it is protected from release under current law, the existence of alternative publicly accessible studies of similar quality, and the need to act, based on statutory or judicial deadlines. The commenter suggests that, instead of waiting to apply enhanced transparency requirements during the rulemaking process when it is often too late to acquire and release the data for public comment, EPA should begin now to identify research studies and models that are likely to become pivotal regulatory science in future rulemakings. The commenter suggests that, once EPA has developed a list, it can begin to contact study authors to determine the optimal path forward to increase transparency while assuring continued protection of the confidentiality of research participants and CBICBI. According to the commenter, this effort will reduce the number of exemptions EPA might give in the future.
Response: The EPA agrees with the commenter, in part, and has finalized the Administrator's exemption provision with additional conditions under which the Administrator may determine that greater consideration of pivotal science for which the dose-response data are not available in a manner sufficient for independent validation is warranted. Under 40 CFR 30.7, the Administrator may determine, on a case-by-case basis, that greater consideration is warranted when: 
    Technological or other barriers render sharing of the dose-response data infeasible;
    The development of the dose-response data was completed or updated before the effective date of the final rule;
    Making the dose-response data would conflict with laws and regulations governing privacy, confidentiality, CBI, or national security;
    Third-party independent validation of the study's underlying dose-response data through reanalysis has been conducted; or
    The factors in 40 CFR 30.5 used in determining the consideration to afford to the pivotal science indicate full consideration is justified. 
The EPA would like to emphasize that the final rule does not require the Agency (or other parties) to disclose or host data, but to determine if dose-response data underlying pivotal science are available and to give greater consideration to those studies for which such data are available in a manner sufficient for independent validation. The EPA may opt, at its discretion, to make dose-response data underlying pivotal science available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
4.1.1.8.3 Study Exceptions Should be Recommended by Non-Partisan, Unaffiliated Expert Scientists
Comment: Commenter (4878) states that the proposal grants the EPA Administrator broad authority to exclude individual studies. According to the commenter, this could have broad-reaching impact depending on the preference of the Administrator at the time and allows the Administrator to overrule scientists regarding their own science. The commenter asserts that allowing politically appointed officials to make decisions about whether a study qualifies for an exception is dangerous. According to the commenter, the Administrator already has broad authority to decide what action to take on an item, it should not have the power to hide evidence that does not support the action. The commenter states that the EPA should consider all relevant, peer-reviewed data when making decisions that impact American's health, and the proposed rule's exceptions process clearly undermines this goal. The commenter states that, if the proposed rule takes effect, EPA should at least require that exceptions decisions are made by an expert in the area of research. For example, the commenter recommends that a panel of non-partisan, unaffiliated expert scientists could be used to make recommendations on exceptions.
Response: The EPA does not agree with the commenter regarding the role of the Administrator in determining whether to grant an exemption and finds that the Administrator is the appropriate decision maker in this context. To ensure that the Administrator's decision is appropriately transparent, the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. Additionally, the EPA program or region responsible for the significant regulatory action or influential scientific information and public commenters can provide input when the Administrator is considering an exemption. 
The EPA would like to emphasize that, under the final rule, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. 
4.1.1.8.4 Request for Specific Study Exceptions
Major Scientific Studies Already Completed or Underway
Comment: Commenters (1012, 5165, 6141) request that EPA develop exemptions for scientific studies already completed or underway.
Commenter (6141) suggests that the Agency attempt to accommodate major scientific studies already completed or well underway. If EPA elects to apply the transparency rules retrospectively, the commenter suggests that EPA consider developing exemptions for those past studies for which it is not possible to release a dataset publicly for legitimate reasons pertaining to the use, for example, of human subject data. According to the commenter, in so doing, EPA should strive to establish clear and objective criteria for granting exemptions based on the inability to provide the underlying data while respecting appropriate data access limits.
Commenter (1012) requests that exceptions to the proposed rule include studies that were previously conducted or already underway prior to the implementation of this regulation as the protocols of these studies were set before the rule was finalized. Studies that were conducted and utilized by the Agency to make regulatory decisions prior to the implementation of this rule should be exempted as these studies had met the requirements previously set for inclusion in regulatory decision-making at the time, and the protocols used in these studies were already determined before the rule was finalized. Case-by-case exceptions shrouded in secrecy are not appropriate and violate the nominal title of this proposed rule. If case-by-case exceptions are sought, every step in the process of determining the exception should be made publicly available from the initial request, to the decision-making steps taken by the Administrator, in addition to the final decision.
Response: The final rule applies equally to all dose-response data underlying pivotal science, regardless of when the study or the data was created. The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of the final rule under 40 CFR 30.7.
To ensure that the Administrator's decision to grant an exemption under 40 CFR 30.7 is appropriately transparent, the EPA has included a provision that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. Additionally, the EPA program or region responsible for the significant regulatory action or influential scientific information and public commenters can provide input when the Administrator is considering an exemption.
Well-Established Effects of Particle Pollution on Mortality Risk
Comment: Commenter (6135) specifically opposes any proposed or future action that would disregard the well-established effects of particle air pollution on mortality risk in the United States, either as part of future regulatory decisions or in cost benefit analyses included in mandated regulatory impact assessments. According to the commenter, these adverse risks have been confirmed and verified both through extensive reanalysis of individual studies and more importantly through additional studies that have been completed using numerous independent study populations in the US, Canada, and many other parts of the world. Regardless of whether these effects are reported by the EPA as ranges or as combined estimates of mortality risk, the commenter asserts that the continued inclusion of mortality impacts in regulatory analysis is critical for federal regulatory decisions that impact air quality in Utah. The commenter requests that, if the proposed transparency rule is promulgated, the associations of particle pollution and mortality risk are specifically and preemptively excluded from the judgment of the EPA Administrator in adhering to data availability requirements.
Response: As described in the preamble to the final rule, the transparency provisions of the final rule apply to dose-response data underlying pivotal science, regardless of when the study or data was created. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption, on a case-by-case basis, from the requirements of the final rule under 40 CFR 30.7.
Study Data from Authors Outside the Agency
Comment: Commenter (6449) notes that underlying study data may be difficult to obtain from authors outside the Agency. The commenter requests that EPA allow exceptions for such instances.
Response: As described in the preamble to the final rule, the transparency provisions of the final rule apply to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption, on a case-by-case basis, from the requirements of the final rule under 40 CFR 30.7.
Studies that Meet a Minimal Threshold of Study Quality
Comment: Commenter (6449) requests that, in order to limit the potential for exempting pivotal studies that do not meet the data transparency requirements the studies in question meet a minimal threshold of study quality.
Response: The EPA finds that establishing a minimal threshold of study quality is not practical in this rulemaking. As described in the preamble to the final rule, the EPA will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. Additionally, under 40 CFR 30.5, the EPA is required to rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. 
Studies that Represent "Best Available Science"
Comment: Commenter (6906) recommends that the EPA amend 40 CFR §30.9 regarding exemptions, to include an exemption from publication of dose response and methods, when not available, for studies EPA concludes are the "best available science," using factors EPA has articulated for "best available science." 
Response: As described in the preamble to the final rule, this is a rule of internal Agency procedure and, therefore, does not interpret or apply any statutory provisions of a particular statute or statutes that it administers. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. 
Studies Involving Human Subjects/Health Data
Comment: Commenter (0671) states that if EPA goes ahead with this counterproductive rulemaking, studies involving human subjects should be excepted. In particular, the commenter requests that clinical and population-based studies should be excepted, which they assert produce data that is almost guaranteed to be identifiable and no IRB would allow such data to become public. The commenter states that if this proposed rule disallows EPA from using the results of such studies in formulating regulations then the EPA will have prevented itself from using the best available science, counter to requirements of the CAA and amendments.
Commenter (1272) asserts that any regulation that may affect the health and wellness of people, should and must be allowed to use research that cannot publish its data. The commenter provides that health data is generally not allowed to be freely published and thus must have an exception.
Response: Based on a consideration of these and other similar comments, the EPA proposed, through a 2020 SNPRM, a modified version of the 2018 proposed rulemaking regulatory text at 40 CFR 30.5. In the 2020 SNPRM, the EPA proposed Agency consideration of studies where there is tiered access to data and models that have CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects. For all other studies, data and models should be publicly available if the studies are to be used as pivotal regulatory science or pivotal science. In the 2020 SNPRM, the EPA also proposed an alternative. Under alternative proposed 40 CFR 30.5, when promulgating significant regulatory decisions or finalizing influential scientific information, the Agency would, other things equal, give greater consideration to studies where the underlying data and models are available in a manner sufficient for independent validation either because the information is publicly available or available through tiered access when the data include CBI, proprietary data, or PII and appropriate techniques have been used to reduce the risk of re-identification.
Based on a consideration of public comments received in response to the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. 
As described in the preamble to the final rule, the transparency provisions of the final rule apply to dose-response data underlying pivotal science. Under this rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule under 40 CFR 30.7.
The requirements included in the final rule provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the EPA maintains a strong scientific basis for its decision-making. The incremental progress made possible by the final rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments.
Pesticide Registration and Tolerance Data
Comment: Commenter (6374) supports the exclusion of data submitted under FIFRA or FFDCA to support specific registration or tolerance decisions, which they say is consistent both with that threshold and with the statutory protections for such data. When issuing its final rule, the commenter requests that EPA expressly clarify that the scope of its final rule does not include such pesticide registration and tolerance data.
Response: In response to this comment and other similar comments, the EPA clarified in the 2020 SNPRM the relationship between this rulemaking, the environmental statutes and their implementing regulations by adding language to proposed 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations would control in the event of any conflicts. 
With this final rule, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
Confidential Business Information 
Comment: Commenter (6148) recognizes that in some circumstances full transparency may not be achievable. For example, some data submissions are protected as CBI and may present transparency challenges to the Agency. In such cases, the commenter provides that it may have to be considered whether to exclude such data from the analysis or determine whether a generic format may be adequate to conceal site or company-specific proprietary information. 
Response: Based on a consideration of these and other similar comments, the EPA proposed, through a 2020 SNPRM, a modified version of the 2018 proposed rulemaking regulatory text at 40 CFR 30.5. In the 2020 SNPRM, the EPA proposed Agency consideration of studies where there is tiered access to data and models that have CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects. For all other studies, data and models should be publicly available if the studies are to be used as pivotal regulatory science or pivotal science. In the 2020 SNPRM, the EPA also proposed an alternative. Under alternative proposed 40 CFR 30.5, when promulgating significant regulatory decisions or finalizing influential scientific information, the Agency would, other things equal, give greater consideration to studies where the underlying data and models are available in a manner sufficient for independent validation either because the information is publicly available or available through tiered access when the data include CBI, proprietary data, or PII and appropriate techniques have been used to reduce the risk of re-identification.
Based on a consideration of public comments received in response to the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. Under this rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule under 40 CFR 30.7.16.8.4.9.
Third-Party Laboratory and Research Organization Studies Should Not Be an Excepted/Exempted from Public Disclosure Requirements
Comment: Commenter (6108) asserts that, for non-peer reviewed commissioned studies such as third-party laboratory or other research organizations that may be commissioned, the underlying data should be made available to the public, along with the conclusions, assumptions, methods used, and a discussion of limits of extrapolation. The commenter notes that such third-party scientific studies may draw conclusions, but do not solicit other peers (a "fourth" party) to review the findings. The commenter contends that, because such studies lack the layers of review that accompany peer-reviewed science, they should be held to a higher standard of transparency if relied upon in rulemaking decisions.
Commenter (6375) recommends that, in cases where EPA does not control the data and models for studies, the EPA should not provide an exemption simply because a third-party does not agree to make the data and models publicly available where it does not have a valid reason (e.g. CBI, privacy, etc.) to prevent their release. In those cases, the commenter recommends that the EPA make every attempt to make as much of the data and models available for independent review as possible. According to the commenter, even if the dose-response data and models per se cannot be made publicly available, the commenter suggests that it may be possible for the results and analyses of this information to be made publicly available. According to the commenter, any such available results should be transparent as to the identification and selection of the key data, and the interpretation of those key data, in supporting the Agency's decision.
Response: As described in the preamble to the final rule, the transparency provisions of the final rule apply to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science. Under this rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule under 40 CFR 30.7. 
The EPA would like to emphasize that the final rule does not require the Agency (or other parties) to disclose or host data, but to determine if dose-response data underlying pivotal science are available and to give greater consideration to those studies for which such data are available in a manner sufficient for independent validation.
Study Exception Requests Would be Burdensome
Comment: Commenter (6373) asserts that, because the Rule's default application is to bar research where underlying data is not publicly released, EPA must separately consider exemption requests for each study, at considerable and likely prohibitive time and expense. 
Response: Based on a consideration of public comments received in response to the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. As described in the preamble to the final rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule under 40 CFR 30.7.
The additional procedures required by the final rule only apply to pivotal science, which is a subset of the total number of studies that the EPA would evaluate. Given the costs of the current robust process for identifying and reviewing scientific studies and documentation that are existing Agency practice, as well as that the determination of dose-response data availability is limited to pivotal science underlying significant regulatory actions and influential scientific information, the EPA anticipates that the incremental costs of this rule will be small. For similar reasons, the EPA does not find that the requirements of the final rule will result in any meaningful delay in promulgating regulations.
4.1.2 Definitions of Research Data (§30.2) and Data
4.1.2.1 Definition of "Research Data"
Comment: Commenter (6880) asserts that the proposal disregards the cited definition of "research data" and the rule applies to a much broader range of data. The commenter states that this presents serious consequences for EPA, the people whose mission it is to protect, and the regulated community because the proposal would allow EPA to ignore research findings if it were not possible to make public all the underlying data, including data that is excluded from the definition of "research data." The commenter states that the proposed definition of "research data" excludes types of data that are nonetheless discussed throughout the proposal to strengthen transparency. The commenter states that EPA disregards this definition throughout the actual proposal, as evidenced by the following: 
 Though EPA references this definition in §30.2 ("What definitions apply to this subpart?") of the proposed rule, the term "research data" is absent from the rest of the proposed regulation (except in §30.10). EPA instead refers only to "data" without ever requiring that "data" be limited by the definition (and exclusions) of "research data."
 In describing the data to which the proposal applies, EPA states in §30.3 "The provisions of this section do not apply to physical objects (like laboratory samples), drafts, and preliminary analyses." As these are already excluded from the definition of "research data," it would be redundant for EPA to exclude them if they were using said definition. Also, by mentioning these exclusions while remaining silent on trade secrets and privacy information that are excluded from the definition of "research data," EPA clearly signals that those data are not excluded from the "data" to which EPA refers in the proposal.
 Given that the cited definition of "research data" excludes privacy data, the following concern about privacy in §30.5 of the proposal would be a non-issue if EPA were to actually adhere to that definition: "The agency shall make all reasonable efforts to explore methodologies, technologies, and institutional arrangements for making such data available before it concludes that doing so in a manner consistent with law and protection of privacy, confidentiality, national and homeland security is not possible."
Commenter (6924) states that there are many different kinds of data generated in a research study. The commenter states that the rule treats all these data as if they were the same when that is not the case, indicating the lack of scientific input into this rulemaking. The commenter states that a 2013 memo from Office of Science Technology and Policy recognizes different data types and specifies that study meta-data, not individual-level data, should be made publicly available.
Commenter (6880) notes that, in §30.10, EPA states "EPA shall implement the provisions of this section consistent with the definition of `research data'". The commenter states that, however, such definition applies only to §30.10. The commenter states that, if EPA meant for it to apply to the entire proposed regulation (as stated in the preamble), they would have stated that they "shall implement the provisions of this part consistent with" the cited definition. 
Commenter (6126) asserts that the term "research data" in the proposed rule is ambiguous and needs clarity to assess the rule's potential impacts and for commenters to understand how EPA intends to implement the proposed rule. The commenter states that EPA's existing "Plan to Increase Access to Results of EPA-Funded Scientific Research" provides for exemptions to public release requirements when "the research data cannot be released due to one or more of constraints (sic), such as requirements to protect confidentiality, personal privacy, proprietary interest, or property rights." The commenter asserts that it is not clear if these same exemptions would apply under the proposed rule. 
Commenters (6137, 6449, 6924) express concern that the definition of "research data" is not defined or impermissibly vague/has problems. 
Response: Proposed 40 CFR 30.2 in the 2018 proposed rule included a definition of "research data." In the 2020 SNPRM, the EPA deleted the proposed definition of "research data." While one commenter on the 2020 SNPRM noted that the exclusions in the proposed definition of "research data" of trade secrets and personal and medical information were not incorporated into the proposed definition of "data," commenters did not request that the EPA maintain a definition of "research data." The EPA is not including a definition of "research data" in this final rule because the Agency is finalizing the definition of "data" in section 30.2 as follows:
      Data means the set of recorded factual material commonly accepted in the scientific community as necessary to validate research findings in which obvious errors, such as keystroke or coding errors, have been removed and that is capable of being analyzed by either the original researcher or an independent party.
In addition, the EPA would like to emphasize that this final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science. The rule allows the EPA Administrator to grant an exemption to the transparency requirements if the Administrator determines that greater consideration is warranted when: 
       Technological or other barriers render sharing of the dose-response data infeasible;
       The development of the dose-response data was completed or updated before the effective date of this final rule;
       Making the dose-response data would conflict with laws and regulations governing privacy, confidentiality, CBI, or national security;
       Third-party independent validation of the study's underlying dose-response data through reanalysis has been conducted; or
       The factors in 40 CFR 30.5 used in determining the consideration to afford to the pivotal science indicate full consideration is justified.
4.1.2.2 Definition of "Data"
Comment: Commenters (0671, 5419, 6373, 9224) express the need for EPA to better define/clarify what "data" the rule applies to.
Commenter (6373) states that the impacts of the proposed changes will be difficult to predict unless EPA better defines what "data" the rule applies to.
Commenter (9224) suggests that if the agency wants a rule focused on the raw data then EPA must better define what "data" would be included within this rule. The commenter asks whether it is just data produced by EPA, the methods and protocols, or the actual raw data?
Similarly, commenter (0671) notes that there is a problem with the definition of "data... underlying pivotal regulatory science". The commenter asks whether this means the raw data or the data used to make the final figures and tables in a paper or the data as it exists at some processing step in between? The commenter further asks, if the raw data, how raw? The commenter notes that some raw genomic data is so large it can't be efficiently stored. According to the commenter, there is no one unique thing called "the data" for a study.
Commenter (5419) asserts that the interpretation of the term "data" has critical implications. The commenter notes that, for complete independent validation and analysis, as stated in the proposed rule, the primary data record would be required. According to the commenter, this could include, for example, raw death certificates and raw chemical monitoring data, both of which are subject to transcription error during creation of an analytical file, as well as, in the case of death certificates, challenges related to mortality coding. The commenter states that, notwithstanding privacy concerns, which is the subject of other portions of the proposed rule, the public posting of these large quantities of data would be unwieldly. The commenter requests clarification of this issue.
Response: As described in the 2020 SNPRM, the EPA agrees that there are different stages of data. The EPA presented the different stages as described in the NAS Workshop Report. Since the purpose of 40 CFR 30.5 is to determine the consideration to afford to studies based on, among other factors, the availability of the underlying dose-response data that would support independent validation via reanalysis of the data underlying pivotal science, the appropriate stage of data would not be the processed data (data that have been computed and analyzed to extract relevant information) or the final clean data set that is provided with a publication. At these two stages of data, the analysis has already been conducted, and the results have already been determined. In order to determine if these results are valid, data that had not already been computed and analyzed are needed. Given concerns about potentially skewed analyses, the final definition of "data" maintains the stage of data in which obvious errors have been removed.
In this final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. As noted by commenters, the step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
The definition of "data" in section 30.2 of the final rule is as follows:
      Data means the set of recorded factual material commonly accepted in the scientific community as necessary to validate research findings in which obvious errors, such as keystroke or coding errors, have been removed and that is capable of being analyzed by either the original researcher or an independent party.
4.2 SNPRM Models, Variability/Uncertainty, and Assumptions/Defaults
Overarching Response: As further described in the preamble to the final rule, based on public comments and other considerations, the EPA is concentrating its efforts in the final rule to increase transparency of dose-response data, as the dose-response data are discrete and the dose-response assessment is a well-defined and impactful step in the quantitative assessment of risk. The EPA is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings. Because this rule only applies to dose-response data, the Agency is not finalizing the definition of "model."
4.2.1 Modeling Definitions 
Comment: Commenters (11401, 12406) express support for the clarity achieved by defining "model." Commenters (12406, 12463) also express support for the clarification of the terms "models" and "model assumptions." 
Conversely, several commenters (11894, 12435, 12720) express criticism of modeling definitions as being unclear or too broad. 
Commenter (12435) indicates the supplemental notice of proposed rulemaking (SNPRM or supplemental proposal) definition of "model" is too broad to be useful. Commenter (12727) states that the definition, among other definitions, fails to adequately comprehensively describe the scope of its proposal. As a result the commenter contends that the EPA has denied the public the information it needs to comment in an informed way. 
Commenter (11894) asks whether the definition of "model" applies only to "computational models". The commenter notes that previous guidance documents use the same definition of "model" to focus on the subset of all models termed "computational models" by the National Research Council (NRC). The commenter provides that these are models that use measurable variables, numerical inputs, and mathematical relationships to produce quantitative outputs. The commenter requests clarification as to whether the Agency envisions the same scope of applicability in the promulgation of the proposed rule. 
Commenter (12720) states the definition of "model" is overly general and vague does not address how EPA engages with models and modeling output in conducting its scientific activities. The commenter adds that the definition does not distinguish between physical models, conceptual models, and software modeling packages, nor does it explicitly account for models applicable within the human health context, even though that type of tool is commonly deployed by EPA in its scientific work. According to the commenter, in failing to define models and distinguish between modeling inputs and outputs, EPA's definition is flawed and insufficient. 
Response: EPA appreciates these comments. As discussed in the overarching response to this chapter, the Agency is not finalizing the definition of "model." Because this rule only applies to dose-response data, the rule does not apply to computational models. 
4.2.2 Models (General) and Best Science Practices 
Comment: Commenter (11484) indicates that by expanding the proposed rule to include all data and models rather than just dose-response data and models, it restricts the use of scientific information for arbitrary and non-scientific reasons. The commenter states that the newest version of the proposed rule expands the range of decisions that will be hampered by lack of access to the best available science.  
Commenter (11347) suggests a connection between the impact of this rule and the present coronavirus pandemic. They note that policy and action should be increasingly based upon scientific results coming from powerful and reliable models in a constant state of evolution as the models themselves improve and as the inputs change. The commenter contends that the supplemental proposal would place such work in limbo. In other words, the commenter states that it doesn't take advantage -- in fact, it denies the efficacy of advances in computational science and other ways of creating reliable and complex models to help us solve these countless problems. 
Commenter (11894) indicates that it is imperative to understand that empirical data do not exist for every element of a risk evaluation and that models, many of which have already been peer-reviewed and validated, are employed to fill the data gaps and allow the Agency to reach conclusions about risk. 
Commenter (12688) states that EPA's proposed rule will require the Agency to ignore both foundational and cutting-edge research in core areas of public health and other scientific inquiry. According to the commenter, these could include, for example, models key to understanding earth and atmospheric systems and a limitless range of other research. In short, the commenter contends that the proposal would undermine EPA's ability to use science to protect the health of US residents across all facets of the agency's work. 
Response: See response to prior comment. The EPA disagrees that implementation of the rule would undermine public health protections. The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the public's understanding of the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. As further described in the preamble to the final rule, the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be hampered by lack of access to the best available science or prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations. 
Finally, this rule will not impact the Agency's use or reliance on models. 
Comment: Commenter (11393) expresses concern regarding the rule's potential impacts on health risk assessments. The commenter specifically expresses opposition to language that removes or dilutes the linear no-threshold (LNT) dose response in the evaluation or establishment of radiation health risk assessments, rule-making or establishment of radioactive recommended action levels or benchmarks (influential scientific information) made available to the public.
Response: After considering comments on both the 2018 proposed rule and the 2020 SNPRM, the EPA has determined that this rule should apply to dose-response data rather than dose-response data and models. Therefore, the rule does not include discussion of any particular dose-response model. Given the specificity of 40 CFR 30.6 to dose-response data and models, and in particular dose-response models, the EPA is not finalizing 40 CFR 30.6. The EPA is adapting one provision of 40 CFR 30.6 as a factor in 40 CFR 30.5 in determining the consideration to afford pivotal science for which the dose-response data are not available for independent validation. Specifically, the EPA is finalizing as a factor in 40 CFR 30.5 the consideration that the EPA would give to high quality studies that explore a broad class of parametric dose-response models, non-parametric models that incorporate fewer assumptions, various threshold models, and models that investigate factors that might account for spatial heterogeneity.
Comment: Commenter (11492) indicates that the EPA continues to assign greater value to dose-response studies, which by definition reduce the use/value of epidemiological studies. Section 30.6 of the supplemental proposal has been revised but continues to give "explicit consideration to parametric dose-response or concentration-response models." As the commenter suggests they stated in its 2018 comments, the EPA itself admits that, "there is frequently a lack of dose-response data available for human subjects. When data are available, they often cover only a portion of the possible range of the dose-response relationship."1The commenter states that this gives more weight to toxicological studies as opposed to epidemiological studies, which form the basis of many human health exposure studies. The commenter asserts that the EPA must continue to use epidemiological studies; to do otherwise is to compromise the health of all Americans. Commenter (12403) concurs, indicating that the SNPRM's identification of specific dose-response modeling techniques that would be given "explicit consideration" is inappropriate and ignores the improvements and refinements that are the hallmark of good science. 
Response: Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science is unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that are unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator, as detailed under 40 CFR 30.7. 
The EPA disagrees that the rule implies a bias against the use of epidemiology studies or data. The final rule includes sufficient flexibility in terms of the allowance for restricted access, considerations in 40 CFR 30.5, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data to minimize such potential unintended consequences. The EPA will continue to consider and evaluate all relevant and appropriate science, including epidemiology studies, in its significant regulatory actions and influential scientific information. 
In addition, after considering comments on both the 2018 proposed rule and the 2020 SNPRM, the EPA has determined that this rule should apply to dose-response data rather than dose-response data and models. Therefore, the rule does not include discussion of any particular dose-response model. 
Comment: In support of existing model availability, commenter (11894) provides the specific example of the trichloroethylene (TCE) case study. The commenter notes that the TCE assessment relied on approximately 15 models. The commenter notes that this is likely to be the norm with the same ones being used repeatedly. The commenter adds that most of the models EPA employs in the TSCA program have been developed by the Agency and are publicly-available. The commenter provides that access can be gained through a number of EPA webpages and/or searches on the internet, and others are described in the peer-reviewed literature, access to which is controlled by the authors. 
Response: After considering comments on both the 2018 proposed rule and the 2020 SNPRM, the EPA has determined that this rule should apply to dose-response data rather than dose-response data and models. Therefore, the rule does not include discussion of any particular dose-response model. 
Comment: Commenter (12465) states that the EPA relies upon many types of data and models to support its regulatory decisions and it is unclear that the current proposal has adequately considered the numerous types. The commenter expresses concern that the treatment of existing data or models under the proposals disincentivize the generation of new data under the new requirements while reducing the importance of consideration of existing data, regardless of the scientific validity or merit of that data.  
The commenter (12465) provides the example of the Westat Survey. The commenter notes that the report entitled "Household Solvent Products: A National Usage Survey" (or the Westat Survey) is a survey of consumer usage for a wide variety of products to determine various attributes and inform exposure assessments. According to the commenter, while the report was completed in 1987, it remains in use today informing the EPA's Consumer Exposure Model and the Westat Survey informed several of the first 10 chemicals currently being evaluated under reformed TSCA. The commenter states that it is unclear in the proposal how the Westat Survey would be treated, let alone the multitude of exposure models that rely upon it. 
Response: As further described in the preamble to the final rule and the overarching response to this chapter, the EPA is not finalizing the definition of "model." The rule only applies to dose-response data and will not impact the Agency's use or reliance on models, including exposure models. Since the Westat Survey is a consumer behavior survey for assessing exposure, it would also not be applicable to this rule. 
The EPA disagrees that the rule disincentivize the generation of data. The final rule includes sufficient flexibility in terms of the allowance for restricted access, considerations in 40 CFR 30.5, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data to minimize such potential unintended consequences. The EPA will continue to consider and evaluate all relevant and appropriate science in its significant regulatory actions and influential scientific information.
4.2.3 Public Availability of Models/Data 
Comment: Several commenters (11392, 11393, 11407, 11576, 11894, 12430, 12472, 12474, 12720, 12727) express opposition to requiring models and data be publicly available. Commenter (11393) opposes these expansions; and contends that the SNPRM takes a bad proposal and makes it worse.
Commenter (12727) indicates that EPA has not demonstrated -- and cannot demonstrate -- that a study cannot be scientifically valid unless the underlying data and models are publicly disclosed. The commenter states that mechanisms are already in place to validate studies for which disclosure of underlying data and models is either illegal or impracticable.
Commenter (12472) indicates that expanding the scope to include all models would place a particular burden on air quality models and potentially have an adverse impact on public health. According to the commenter, if air quality models are included, the data access requirements described in the SNPRM may pose difficulties in some instances, particularly for emissions inventories used in modeling.
Commenter (11576) highlights the impact on environmental-fate studies and models if these types of studies are included in its new, broader scope. The commenter provides that these data and models are critical for the EPA's assessments of the environmental risks of pesticides, for example, and the Agency has relied on well-established studies that were conducted before mechanisms existed for sharing data publicly. The commenter adds that the EPA also mentions bioaccumulation data as an example of the types of studies included in the new expansion, which they contend could unjustifiably undermine established science on the accumulation of mercury, lead, and other metals in aquatic systems and wildlife, as well as models used by the EPA for decades to describe accumulation.2 The commenter contends that the requirement for publicly available data underlying all model parameters would be particularly problematic and potentially impossible for models developed iteratively over decades, in which model parameters and structure evolve with growing science without the need for publicly available documentation of all the underlying parameters' source data. 
Commenter (11894) points out that databases of publicly available models already exist, such as the Registry of EPA Applications, Models and Data Warehouses (READ) (available at https://ofmpub.epa.gov/sor_internet/registry/systmreg/searchandretrieve/basic/search.do), which catalogs publicly-available models that the agency uses. The commenter provides that there are currently 2,197 entries in this library of tools, a subset of which are models. And, specific to EPA's Existing and New Chemicals programs, Office of Chemical Safety and Pollution Prevention (OCSPP) hosts a web page (Predictive Models and Tools for Assessing Chemicals under the TSCA, available at https://www.epa.gov/tsca-screening-tools). The commenter adds that using these two sources and others available by searching the internet, one can access the publicly-available, Agency-developed models that it employs. The commenter notes that if the Agency has to resort to the use of a proprietary model to conduct some aspect of its work, it is not within its purview to release the code to it. Under that circumstance, the commenter notes that one must arrange that with the owner of the model. This, according to the commenter, however, has become an increasingly rare circumstance. 
Commenter (11894) notes that securing the release of proprietary modeling information is difficult, and places an undue burden on EPA. The commenter adds that if the Agency has to resort to the use of a proprietary model to conduct some aspect of its work, it is not within its purview to release the code to it. 
Commenter (12430) states that models are critical to help humans assess and analyze large quantities of data, but they are nearly always proprietary. The commenter provides that most scientists and agencies purchase access use these models for data. The commenter contends that restricting models to those that are publicly available would severely limit the science that U.S. EPA may consider or reference. 
Commenter (12720) indicates that for many studies, proprietary software and/or proprietary code is necessary in order to effectively work with underlying study data. However, the commenter states that these models are generally not freely available to the public, and many require advanced technical training and considerable technical resources to operate. The commenter notes that the EPA has not discussed this issue in the proposal, even though it presents a major obstacle to the goals of the proposal. According to the commenter, for already burdened researchers, it is not feasible to expect them to provide underlying code (which may be subject to other release requirements and advance permissions) to the Agency and places an undue burden on the EPA.
Conversely, commenter (12712) expressed support for making models/data publicly available wherever possible. The commenter suggests that the default rule for all work funded by the EPA must be that relevant code and data are available online or on request. The commenter adds that the EPA must be clear that making data available means that the actual final dataset used in statistical analysis is made available.
Response: As further described in the preamble to the final rule, based on public comments and other considerations, the EPA is concentrating its efforts in the final rule to increase transparency of dose-response data, as the dose-response data are discrete and the dose-response assessment is a well-defined and impactful step in the quantitative assessment of risk. The rule does not apply to other types of data and models, including bioaccumulation data and environmental fate and transport models. The EPA is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings. 
EPA does not take the position that studies for which the underlying data are not publicly available are per se invalid. Rather, EPA acknowledges that where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty.
EPA has implemented the public access requirements for EPA researchers, grantees and contractors as required by the February 22, 2013, OSTP memorandum "Increasing Access to the Results of Federally Funded Scientific Research" and the 2016 EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research." EPA researchers, grantees, and contractors are required to make peer-reviewed journal articles and the underlying data and models resulting from EPA-funded research accessible to the public. 
4.2.4 Weighting of Models 
Comment: Commenter (9362) supports the wording of section 30.6 "When available, EPA shall give explicit consideration to high quality studies, including but not limited to those that explore: A broad class of parametric dose-response or concentration-response models; a robust set of potential confounding variables; nonparametric models that incorporate fewer assumptions; various threshold models across the dose or exposure range; and models that investigate factors that might account for spatial heterogeneity." However, the commenter states that "nonparametric models that incorporate fewer assumptions" should be explicitly stated in this sentence. The commenter notes that parametric models may be more appropriate in many cases.
Commenter (11395) suggests that the "explicit consideration" language of section 30.6 is inappropriate because the scientific information, techniques and models used in risk assessments are developing rapidly. Therefore, the commenter suggests that risk assessment methodology should be addressed in guidelines, which can be adjusted as more information becomes available, rather than with fixed regulations. Instead of establishing inflexible regulatory requirements, the commenter adds that the Agency should focus on continuing to improve its existing guidelines, in concert with other federal agencies. The commenter adds that the specific identification of modeling considerations in a regulation is inappropriate because multiple analyses using a variety of models do not necessarily make a study more credible, there is no scientific justification for this stipulation. The commenter adds that model choices consider such factors as biological plausibility and the degree of protection to human health. According to the commenter, an evaluation of a study that uses modeling should assess model assumptions, the fit of the model to the dataset, the goal of the analysis, and other issues. The commenter contends that the use of a greater number of models does not in itself improve results or make a study more plausible. Therefore, the commenter asserts that giving priority to studies based on the number or range of models used is scientifically inappropriate and may create an incentive for using inappropriate models without scientific justification, which may generate inaccurate outputs. 
Commenter (10740) states that, under the supplemental proposal, EPA would be required not only to prepare its own models with multiple assumptions, but also to "give explicit consideration" to a long list of models that could be prepared by outside stakeholders. The commenter provides that the proposal presents a list of these models that EPA must explicitly consider, such as a "broad class of parametric dose-response or concentration-response models; a robust set of potential confounding variables; nonparametric models that incorporate fewer assumptions; various threshold models across the dose or exposure range; and models that investigate factors that might account for spatial heterogeneity." 
The commenter (10740) suggests that perhaps it is the intent of this proposed rule to preclude scientific evidence based on data and models informed by raw data, including medical records, that is required to be held confidential. The commenter asserts that this is not transparency and nor is it science. 
Response: As further described in the preamble to the final rule, based on public comments and other considerations, the EPA is concentrating its efforts in the final rule to increase transparency of dose-response data, as the dose-response data are discrete and the dose-response assessment is a well-defined and impactful step in the quantitative assessment of risk. The rule does not apply to dose-response models or other types of data and models. Therefore, the rule does not include discussion of any particular dose-response model. The EPA is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings. 
Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations. 
4.2.5 Scope of Models Included 
Comment: Commenter (11894) contends that the EPA has not adequately explained why the scope should be expanded to include far more models. The commenter asserts that when a federal Agency proposes to make such far ranging changes, it is obligated to explain why and to support that with data and information. 
Response: As further described in the preamble to the final rule, based on public comments and other considerations, the EPA is concentrating its efforts in the final rule to increase transparency of dose-response data, as the dose-response data are discrete and the dose-response assessment is a well-defined and impactful step in the quantitative assessment of risk. The rule does not apply to dose-response models or other types of data and models. 
Comment: Commenter (12472) expresses that the expansion could potentially impact on air quality models, and requests that EPA make explicit whether these models are included under the rule. The commenter specifically requests that the EPA clarify whether air quality and associated models such as Comprehensive Air Quality Model with Extensions (CAMx) are included as models subject to the rule. 
Response: As further described in the preamble to the final rule, based on public comments and other considerations, the EPA is concentrating its efforts in the final rule to increase transparency of dose-response data, as the dose-response data are discrete and the dose-response assessment is a well-defined and impactful step in the quantitative assessment of risk. The rule does not apply to dose-response models or other types of data and models, including air quality models. 
Comment: Commenter (12699) quotes § 30.3 "The provisions of this part apply to data and models, underlying pivotal science supporting influential scientific information and/or underlying pivotal regulatory science used to justify significant regulatory decisions regardless of the source of funding or identity of the party conducting the science." The commenter notes that the comma after "data and models" appears to be communicating that the provisions apply to data and models, underlying pivotal science, and/or underlying pivotal regulatory science. Unfortunately, the commenter contends that this language could create confusion by giving the impression that the provisions only apply to data and models. According to the commenter, since this is unlikely what the EPA intends, they recommend that EPA make § 30.3 clearer. 
Commenter (12699) adds that, if in fact the EPA does only intend to cover data and models, then this would make little sense and beg the question why it would not cover pivotal science and pivotal regulatory science. The commenter notes that both are critical in practice and for purposes of the rule. The EPA would be creating a transparency rule to help the public evaluate the science, and then arbitrarily only allow the public to evaluate a subset of what informs influential scientific information and significant regulatory actions. 
Response: As further described in the preamble to the final rule and the overarching response to this chapter, the Agency is not finalizing the term "model." Therefore, the rule does not apply to dose-response models or other types of data and models. The text from 40 CFR 30.3 quoted by the commenter has also been clarified to now read: "The provisions of this part apply to science that serves as the basis for informing a significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science." 
The EPA acknowledges confusion over the definition of "science that serves as the basis for informing a significant regulatory action" and is altering the definition of the term in 40 CFR 30.2 to mean "studies, analyses, models, and assessments of a body of evidence that provide the basis for EPA significant regulatory actions." Examples of models include those used in regulatory impact analyses. Examples of assessments of a body of evidence include risk assessments, hazard identifications, IRIS assessments, and criteria documents. The EPA also clarifies in the final rule that the scope of science that serves as the basis for informing a significant regulatory action is equivalent to the science included in the public docket as part of a rulemaking, but not all of that body of science would typically be considered "pivotal science." 
4.2.6 Assumptions/Defaults-Variability/Uncertainty 
Comment: Commenter (11403) expresses that in peer-reviewed scientific publications, scientists state their assumptions as well as all model assumptions in the method sections (or else cite a publication that discusses those assumptions), so the peer-review process already provides this level of transparency. In addition, the commenter states that parametric models should not be favored over non-parametric ones. The commenter asserts that favoring one type of modeling over another does not support good science.
Response: As detailed in OMB's Final Information Quality Bulletin for Peer Review, "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review does not typically include reanalysis of the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information) and, thus, peer review is not considered a replacement for the data availability requirements of this rule. Where underlying dose-response data in pivotal science are available, subject matter experts could do more than evaluate the validity of an assumption; experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. The EPA is confident that this approach will result in significant regulatory actions and influential scientific information that are based on the highest quality, most relevant studies and ensure adequate human health and environmental protections, while maximizing transparency and maintaining consistency with the statutes the EPA administers.
Finally, the final rule does not apply to dose-response models and does not prioritize types of dose-response models. 
Comment: Commenter (12430) states that requiring justification for all default assumptions ignores the establishment of defaults through prior public processes and external public peer reviews. For example, the commenter notes that requiring justification for the use of default assumptions for every toxicity health factor derivation ignores the detailed development process of those default assumptions. The commenter states that those default assumptions were developed in several U.S. EPA documents (including the 2005 Guidelines for Carcinogen Risk Assessment3), which received public comment and extensive external peer-review. Requiring EPA to re-justify those default assumptions every time they are used, as the SNPRM seems to require, would serve no purpose other than to add unnecessary delay to risk assessments, regulatory action, and publication of influential scientific information. 
Conversely, commenter (9362) expresses support for language that states that the "EPA shall describe and document any assumptions and methods used and shall describe variability and uncertainty". The commenter further supports language that states "EPA shall clearly explain the scientific basis for critical assumptions used in the analysis that drove the analytical results and subsequent decisions and shall present analyses showing the sensitivity of the modeled results to alternative assumptions." The commenter notes that it is good if all source information is disclosed. The commenter further supports language that states "[b]ecause transparency in regulatory science includes addressing issues associated with assumptions used in models, EPA shall ask peer reviewers to articulate the strengths and weaknesses of EPA's justification for the assumptions applied and the implications of those assumptions for the results," noting that it is great as long as all pertinent scientific information is included. 
Response: The EPA clarified in the SNPRM that the use of the terms "model assumptions," "assumptions" and "models" in the proposed regulatory text at 40 CFR 30.6 apply to the critical assumptions that drive the model's analytic results, not to each assumption used in the model. In the final rule, the EPA maintained the use of "critical assumptions" rather than "all assumptions" in what is now 40 CFR 30.5(e). The EPA also made edits to the text that the commenter quoted regarding peer review in what is now 40 CFR 30.6 to conform with the applicability of the rule to dose-response data. The EPA agrees with the commenter that greater data availability will allow peer reviewers to better evaluate and articulate the strengths and weaknesses of EPA's justification for the assumptions applied and the implications of those assumptions for the results. 
Comment: Commenter (11361) provides that the EPA describes its intent for the proposed public data requirements to "allow stakeholders to reanalyze the data and models and explore the sensitivity of the conclusions to alternative assumptions." The commenter contends that this begs the questions of who the assumed "stakeholders" are and what the purposes are behind developing alternative assumptions. 
The commenter (11361) adds that the EPA goes on to say the rule would "allow assessment of the robustness of the original analysis and conclusions by, for instance, showing the variability that can occur when a previously omitted variable is added to the statistical model, different functional form assumptions are made (e.g., a linear marginal effect of treatment), or different assumptions are made when estimating standard errors and drawing statistical inferences (e.g., allowing for spatial correlation in error terms)." According to the commenter, this is highly problematic, however, and is an overt way to attack credible science with unfounded critiques. The commenter notes that all statisticians know that including too many variables is at least as flawed, and often more so, than including too few variables, and one can artificially increase or decrease model fit without any theoretical grounding. Likewise, the commenter adds that decisions about functional forms are typically grounded in theory, and even if they are not, clear relationships can be obfuscated by limiting or overwriting functional forms. According to the commenter, applying a different functional form (e.g. linear rather than log-transformed) would always change the results and could obscure meaningful relationships. The commenter contends that the EPA is not providing an opportunity to improve science through this proposed rule. The commenter states that, for research that has gone through peer review (as the EPA, thankfully, clarifies it expects for research that is included in significant agency decisions ), altering model inputs, structure, and assumptions are easy4mechanisms for unsophisticated and inappropriate muddling of credible and substantiated models.
Commenter (11495) notes that the proposed rule suggests that data reanalysis will be completed by "stakeholders", who will have the opportunity to "reanalyze the data and models and explore the sensitivity of the conclusions to alternative assumptions while accessing only the data and aspects of the models that they need". The commenter states that there is a lack of clarity over who the "stakeholders" involved in this analysis may be, and that the suggestion that they may cherry pick data to force into models that support "alternative assumptions" goes directly against the scientific process and certainly looks at least superficially like exactly the sort of bias the EPA is hoping to avoid with their rule. According to the commenter, while good faith use of alternative methodologies to test sensitivity of various inputs is good scientific practice, expecting the same research data to yield the same results when put through alternative methodologies, potentially employed by undefined stakeholders who have the potential to intentionally introduce bias, is not a credible deployment of the scientific process. 
Response: The EPA agrees that the word "stakeholders" in the SNPRM preamble was not defined. In this context, stakeholders meant any subject matter expert who has not contributed to the development of the study. The EPA fixed this inconsistency in the final rule and only uses the term "subject matter expert." Researchers generally use a number of assumptions in their development of a scientific assessment. The impact of the use of the assumptions is not always clear, particularly when the underlying data are not available. The intent of the language was to explain a benefit of greater transparency in the underlying data -- that subject matter experts could validate the study conclusions using the same assumptions, but also reanalyze the data using alternative assumptions to better understand the sensitivity of the results to the assumptions. 
The EPA disagrees that the rule does not offer an opportunity to improve science. The benefits of greater data transparency and the significance of reanalyzing and validating study results are well-documented in scientific literature. McNutt (2014) noted, "reproducibility, rigor, transparency, and independent verification are cornerstones of the scientific method." The National Academies of Sciences, Engineering, and Medicine (NAS) workshop on Reproducibility and Replicability in Science also noted that "certainly, reproducibility and replicability play an important role in achieving rigor and transparency." Munafò et al. (2017) state, "the credibility of scientific claims is rooted in the evidence supporting them, which includes the methodology applied, the data acquired, and the process of methodology implementation, data analysis and outcome interpretation. Claims become credible by the community reviewing, critiquing, extending and reproducing the supporting evidence. However, without transparency, claims only achieve credibility based on trust in the confidence or authority of the originator. Transparency is superior to trust." The 2019 NAS workshop on Reproducibility and Replicability in Science also concluded, "the scientific enterprise depends on the ability of the scientific community to scrutinize scientific claims and to gain confidence over time in results and inferences that have stood up to repeated testing." The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety.
Comment: Commenter (12472) provides that non-public data or tools are often used in EPA modeling, replication and verification of data and models, therefore, the commenter suggests that the use of these data and tools should be addressed in the SNPRM. For example, the commenter notes that the Integrated Planning Model (IPM) is used to generate future year Electricity Generating Units (EGU) emissions. The commenter states that the IPM is a proprietary model which is used extensively by EPA and cannot be verified or replicated by states. The commenter requests that the EPA provide additional details on the suitability of using EPA's modeling results in specific regulatory settings based on these variability, uncertainty, and verification issues. 
Response: As further described in the preamble to the final rule, based on public comments and other considerations, the EPA is concentrating its efforts in the final rule to increase transparency of dose-response data, as the dose-response data are discrete and the dose-response assessment is a well-defined and impactful step in the quantitative assessment of risk. Because this rule only applies to dose-response data, the Agency is not finalizing the definition of "model." 
4.2.7 Impacts on Older/Existing Models 
Comment: The supplemental proposal maintains its original position in section 30.5 that the rule would apply to studies, data, and models regardless of when they were published, but requests comments on whether this should apply only to data and models that are generated after the effective date of the rule making. 
Several commenters (11386, 11407, 11490, 11491, 12474, 12546, 12702, 12727) express opposition to applying the rule to existing/older models. Commenter (11576) provides that the EPA's own SAB has warned that "[f]or studies published many years ago, it may not be feasible to deliver public access to data and analytic methods[,]" and "[t]here are also sensitive situations where public access may infringe on legitimate confidentiality and privacy interests, and where exceptions from complete public access may be appropriate." 
Commenter (11894) states that the Agency is again asking for commenters to weigh options for addressing older studies without providing any analysis of the relative costs, benefits and disruption not only to the Agency and the processes it manages, but for those scientists who produced the studies in question.
Commenter (12710) states that, in 2018, they stated their concern that most studies prior to the digital age would not meet the proposed requirements for data transparency; this would lead EPA to dismiss most of the science conducted in the 20th century, including seminal studies on the health and environmental effects of pesticides. The commenter provides that the preamble of the SNPRM clearly confirmed this concern by stating "[p]roposed 40 CFR §30.5 would maintain the temporal approach to data and models taken in the regulatory text of 40 CFR §30.5 of the 2018 proposed rulemaking, and thus would apply to data and models evaluated at the time a significant regulatory action or influential scientific information is developed, regardless of when the data and models were generated." The commenter asserts that if EPA were to go forward with the proposal, we strongly recommend that the new transparency guidelines apply only to studies conducted after the proposal goes into effect. 
Commenter (11490) opposes applying the rules to older models and studies. The commenter states that high quality studies published to date must not be excluded on the grounds that authors did not follow data sharing practices which were not required nor expected at the time of publication.
Commenter (11386) states that if this rule were to be put into practice, it is clear that this should only apply to data and models that are generated after the effective date. According to the commenter, even if the transparency of data and ability to be reanalyzed did indicate quality of science, then this information was available to EPA when considering past rules. The commenter adds that nothing about the science used to make previous rules will be changed as a result of this new rule. Thus, the commenter asserts that to suddenly treat previous studies in a way different from how they were previously treated by the EPA would be arbitrary and capricious, as nothing about the studies themselves had changed. The commenter notes that if new information came to light, such as new studies or a reanalysis of previous studies, then it would be quite reasonable for this to potentially change how older studies are considered. However, the commenter concludes that to apply different consideration to previous studies due to factors that have nothing to do with the science itself would be completely arbitrary.
Commenter (12727) states that there are compelling reasons why it may be impossible or infeasible to provide access to data and models underlying older studies. The revised and alternative approaches therefore fail to address one of the most substantial shortcomings of the original proposal -- one that EPA has explicitly acknowledged in its supplemental proposal.
Commenter (11921) asserts that older studies that did not and are unable to comply after a prolonged interval still have substantial scientific merit and value for informing regulations. Some such studies, e.g. the NIOSH/NCI Diesel Exposed Miners Study (DEMS) have made their data available through federal Research Data Centers. However, the data for other critical studies, e.g. the North American Rubber Workers Study, which has played and continues to play a key role in assessment of risk from 1,3-butadiene and styrene, was designed and implemented at a much earlier time and would not be readily available. Although the proposed supplemental proposal does suggest that the age of a study and the difficulty of gaining access to the underlying data could be a basis for the Administrator finding it impractical to make such data and methods available, and therefore exempting the study from this rule, such after-the-fact and potentially arbitrary determination would obstruct the continued use of a larger body of evidence  -  especially occupational studies that are critical to consider the effects of the toxicity of a number of specific chemicals. 
Conversely, several commenters (11499, 11922, 12454) support applying the rule to older/existing models generated/developed before the effective date of the rule. Commenter (11499) supports including all existing models and studies in the scope of the proposed rule and giving the Administration discretion over the case-by-case exemptions.
Commenter (12454) states that given the rule's goal of enhancing transparency in science, the support existing data and models being covered if they underlie "pivotal regulatory science" in a rulemaking or "pivotal science" in the development of influential scientific information. 
Commenter (11922) states that the rule should apply to all models and studies when evaluated for pivotal or influential utilization in decision-making as opposed to only those developed after this rulemaking.
Commenter (11491) asserts that a case-by-case exemption granted by the Administrator for the consideration of older data and models to be unacceptable. If the proposed rule is finalized, the commenter states that it should prevent all retroactive application by placing older data and models outside the scope of the rule. The commenter notes that case-by-case exemptions are not an adequate substitute for removing this older category of science from the rule's scope; not only does the use of exemptions for this purpose increase uncertainty and internal process, but it introduces the possibility of bias by this or a future EPA. 
Response: The EPA considered these comments in the development of the final rule. The EPA is maintaining that the rule provisions apply equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that is unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator as detailed under 40 CFR 30.7. One of the criteria the Administrator may use in his/her determination to grant a waiver is that the development of the dose-response data was completed or updated before the effective date of the rule. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations. 
The EPA also maintains that the Administrator is the appropriate decision maker in the context of granting exemptions under this rule. To ensure that the Administrator's decision is appropriately transparent, in the final rule the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. This documentation would typically be provided as part of the proposed rulemaking given this will be part of the decision of what is the pivotal science for the rule. Regardless, the final rulemaking will also provide clear documentation. 
4.2.8 Linear vs Nonlinear Modeling 
Comment: Commenter (11479) objects to EPA's recommendation to use non-linear modeling over linear modeling. The commenter states that force-fitting non-linear models, without scientific justification, could bias the data to such a degree that low doses of a pollutant may appear less harmful  -  or even beneficial  -  to the health of person. The commenter adds that if scientifically unjustified weight is given to non-linear models to assess relationships between pollutants and health in EPA policy decisions, these decisions are likely to disproportionately affect the sensitive populations that EPA is charged with protecting. 
Commenter (11499) states that the EPA should not be so prescriptive in the use of linear vs non-linear modeling. The commenter recommends that EPA tweak the language in proposed 40 C.F.R. Section 30.6 to say: "EPA shall evaluate the appropriateness of using default assumptions on a case-by-case basis, including assumptions of a linear, no-threshold dose response and other default assumptions as appropriate." The commenter notes that the public needs to understand clearly why EPA selects particular models to evaluate dose-response effects of exposure to different substances and what uncertainty factors are included for the model chosen in regulatory decision-making for those substances. The commenter adds that public access to information on model selection and uncertainty factors is as important as transparency of underlying data. According to the commenter, if the final rule fails to include this discussion, the Agency has lost an important opportunity to guide regulators and the public on evaluating these underlying default models, particularly when they involve metals risk assessments. 
Commenter (12454) supports the use of a non-linear model relationship because many metals are essential nutrients that result in non-linear dose-response toxicity relationships, but EPA often fails to consider non-linear dose-response approaches in assessing metals and metal compounds. The commenter posits that the Agency can include a specific reference to "assumptions of a linear, no-threshold dose response" without implying that the Agency should not evaluate other default assumptions by tweaking the language in proposed 40 C.F.R. section 30.6 as recommended by the North American Metal's Council (NAMC) in their comments (references commenter 11499).
Response: In the 2020 SNPRM, in response to public comments, the EPA proposed a variation of the regulatory text which did not include the phrase "including assumptions of a linear, no threshold dose-response," because this could imply that the regulation is specific to those particular assumptions. After considering comments on both the 2018 proposed rule and the 2020 SNPRM, the EPA has determined that this rule should apply to dose-response data rather than dose-response data and models. Given other changes to the regulatory text, the text quoted or referenced by the commenters is now in 40 CFR 30.5(e) and reads: "The EPA shall also describe critical assumptions and methods used in its dose-response assessment and shall characterize the variability and uncertainty of the assessment. The EPA shall evaluate the appropriateness of using default assumptions on a case-by-case basis. The EPA shall clearly explain the scientific basis for critical assumptions used in the dose-response assessment that the EPA relied on for the significant regulatory action or influential scientific information." 
4.2.9 Impacts on Specific Models 
Comment: Commenters (11894, 12727) question the impacts of the supplemental proposal on Greenhouse Gases, Regulated Emissions, and Energy Use in Transportation (GREET)[9], Motor Vehicle Emission Simulator (MOVES)[10], Benefits Mapping and Analysis Program (BENMAP)[11] CE, Integrated Planning Model (IPM)[12], Model for the Assessment of Greenhouse Gas Induced Climate Change (MAGICC)[13] and Global Change Assessment Model (GCAM)[14] models, as well as various Community Multiscale Air Quality Modelling System (CMAQ)[15] models. The commenters suggest that it is incumbent on EPA to evaluate potential impacts of the proposal on all of its standard modeling tools and then provide the public with notice and opportunity for comment on these findings. 
Commenter (12720) provides comments on the impacts of the proposed rule on the PBPK model. The commenter states that the EPA convened a  FIFRA Scientific Advisory Panel in October 2017 to assess its use of a Physiologically Based Pharmacokinetic Model (PBPK) used in its risk assessment for the pyrethroid pesticides. The commenter notes that the PBPK model was sponsored and submitted to EPA by the Council for the Advancement of Pyrethroid Human Risk Assessment, L.L.C. (CAPHRA). CAPHRA identifies its participating parties as chemical and agrochemical manufacturers. CAPHRA describes its intentions as follows: "The general area of CAPHRA's planned activity is to generate and submit to the [U.S. EPA] studies necessary to address EPA's concerns for the potential for age-dependent sensitivity to Pyrethroids." 
Despite the central role of the pyrethroid PBPK model in EPA's regulatory approval for pyrethroid pesticides, the commenter states that it appears that scientific peer reviewers on the FIFRA Scientific Advisory Panel (SAP) were unable to obtain the raw data necessary to provide a robust peer review of the model. According to the commenter, SAP Panelist Dr. Dale Hattis requested these data from EPA on September 6 and September 12 without receiving them, including "key data" for "evaluating the uncertainty in the modeling" and "data needed for assessment of the calibration of the PBPK models." 
Commenter (12720) provides comments on the impacts on the BenMAP, CO-Benefits Risk Assessment (COBRA) Health Impacts Screening and Mapping Tool,16 and other PM25 models. The commenter notes that a key component of those models is the application of exposure-response epidemiology to estimate the health implications of specific energy, climate, and air quality scenarios. The commenter states that the proposed rule does not explain how significant and burdensome data transparency requirements imposed on data and models used by the Agency would affect the existing models that EPA currently hosts and distributes to the international research community. 
The commenter (12720) adds that it is not clear that the dose-response studies incorporated into these models would remain and, if not, the detrimental impact that change would have on efforts to describe health implications of future proposed rules. Importantly, the commenter notes that the EPA itself uses the BenMAP model to conduct health impact assessments as part of proposed rules; and the supplemental proposal does not include any discussion of the implications of data transparency requirements on the future implementation and use of EPA-endorsed software packages. 
Response: As further described in the preamble to the final rule, based on public comments and other considerations, the EPA is concentrating its efforts in the final rule to increase transparency of dose-response data, as the dose-response data are discrete and the dose-response assessment is a well-defined and impactful step in the quantitative assessment of risk. The rule does not apply to dose-response models or other types of data and models, including the models noted by the commenters. 
Also as described in the preamble to the final rule, based on the comments on the 2018 proposed rule and the 2020 SNPRM, including that the number of studies that would be subject to the rule might hinder rulemaking, the EPA determined that the Agency should pursue an incremental approach to maximizing transparency in the science that it relies upon by focusing the final rule requirements on dose-response data and, in particular, only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). The EPA considered commenters' assertions that the scope of the 2018 proposed rule would be so broad as to make implementation infeasible. The 2018 proposed definition of "dose-response data and models" would apply to dose-response data [and models] "used to characterize the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and the magnitude of a predicted health or environmental impact." This relationship of the dose-response data to the magnitude of a predicted health or environmental impact would require the consideration of an array of studies beyond those that characterize dose-response relationships, including, for example, studies that inform the dose-response modeling (e.g., benchmark response selection); studies that identify data for toxicokinetic adjustments that inform calculation of a human-equivalent point of departure (POD); and studies that inform the selection of uncertainty factors. The number of studies that are used to establish the relationship between dose-response data and models and the magnitude of a predicted health or environmental impact can potentially be very large. This may make implementing the rule as proposed infeasible for at least some significant regulatory actions and influential scientific information. While transparency in EPA decision making is the purpose of this action, transparency cannot be effectively accomplished if the means to that transparency are not feasible. Rather than having this final rule apply to all the studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact, the EPA is balancing transparency and feasibility by focusing on those studies that describe the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect. Specifically, the scope of dose-response data in this final rule is those studies consisting of the data integral to characterizing dose-response relationships. Therefore, the rule does not apply to PBPK models.
4.3 SNPRM Data, Variability/Uncertainty, and Assumptions/Defaults
4.3.1. Clarity of Proposed Definitions
Comment: Commenter (11391) quotes the SAB's review of the proposal that in the new requirement that all "data be made available is vague and, as a result, can be interpreted in different ways" and carries that sentiment to the supplemental proposal definition of data.
Response: This final rule does not require that all data be made available. Instead, the EPA may give greater consideration to pivotal science where the underlying dose-response data is available for independent validation.
Comment: Commenter (11894) asks whether the definition of "model" applies only to "computational models". The commenter notes that previous guidance documents use the same definition of "model" to focus on the subset of all models termed "computational models" by the National Research Council (NRC). The commenter provides that these are models that use measurable variables, numerical inputs, and mathematical relationships to produce quantitative outputs. The commenter requests clarification as to whether the Agency envisions the same scope of applicability in the promulgation of the proposed rule. 
Comment: Commenter (11919) also encourages EPA to better define what data must be publicly available. The commenter asserts that without a clear definition, the requirements of the rule will be open to a variety of interpretations leading to confusion and potential misuse.
Commenter (12435) indicates the supplemental notice of proposed rulemaking (SNPRM or supplemental proposal) definition of "model" is too broad to be useful. Commenter (12727) states that the definition, among other definitions, fails to adequately comprehensively describe the scope of its proposal. As a result the commenter contends that the EPA has denied the public the information it needs to comment in an informed way.
Commenter (12720) The commenter states the definition of "model" is overly general and vague does not address how EPA engages with models and modeling output in conducting its scientific activities. The commenter adds that the definition does not distinguish between physical models, conceptual models, and software modeling packages, nor does it explicitly account for models applicable within the human health context, even though that type of tool is commonly deployed by EPA in its scientific work. According to the commenter, in failing to define models and distinguish between modeling inputs and outputs, EPA's definition is flawed and insufficient.
Conversely, commenters (9587, 11922, 11390, 11401, 12406, 12463) assert that the definitions of data and models provided in the supplemental proposal are clear. Commenter (11390) notes that it is hard to imagine a scientific study, dataset, or model that would not be subject to the supplemental proposal.
Response: The EPA has removed requirements pertaining to model because this regulation applies only to dose-response data, and therefore has removed the definition for model from 40 CFR 30.2. 
In this final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. As noted by some commenters, the step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
4.3.1.1 Stage of Data being Requested
Comment: Commenters (10740, 11495, 12435, 12471, 12546, 12712, 12720) note that there is a lack of clarity in whether the data being requested is the raw data collected throughout the experiment or only the finalized data set used to generate the study results. Specific examples of comments include:
Commenter (10740) states that, while the rule proposes an exclusionary test that eliminates individual studies based solely on whether the data is transparent, there is, however, no clear mandate that the raw data underlying these models be publicly available in a manner sufficient for independent evaluation.
Commenter (11495) discusses the level of data necessary for validation or replication. They highlight that epidemiological studies utilize several individual, but related, data sets to formulate conclusions. The commenter notes that only after the evaluation of these data sets by experts of different fields is a "final data set" established from which the study findings are finalized. The commenter notes that the supplemental proposal toggles between the need for all data from a study and only requiring the final data set that includes the interactions of all underlying data, and also indicates that the EPA's discussion of data is alternating and confusing.
Commenter (12435) asserts that the definition of data is too broad to clarify the application of the rule and leaves room for the coverage of potentially any information. As published scientific results are often the final steps in a process involving several processing and analysis steps, the commenter contends that the definition provided does not clarify what step of data processing would be subject to this rule.
Commenter (12471) recommends that an additional definition for "Validated Data" be provided to mean "data with a proper level of quality assurance and has all obvious errors removed and that is capable of being analyzed by either the original researcher or an independent party" and the definition of "Data" in the supplemental proposal be edited to read "the set of observed and recorded material in a format that is commonly accepted in the scientific community as necessary to validate research findings.
Commenter (12546) indicates that "Cleaned-up" data is not the same as a data set in which "obvious errors" have been removed. This cleaned data set may also include harmonizing or transforming data to ensure consistency. The commenter notes that when comparing the definition of data used in the supplemental proposal and the 2016 EPA Plan to increase access to results of EPA-funded research (2016 EPA Plan), the 2016 EPA Plan's definition is several steps after the raw data collection, indicating the EPA is inconsistent in its use of "raw" and "clean" data.
Commenter (12712) states the EPA must be clear that the actual final dataset used in statistical analysis is what needs to be made available, referencing an unacceptable form of data presentation in "Assessing PM2.5 Exposures with High Spatiotemporal Resolution across the Continental United States."
Commenter (12720) asserts that the definition of "data" is inadequate and an additional definition for "raw" data must be published by EPA.
Response: In this final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. As noted by commenters, the step at which the final clean data set will be generated will vary from study to study. The level of detail required for this rule would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information. However, given concerns about potentially skewed analyses, the final definition of "data" maintains the stage of data in which obvious errors have been removed. The EPA acknowledges that this definition of data is different the 2016 EPA Plan to increase access to results of EPA-funded research, but emphasizes that the definition for this final rule has been chosen in order to best meet the goal of reanalysis.
4.3.1.1.1 Accessibility of Raw Data 
Comment: Commenters (11408, 11576, 10742) express concern that this rule may exclude older studies where the raw data and/or models may no longer exist and/or are difficult to access due to outdated methods of storage.
Commenter (11576) states that the EPA's own SAB has warned that "[f]or studies published many years ago, it may not be feasible to deliver public access to data and analytic methods[,]" and "[t]here are also sensitive situations where public access may infringe on legitimate confidentiality and privacy interests, and where exceptions from complete public access may be appropriate." The commenter states that the rule could have the effect of excluding studies where it is not possible to make the underlying data publicly accessible would unnecessarily and irrationally exclude reliable science for no legitimate reason. The commenter states the requirement for publicly available data underlying all model parameters would be particularly problematic and potentially impossible for models developed iteratively over decades, in which model parameters and structure evolve with growing science without the need for publicly available documentation of all the underlying parameters' source data. The commenter states that, while the size and storage practices for many large datasets and model outputs may not be suited for open sharing, this should not exclude them from consideration. 
Commenter (10742) provides that the rulemaking is not clear if this rulemaking will allow EPA to cite and use studies where the data is no longer available due to antiquated technology or shifting norms in science (i.e. studies published before posting data and code publicly was widely encouraged and expected).
Commenter (11576) notes that raw data is less likely to be available over time compared to more processed data.
Response: In this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. 
The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
4.3.1.1.2 Definition Fails to Acknowledge Common Errors
Comment: Commenter (12430) states that unscrubbed "raw" data can inappropriately skew results and cause significant issues in modeling and analysis for third parties who were not familiar with the data collection process due to various errors in the data set. The commenter notes that these errors can include outliers or incomplete data, which are not covered by the SNPRM's definition. The commenter recommends scrubbed data that removes these errors remain valid under the proposed definition.
Response: In this final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. As noted by commenters, the step at which the final clean data set will be generated will vary from study to study. The level of detail required for this rule would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information. However, given concerns about potentially skewed analyses, the final definition of "data" maintains the stage of data in which obvious errors have been removed.
4.3.1.1.3 Definition Oversimplifies Data
Comment: Commenter (12720) references the SAB's suggestion that "it might be reasonable to consider the initial compilation of data (original data) in a spreadsheet as "raw" data," and points out that the Agency uses data from researchers that are not working from simple spreadsheets.
Response: The EPA is not finalizing its definition of "data" as specifically being the "initial compilation of data in a spreadsheet."
4.3.1.2 Definition Is Not Corroborated With Scientific Community
Comment: Commenters (12720, 12727) provide that the definition of data used within the supplemental proposal is not in line with the definition generally used among the scientific community.
Commenter (12720) first asserts that the general conception of data does not include analysis by "an independent party" and that EPA does not provide rationale for the phrase's inclusion in the definition. Additionally, the commenter considers the definition overly broad and that it is the scientific method that allows researchers to arrive at the study result. Furthermore, the commenter asserts that "obvious errors" are not "data," even though some errors may be part of raw data sets not included in the analytical data sets used for the investigation and validation. Lastly, the commenter notes that the "original researcher" in question is not defined nor is there any consideration for typical scientific practice where teams of researchers and others are involved in the collection, analysis, and interpretation of data. 
Commenter (12727) contends that EPA is attributing a flawed notion of validation to the scientific community through the definition and has failed to provide any analysis to suggest what the scientific community believes to be involved in study validation.
Response: The EPA disagrees that the definition of data provided conflicts with that generally accepted by the scientific community. The EPA has included "capable of being analyzed by either the original researcher or an independent party" in its definition of data because this rule is a reflection of the value the EPA places in the ability of subject matter experts being able to independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information.
4.3.1.3 Definition of Data Excludes Information
Comment: Commenter (11361) raises a concern that metadata is explicitly omitted from the requirement that clean data be made public. According to the commenter, metadata is used to orient users to the data, including its design, purpose, collection protocols, measurement precision, data architecture, etc. The commenter contends that, without this information, users may misuse or misinterpret the data, incorrectly calling data and analyses into question.
Response: The EPA notes that in 40 CFR 30.5(f), the EPA clarifies that "dose-response data is considered `publicly available in a manner sufficient for independent validation' when it includes the information necessary for the public to understand, assess, and reanalyze findings," and provides a list of information that may compose the relevant metadata.
Comment: Commenter (11892) asserts that due to tiering exemptions, the data that would be available for analysis by a third party would be limited. The commenter suggests that "supporting QA/QC [Quality Assurance/Quality Control] information" be included in addition to the "set of recorded factual material" to address this limitation.
Response: The EPA disagrees with the commenter, as data is factual material beyond simply information supporting QA/QC, and has therefore not made the suggested change in the final rule.
4.3.1.4 Expansion from Dose-Response Data and Models to All Data and Models
4.3.1.4.1 Opposition/Support for Expansion
Comment: Commenters (9493, 11103, 11362, 11390, 11392, 11393, 11396, 11399, 11400, 11404, 11479, 11484, 11492, 11576, 11895, 11920, 12412, 12430, 12466, 12474, 12540, 12546, 12688, 12715, 12719, 12727, 13549, 14397) oppose expanding the applicability from "dose-response" data and "dose-response" models to all types of data and models.. Specific impacts include:
Commenter (9493) contends that this was a significant expansion. As a result of this, members of the public will be required to address the implications of limiting the Agency's reliance on "environmental fate studies, bioaccumulation data, water-solubility studies, environmental fate models, engineering models, data on environmental releases, exposure estimates, quantitative structure activity relationship data, and environmental studies".
Commenter (11103) notes that the data included in the new definition would include those that inform decisions such as the 2011 Mercury and Air Toxics Standards (MAT), which saves up to 11,000 premature deaths in the United Sates according to the EPA's own projections. The SNPRM, according to the commenter, would undermine EPA's fundamental mission to develop and enforce regulations that protect human health and the environment.
Commenter (11362) states that internal scientific assessments as well as policies under the CAA, Safe Drinking Water Act (SDWA), and TSCA require information that would be covered by the new definition, thus requiring them to disregard crucial and accurate data based on accessibility.
Commenter (11390) indicates that the proposed expansion is virtually all inclusive - given the parameters, it is hard to imagine a scientific study, dataset, or model that would not be subject to the proposed rule.
Commenters (11392, 11484, 11895, 11920, 12466, 11492) contend that the proposed expansion of the rule to all models would hinder EPA's use of the best available science in regulatory decisions.
Commenter (11392) provides that the expansion would impact the application of several data types (bioaccumulation data, data on environmental releases, and exposure estimates) used in EPA's regulation of toxic chemicals. Also, the commenter states there is no rationale for how this expansion would improve the Agency's ability to consider the best available science nor any evidence provided in support of the expanded scope.
Commenters (11393, 14397) note that the expansion would create an even higher burden on scientists or the Agency to maintain the increased amount of information that this definition provides for.
Commenter (11404) contends that the calculation of economic benefits from environmental regulation would not be possible without the data now covered by the expanded definition. The commenter provides the EPA's calculation that a Particulate Matter of 2.5 um or smaller (PM2.5) standard of 11 micrograms per cubic meter would contribute to health gains and lives saved worth as much as $20 billion a year and the Bay Area's estimated cost savings of $456,400 per ton of PM2.5 reduced in the area.[,] According to the commenter, epidemiological data is critical to these analyses and should not be overlooked. Furthermore, the commenter states that covering all data would be harmful from the national level down to the local level and undermine the ability of environmental health agencies to carry out their mission.
Commenters (11479, 12430) indicate that the expanded rule places more burden on modelers, and does not necessarily support best available science. Commenter (12430) states that the expanded scope would limit many studies on public health, and provides the example of fecal indicator bacteria concentrations at beaches, EPA lifetime health advisories for Perfluorooctanoic Acid, The Harvard Six Cities study and the American Cancer Society studies that informed the NAAQS.
Commenter (11576) asserts that the expansion introduces new types of privacy and sensitivity concerns for a broader set of public-health and epidemiological studies and a new set of concerns for many types of environmental data.
Commenter (12540) notes that the move from only dose-response data to all environmental and health data would eliminate studies from consideration by EPA, including those that underly EPA's rules under the SDWA, NAAQS, CAA, and Federal, Insecticide, Fungicide, and Rodenticide Act (FIFRA).
Commenter (12727) states that the expanded SNPRM would conflict with EPA's ability to regulate emissions of hazardous pollutants under the CAA. The commenter references section 112 of the CAA, providing that EPA's definition of "data and models" to include "data on environmental releases" would eliminate data underlying emission standards under the section.
Commenter (12715) contends that while risk assessment evaluations performed under EPA's IRIS are not regulatory in nature, the toxicity factors provided by IRIS are used as the basis for many regulatory standards developed by EPA and numerous states, as well as federal and state guidance values and cleanup decisions. The commenter notes that some states' regulations require that IRIS toxicity factors be used as the basis for human health standards for environmental contaminants. According to the commenter, the SNPRM would significantly limit IRIS's consideration of and reliance on many key studies that are crucial for the assessment of human health effects and development of toxicity factors for environmental contaminants.
Commenter (12719) states that the expansion of the definition would have likely lead to the exclusion of data that contributed to the 1990 amendments to the CAA, which EPA estimated as preventing 230,000 early deaths in 2020.
Conversely, commenters (9587, 11397, 11401, 11499, 11500, 11922, 12406, 12467, 12612) promote the expansion of data coverage. Support for this expansion is due to:
Commenters (11397, 12467, 12612) state that limiting the scope of the proposal to the narrow definition of dose-response data and dose-response models would limit the efficacy of the proposal. Additionally, the commenter (11397) provides that the original limitation to dose-response data would have failed to carry out the directives in Executive Order 13777. 
Commenter (11500) specifically outlines the necessity to apply the transparency rules to NAAQS proceedings, which would require the expansion as there are several types of data in addition to dose-response materials that are key considerations.
Commenter (11922) asserts that the public and regulated entities will have greater confidence in EPA's implementation of its policies and regulatory programs with the expansion of the supplemental proposal to cover all data and models.
Response: The EPA has considered these comments and determined to limit the scope of this final rule to dose-response data. The EPA is focusing on the underlying dose-response data for this rulemaking because of the influence these data have on particularly impactful decisions at the Agency. Risk assessments and regulations that target emissions and risk reduction of one or more pollutants, contaminants, or substances are integral to the Agency's mission and the underlying dose-response data that inform the quantitative value used to evaluate and mitigate potential risk are critical to understanding the assessment or regulatory action. In addition, the data underlying the dose-response assessment are more distinct than the broad range of data informing an entire risk assessment. Therefore, the EPA is concentrating its current efforts to increase transparency on a well-defined step in the quantitative assessment of risk supporting specific Agency actions.
Comment: Commenter (12472) expresses that the expansion could potentially impact on air quality models, and requests that EPA make explicit whether these models are included under the rule. The commenter specifically requests that the EPA clarify whether air quality and associated models such as Comprehensive Air Quality Model with Extensions (CAMx) are included as models subject to the rule.
Response: The EPA has removed requirements pertaining to model because this regulation applies only to dose-response data, and therefore has removed the definition for model from 40 CFR 30.2. 
Comment: Commenter (12699) quotes § 30.3 "The provisions of this part apply to data and models, underlying pivotal science supporting influential scientific information and/or underlying pivotal regulatory science used to justify significant regulatory decisions regardless of the source of funding or identity of the party conducting the science." The commenter notes that the comma after "data and models" appears to be communicating that the provisions apply to data and models, underlying pivotal science, and/or underlying pivotal regulatory science. Unfortunately, the commenter contends that this language could create confusion by giving the impression that the provisions only apply to data and models. According to the commenter, since this is unlikely what the EPA intends, they recommend that EPA make § 30.3 clearer.
Commenter (12699) adds that, if in fact the EPA does only intend to cover data and models, then this would make little sense and beg the question why it would not cover pivotal science and pivotal regulatory science. The commenter notes that both are critical in practice and for purposes of the rule. The EPA would be creating a transparency rule to help the public evaluate the science, and then arbitrarily only allow the public to evaluate a subset of what informs influential scientific information and significant regulatory actions.
Response: The EPA acknowledges the lack of clarity and has altered the portion of 40 CFR 30.3 relevant to this comment to read, "to dose-response data underlying pivotal science."
4.3.1.4.2 Necessity for Expansion
Comment: Commenter (10972) contends that the rulemaking effort proceeds on the premise that you can have the same set of requirements for models and data across different fields. The commenter states that no factual basis is presented for the choice to shoehorn all models and all data in all sectors under the same standard; it is an arbitrary approach. The commenter provides that for example, the preference for non-parametric models in section 30.6 may have been understandable in the context of dose-response models, but it seems irrational to simply extend that attitude to all other models and data in all sciences with no meaningful consideration of the propriety of this approach to those other circumstances. According to the commenter, thoughtful assessments are required to create clear-cut requirements in each sector based on the terms of statute concerned, the goals of the statute concerned, sectoral privacy and other legal requirements and challenges, sectoral ethical issues, the nature of the data predominantly used in a specific sector, and other concerns. The commenter also notes that sectoral sensitivity is the route counselled by the statutory considerations stated above and the nature of the EPA as an Agency with expertise. The commenter states that EPA highlights the former when it notes that there is some conflict between this proposed regulation and some of the laws which it is meant to effectuate and that the generalist approach in this rulemaking ignores relevant factors and considerations. 
Response: The EPA has limited the scope of this final rule to dose-response data, thereby addressing the commenter's concern that the Agency was implementing a "one-size-fits-all" approach to models and data across different fields.
Comment: Commenter (11490) states that their previous testimony's question of "What is the problem this proposed rule seeks to fix? Where is the study for which the lack of access to raw data resulted in misinterpretation or in the promulgation of an inappropriate regulatory standard?" According to the commenter, these questions remain unanswered with respect to dose-response data in regulatory science, and no new information has been provided that rationalizes extending the rule to other data and scientific assessments at EPA. 
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Comment: Commenter (11894) asserts that the proposal overwhelmingly focused on health effects data, and specifically epidemiology, yet the supplemental proposal greatly expands the range of studies while not providing discussion or investigation that the same "problems" asserted in the originally covered studies are also present among the much greater range covered in the supplemental proposal. The commenter notes that such far ranging changes require explanation and supporting data and information.
Response: The EPA has limited the scope of this final rule to dose-response data, thereby addressing the commenter's concern.
4.3.2 Application of the Rule in Respect to Data
4.3.2.1 Variable Application to Types of Data
4.3.2.1.1 Application of Rule to Variety of Data Types
Comment: Commenter (12720) asserts that there is an enormous diversity of data used in EPA decisions and they reference the SAB's urging of EPA to consider how it could tailor approaches to varied data sets, including a trusted third party like the Health Effects Institute (HEI), Data Use Agreements, certification processes, a data archive accessed with confidentiality agreements and others. The commenter notes that the supplemental proposal does however propose a single approach for all types of data and no analysis of how the likely exclusion of specific data types due to this would affect its decisions. 
Commenter (9587) states that the widened scope of the supplemental proposal will require detailed follow-up rulemaking, to apply the rule appropriately to every subcategory of data and models used by EPA. The commenter suggests the rulemaking include provisions for committees deputed to provide detailed guidance for how it should be implemented in every relevant subcategory.
Response: The EPA has limited the scope of this final rule to dose-response data, thereby addressing the commenter's concern that the Agency was implementing a "one-size-fits-all" approach to models and data across different fields.
Comment: Commenter (11892) notes that the example assumptions of "parametric dose-response or concentration-response models; a robust set of potential confounding variables; nonparametric models that incorporate fewer assumptions; various threshold models across the dose or exposure range; and models that investigate factors that might account for spatial heterogeneity," do not typically apply to the data used by wetland managers. Instead, the commenter provides, wetland data is mostly obtained from the field (e.g. plant species occurrence and distribution, and surface and groundwater levels and movement), although laboratory analyses may be incorporated if results are found useful. The commenter also states that wetland managers rely on expertise in the literature and their own expertise in order to determine whether or not their data or other reports yield usable information with implications for wetland management. Furthermore, the commenter provides several reasons the additional transparency requirements do not neatly fit wetland data:
      :: Wetland data is frequently collected according to peer-reviewed procedures approved by EPA through State Wetland Program Development Grants and 106 programs.
      :: While not unique to the topic of wetlands, states and tribes typically have limited independent resources for wetland data collection and rely on peer-reviewed publications for science applicable to their management concern.
      :: Wetland data, when published in peer-reviewed journals, doctoral dissertations or masters' theses represent very specific situations and often limited geographic areas making independent validation difficult to conduct or inappropriate for informing managers in other areas or for similar, but related water resources. Some data are collected on private land and the landowner(s) may not allow others to access these sites.
      :: More current data is often unavailable. However, many older, but still critical data and analyses remain useful in making technical and scientific assessments that can guide management decisions.
      :: Formal literature reviews which achieve the rigor of research best practices and peer review should be included in available science used by EPA when used in concert with the data from the original studies. The in-depth comparative analysis of a formal literature review can add insights from the body of science about a specific topic.
       :: Wetland management is highly influenced by stream management. Stream management often relies on use of engineering models, which typically are in the public domain for use by applicants for regulated activities.
Response: The EPA has limited the scope of this final rule to dose-response data. Wetland data will only fall under this rule insofar as they are integral to characterizing ecological dose-response relationships.
4.3.2.2 Logistics of Handling Data
4.3.2.2.1 Data Sharing of Large Datasets is Logistically Challenging
Comment: Commenters (9362, 11386, 11392, 11495, 11576, 12617, 12720) note that the rule will be burdensome due to the extremely large nature of the raw data being worked with. Specifics:
Commenter (9362) agrees that the rule would require scientists provide their data upon request so that their analyses can be repeated, but does not believe all scientific papers could be applicable due to the volume of data being analyzed.
Commenter (12617) states that the broad application of public-data requirements would eliminate many instructive and valid studies from consideration because of the burden of finding ways to publicly share all code and data. The commenter states that many studies crucial for EPA to consider when developing rules related to these chemicals, may not be considered due to the proposed data access requirements. The commenter states that, since any such studies are based upon very large data sets analyzed with chemometrics, data reduction, and commercial or customized computer programs, it would be no small task to host all data and code in a public manner.
Commenter (11386) uses the example of Tropospheric Monitoring Instrument (TROPOMI), a satellite instrument that provides daily, global observations of column methane in 5.5 x 7km[2] pixels at the highest available resolution. As such, the commenter states the data are not easily stored, let alone manipulated, without significant computing resources that are unavailable on a typical personal computer.
Commenter (11392) asserts that the data required by the supplemental proposal would encompass almost all of the raw data generated in a study. The commenter notes that EPA would be obligated to ensure many different types of data, including meteorological data, pollution values, and data that serves as the basis for epidemiological studies and other research, were publicly availability in an organized format. According to the commenter, this could mean hundreds of files for any given study and require its own unique code simply to maintain the necessary organization. Currently, the commenter states that the EPA does not possess any data sharing system capable of performing this task, nor does the supplemental proposal address how the Agency would develop the additional capacity necessary to comply with the rule. The commenter provides that the rule does not explain the logistical methods that EPA would use to process and manage this enormous amount of data, nor how the Agency would make it accessible to the public. Additionally, the commenter notes that the rule does not clarify whether the Agency intends to develop an internal data storage system, or whether the Agency would utilize external parties.
Commenter (11576) states that studies involving continuous monitoring of air or water pollutants, greenhouse-gas emissions, or any other type of time-series data require sizable infrastructure and data-handling considerations to accommodate the vast size of these datasets. The commenter contends that without advance notice and creation of the infrastructure required to securely support the storage and transfer of large volumes of data, the EPA cannot require researchers to categorically accommodate publicly available data and models for all types of pivotal regulatory science and pivotal science.
Commenter (12617) provides examples of per- and polyfluoroalkylated substances (PFAS) research data sets that include extensive data sets analyzed with chemometrics, data reduction, and commercial or customized computer programs. The commenter asserts that hosting this data and code would be no small task.
Commenter (12720) states that the supplemental proposal does not engage with the idea of "big data" and large datasets that may not lend themselves to data sharing.
Response: The EPA agrees with the commenters and has limited the scope of this final rule to dose-response data underlying pivotal science. Rather than considering all studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact as "pivotal science," the EPA is balancing transparency and feasibility by focusing on those studies that inform the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect. Thus, "pivotal science" includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). This limited scope increases the feasibility of this rule at this stage, and addresses the commenters' concerns.
4.3.2.2.2 Capability of EPA to Host Data
Comment: Commenters (11192, 11193, 11361) state that EPA is not prepared to host information in a manner sufficient for the proposal. Specific concerns include:
Commenters (11192, 11361) assert the EPA must identify and create a structure for publicly available data before requiring studies to make their data public or make decisions about what data can safely be made public through that architecture. The commenters state that, if the EPA seeks to make more studies' data public, then the Agency should, at the minimum: (1) create a public repository and user interface for such data, (2) provide funding explicitly for data de-identification in new federally funded research grants, and (3) admit de-identified data to the repository and into full consideration for influential scientific information.
Commenter (11193) contends that the Agency has provided no methods or structure by which they will inventory, store, process, and analyze the data, analyses, and models they would require to be made public. The commenter notes that currently, the Agency struggles to maintain its data systems consistent with current standards and to serve their own data efficiently to the public. According to the commenter, requiring the Agency to inventory, manage, distribute, analyze, etc. the quantity and density of data that would be involved in the proposed rule and supplemental proposal has not been thought out and no presentation of any such management system or standard operating procedure has been provided. The commenter states that no infrastructure exists for such a system, such that there is effectively no way to implement this program. Also, the commenter notes that the management of such a data system will involve a substantial cost for storage, management of flow, and security. The commenter asserts that no description of the Agency's ability to pay for this effort was presented.
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenters (9372, 11900) suggest that, to adequately protect privacy, all data associated with human study participants should only be accessible through secure data enclaves. Commenter (9372) states that, even for data that is already anonymized, most data providers are unwilling to budge on the requirement that data are kept on secure servers. 
Response: The EPA agrees and clarifies that in this final rule the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. 
Comment: Commenter (11176) provides reservations concerning the data sharing infrastructure for classes of data that could inform EPA actions:
 Data owned by the EPA or other federal entity: While there is a push to provide expanded access to data in the federal government, the commenter states that the effort is still in its infancy. The commenter provides that the National Center for Health Statistics (NCHS) Research Data Center (RDC) is one example for providing some level of tiered access, and the National Opinion Research Center (NORC) Data Enclave and the Inter-university Consortium for Political and Social Research (ICPSR) Virtual Data Enclave may provide additional flexibility of providing access. However, the commenter notes that EPA's involvement in such programs is minimal, limited to a single survey. Given this limited involvement, it is premature to enact this rule.
 Federally funded non-EPA data: For a snapshot of the status of data repositories for federally funded research, the commenter considers the NIH, the federal government's likely leader in federally funded data sharing, an effort started more than a decade ago. The commenter provides that, in its frequently asked questions (FAQ) page for data sharing, the NIH responds, "maybe," to question #20 on whether researchers should consider contributing their data to a data archive. The commenter contends that the reply equivocates, of course, because of the many factors involved but also indicates how far the NIH is from achieving full data sharing for the research it funds. For the next question of where to find guidance on preparing data for sharing and archiving, the commenter notes that the NIH FAQ does not provide its own guidance but, except for molecular biology information, refers readers of the guidance to an outside entity, the afore-mentioned ICPSR. According to the commenter, EPA is a relative newcomer to encouraging data sharing and, while it will benefit from the advances and experiences of other federal entities, is still likely to be years away from providing or encouraging the data access assumed in this rule. The commenter also notes that EPA does not seem to require a data management plan, unlike the National Science Foundation and NIH, which it planned to do by 2018 in the 2016 EPA Plan. The commenter states that such a requirement might be a first step towards the eventual goal of EPA funded research data being more widely accessible. And while data management plans and sharing strategies remain work in progress, the commenter contends that they are a mechanism to ensure that large swaths of data become accessible to as broad a public as possible, including through some of the same restricted-access mechanisms mentioned previously. According to the commenter, without ready access to this category of data, the commenter believes it is premature to enact the rule but encourages EPA's continued work to make such data available
Response: In this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA acknowledges that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenter (11361) states that there is currently no central place for academics to publish their data, let alone to publish data with tiered access. The commenter provides that when discussing tiered access, EPA offers the Centers for Disease Control and Prevention (CDC) RDC as an example repository and system. However, the commenter notes that EPA fails to discuss, however, the costs associated with building, maintaining, and providing access to that repository, nor the cost of reviewing and evaluating applications for access. According the commenter, EPA must identify and create a structure for publicly available data before requiring studies to make their data public or make decisions about what data can safely be made public through that architecture. The commenter suggests that the Agency should, at the minimum: (1) create a public repository and user interface for such data, (2) provide funding explicitly for data de-identification in new federally funded research grants, and (3) admit de-identified data to the repository and into full consideration for influential scientific information.
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenter (11479) states that even with the clarifications provided in the supplemental proposal, there is no information on where the raw data collected would reside, how the resources to manage the data would be obtained, or how EPA would deal with legally and ethically protecting confidential or sensitive data. The commenter notes that ensuring all this extra information is received, stored, and made publicly accessible would be no small undertaking. Additionally, the commenter provides that EPA's SNPRM references a system used by the NIH to collect health data, but the NIH system is not publicly accessible and requires significant resources to manage and share with researchers  -  a clear contradiction to the administration's apparent interest in transparency. The commenter contends that at a time when EPA's budget and capacity continues to decrease, it is unreasonable for the Agency to initiate such an undertaking.
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenter (11408) quotes a 2013 study that reviewed requests through the Information Quality Act between 2002 and 2012 that suggests the Agency should establish a database that "registers studies and obtains systematic sets of parameters and results" rather than making all raw data available, such as the NIH's clinical trials database.
Response: The EPA disagrees, and notes that where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information.
Comment: Commenters (11894, 11900) express concern that information regarding the RDC pilot project has not been provided for public comment. Commenter (11900) requests that the details and results of that pilot study be made public.
Commenter (11894) states that EPA staff confirmed that the EPA was working with the NCHS at the CDC to conduct a pilot study using a secure data enclave to host EPA data sets in a restricted use environment to examine how PII data could be protected. The commenter states that they suggested the pilot would be completed in about six months (i.e. end of March) and that "the final rule would build upon the information obtained from the pilot." The commenter states that apparently, this work, which was to be a foundation for the final rule, was not completed in time to inform the SNPRM or the EPA chose to move forward without waiting to get its results. The commenter states that the preamble states that "EPA is currently conducting a pilot study using the RDC's secure data enclave" referring to the Research Data Center (RDC) of the NCHS/CDC. The commenter states that this key study that the EPA told SAB would be "used to inform the final rule" has not been made available for public comment. The commenter states that, apparently, in its place, the preamble notes that the RDC itself is "a model of tiered access for data involving PII." The commenter states that the EPA's inability to define tiered access is emblematic of the utter disregard of both the tradition of sound science/policy principles and the rulemaking process at EPA. The commenter states that this is a violation of adequate notice and opportunity for comment.
Response: The EPA disagrees with the commenter because this final rule does not depend on the RDC pilot study. This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available.
Comment: Commenter (11900) asks how the EPA intends to address issues of ownership and responsibility for data deposited in the restricted enclave. The commenter states that stewardship of data involves maintenance of the infrastructure for storing and accessing the data, accountability for the data quality and underlying methods that generated it, and moral responsibility to the study participants. The commenter states that researchers may be reluctant -- or unable -- to cede moral responsibility and accountability for the data to EPA, even if they can cede maintenance of the data. The commenter states that, in light of their ongoing responsibilities, the original study investigators should be granted some input over who is allowed access to their data and for what purposes. The commenter states that one way to do this would be to give the original study investigators an automatic seat on the Data Access Committee for any application seeking to use their data. The commenter states that, in this way study investigators will have the opportunity to review and comment on research proposals seeking to use their data. 
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Further, it does not direct parties outside the EPA on how to provide access to data deposited in a restricted enclave. This is a rule of internal agency procedure and does not regulate the conduct or determine the rights or obligations of any entity outside the federal government. 
Comment: Commenters (11184, 11407, 12412, 12474, 13549) state that, in August 2018, the National Tribal Air Association (NTAA) expressed concerns about EPA's belief that confidential or private information could be dealt with by the application of solutions in use across other parts of the federal government. The commenter states that the NTAA also expressed concerns about the cost of doing so and the reduction in studies this would cause the EPA to consider. The commenter states that eighteen months later the SNPRM did not address any of these concerns or analyze the costs or impacts. The commenter states that, before the EPA can finalize a rule that relies on certain approaches to provide, yet protect data, it must lay out and analyze those approaches for public review and comment.
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenters (12546, 12720) express concern that, while the SNPRM describes development of standard data repositories as "ongoing," the EPA does not explain why the EPA should introduce burdensome new public data release requirements at a time when such repositories have not demonstrated the ability to handle such a massive influx of data and models in a way that actually improves public access to data in a meaningful way. 
Commenter (12546) notes that in the SNPRM, the EPA indicates that it "...is currently conducting a pilot study using the RDC's secure data enclave to host EPA datasets in a restricted use environment. Development of standard data repositories is still ongoing." The commenter states that the EPA is proposing to change the scientific basis of their rulemaking based on an untested and unverified method for data security enclaves. The commenter states that the EPA needs to put out a rule that has a tested security method in place, not a `plan' for a system that has not been tested and evaluated. The commenter states that the EPA should not be basing regulations on pilots, but rather should focus on implementing the EPA's own 2016 Plan to increase access to results of EPA-funded scientific research.
Response: The EPA disagrees that the practices informing this final rule are untested. The transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency. The EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. For example, in 2002 the OMB released its Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies, which includes discussion of the importance of the reproducibility of analyses underlying influential information. The EPA's 2016 Plan to Increase Access to Results of EPA-Funded Scientific Research noted that "transparency is a core EPA value" and that increased availability of research data would accelerate scientific breakthroughs that support the Agency's mission and policymaking efforts. The EPA's Open Government Plan 5.0 also details the EPA's progress in implementing the tenets of the numerous data transparency initiatives in the federal government prior to 2018, including OMB M-10-06, the Office of Science and Technology Policy Memorandum of February 22, 2013, and OMB M-13-13. In 2019, Congress passed the Foundations for Evidence-Based Policymaking Act of 2018 (or OPEN Government Data Act, Pub. L. 115-435) into law, which included requirements for federal agencies to prioritize making their data available to the public, and OMB has released additional guidance for implementing the act.[,]
4.3.2.3 Addressing Data From Various Sources
4.3.2.3.1 Application to States and the Federal Government
Requirements of Federal Agency Research
Comment: Commenter (12402) recommends that studies and data developed under EPA testing programs, where EPA test guidelines or other validated and accepted test methods such as those developed by the Organisation for Economic Co-operation and Development (OECD), American Society for Testing and Materials (ASTM) International have been used, and EPA Good Laboratory Practices have been followed and certified should be candidates for exemption from the SNPRM. It can reasonably be assumed that such studies and data have already been subject to adequate validation review by the Agency and have been subject to appropriate public notice and comment in accordance with statutory requirements of the relevant EPA programs. For example, the commenter acknowledges an extensive chemical fate and environmental effects testing program under an enforceable testing Consent Order under the TSCA for which EPA was involved in all aspects of the data development. The commenter notes these same studies were reviewed again by the Office of Water as part of an extensive review of available data for the development of Ambient Aquatic Life Water Quality Criteria for Nonylphenol (NP) in 2005. According to the commenter, it is likely that these same studies will be reviewed again when EPA conducts its TSCA risk evaluation on this substance, which is listed on EPA's 2014 update to the TSCA Work Plan. Considering that these studies are now thirty years old, the commenter states that providing public access to raw data will likely be difficult and should be unnecessary considering that the studies were developed under EPA oversight and reviewed previously under EPA programs.
Response: The EPA believes that its efforts under this rule can be additive to existing validated and accepted test methods. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Further, the EPA directs the commenter to the factors in 40 CFR 30.5(d) that the EPA shall evaluate when determining the amount of consideration to grant pivotal science where the underlying dose-response data are not available for independent validation.
Comment: Commenter (12720) asserts that EPA has not explained or anticipated the impact of the supplemental proposal on other federal agencies that rely on similar types of scientific data and models to develop influential scientific information. The commenter provides the recent joint publication of the Safer Affordable Fuel-Efficient(SAFE) Vehicles Rule for Model Years 2021-2026 Passenger Cars and Light Trucks by EPA and the National Highway Traffic Safety Administration as an example of potentially impacted projects. Additionally, the commenter states that the supplemental proposal does not explain how requirements introduced at EPA would affect significant scientific work at other federal agencies and appears to undermine longstanding scientific review processes at other federal agencies, including the Department of the Interior.[,]
Response: This is a rule of agency procedure internal to the EPA. Where the EPA is collaborating on a significant regulatory action or influential scientific information with another federal agency, this rule would apply. However, the EPA notes that The transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency. The EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. For example, in 2002 the OMB released its Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies, which includes discussion of the importance of the reproducibility of analyses underlying influential information. The EPA's 2016 Plan to Increase Access to Results of EPA-Funded Scientific Research noted that "transparency is a core EPA value" and that increased availability of research data would accelerate scientific breakthroughs that support the Agency's mission and policymaking efforts. The EPA's Open Government Plan 5.0 also details the EPA's progress in implementing the tenets of the numerous data transparency initiatives in the federal government prior to 2018, including OMB M-10-06, the Office of Science and Technology Policy Memorandum of February 22, 2013, and OMB M-13-13. In 2019, Congress passed the Foundations for Evidence-Based Policymaking Act of 2018 (or OPEN Government Data Act, Pub. L. 115-435) into law, which included requirements for federal agencies to prioritize making their data available to the public, and OMB has released additional guidance for implementing the act.[.]
States
Comment: Commenter (12456) asserts that it may not be possible for certain information to be made public due to state requirements that prevent the release of specific information related to private lands under the Colorado Open Records Act. It is also noted that the data covered by this Act should not be at the discretion of the EPA Administrator, nor excluded from consideration due to the study taking place on private lands in Colorado.
Response: The EPA does not require external parties to make their dose-response data available. Where the Agency is making dose-response data publicly available, it shall do so in a fashion that is consistent with law, protects privacy, confidentiality, CBICBICBICBI, and is sensitive to national security.
Comment: Commenter (10974) is concerned that the supplemental proposal could undermine state-level efforts at developing health-based benchmark values used in air toxics and permitting reviews. 
Response: The EPA disagrees with the commenter that this final rule will undermine state-level air toxics activities or permitting reviews. The Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
4.3.2.3.2 Treatment of Studies From Subscription and Open Source Journals
Comment: Commenter (12455) raises a concern that depending on the nature of the journal the data is published in; the data will be treated differently.
Response: Consistent with existing Agency practice, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information. The EPA will continue to use the following, established factors to assess the quality of studies used to develop significant regulatory actions and influential scientific information: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review.[,] When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Only after a study has been designated as pivotal science, the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation.
4.3.2.4 Reporting Funding Sources
Comment: Commenter (11892) recommends that the rule make it clear that the same stringent standards for transparency and quality of data to all information used in decision-making regardless of its source, whether provided by public agencies, the academic community, regulated entities or other sources. Moreover, the commenter states transparency would be increased by requiring information regarding financial support for the research. According to the commenter, funding by federal agencies and many foundations are typically identified in research reports, but corporate and other private funders may not be.
Response: The EPA notes that the final rule codifies in 40 CFR 30.3: "The provisions of this part apply to science that serves as the basis for informing a significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science." The EPA anticipates that giving greater consideration to pivotal science where the underlying dose-response data is available for independent validation will offer increased transparency, but notes that this rule is an incremental part to increasing transparency in EPA significant regulatory actions and influential scientific information.

 Chapter 5: Influential Scientific Information; Expansion of Proposal; Determining and Weighting ISI (Supplemental Notice of Proposed Rulemaking (SNPRM))
In the supplemental notice of proposed rulemaking (SNPRM), the EPA proposes to expand the scope of the notice of proposed rulemaking (NPRM) to apply to "influential scientific information", defined as follows:
      Influential Scientific Information means scientific information the Agency reasonably can determine will have or does have a clear and substantial impact on important public policies or private sector decisions.
The SNPRM also provides revisions to the NPRM regulatory text at 40 Code of Federal Regulations (CFR) 30.5 from that in the Proposal. Under the proposed new approach at 40 CFR 30.5, when promulgating significant regulatory decisions or finalizing influential scientific information, the EPA would only use pivotal regulatory science and/or pivotal science if the data and models are available in a manner sufficient for independent validation. The SNPRM provides that this includes studies with data and models that are publicly available as well as studies with restricted data and models (i.e., those that include confidential business information (CBI), proprietary data, or personally identifiable information (PII) that cannot be sufficiently de-identified to protect the data subjects) if there is tiered access to these data and models in a manner sufficient for independent validation.
As an alternative, the SNPRM preamble states that under proposed 40 CFR 30.5, when promulgating significant regulatory decisions or finalizing , other things being equal, the EPA would give greater consideration to studies where the underlying data and models are available in a manner sufficient for independent validation either because they are publicly available or because they are available through tiered access when the data includes CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects. The SNPRM preamble further states that EPA would identify those studies that are given greater consideration and provide a short description of why greater consideration was given. 
Several comments were received regarding the SNPRM coverage and use of influential scientific information. These comments are organized under the following topics:
 Definition of Influential Scientific Information
 Expansion of Scope to Include Influential Scientific Information
 Determining What Constitutes Influential Scientific Information
The specific comments provided in this Chapter reflect the comment letters that provide arguments supported by detailed rationale, examples, and references/documentation. The Docket ID numbers for these commenters are cited throughout this Chapter. In addition, some general comment letters may also include comments related to comments included and responded to in this Chapter. These comments are summarized in Appendix B. Note that points and issues raised in these general comment letters may also be raised by the more detailed comments summarized in this Chapter. The responses provided in this Chapter adequately address these more general comments.
5.1 Definition of Influential Scientific Information
 5.1.1 General Support/Dissent for Definition
Comment: Commenter (14397) states that the term "influential scientific information" remains vague and unworkable. 
Commenter (11390) asserts that the new definitions provided in the supplemental proposal are so expansive and vague as to render the proposed rule applicable to any scientific study that EPA considers for nearly any meaningful purpose, outside of the supplemental proposal's exceptions for adjudications and enforcement activities. 
Commenter (12546) states that this definition of "influential scientific information" essentially includes all decisions and documents that EPA generates which use science, in addition to the already expanded definition of "pivotal regulatory science." The commenter also notes that the definition does not include impact on human health/public health, which is EPA's mission.
Conversely, commenters (9587, 11401, 11499, 12406, 12471) support EPA's proposed definition for "influential scientific information". Specific comments include:
Commenter (12471) states that having such a definition seems appropriate as the term appears in the Memorandum and yet the Memorandum does not specify a definition, nor is one found in the 2002 Office of Management and Budget (OMB) Guideline. The commenter suggests a minor revision to the definition of influential scientific information as follows: 
      Influential scientific information means pivotal science in an area of inquiry, when taken together, represents the synthesis of scientific information the Agency reasonably can determine will have or does have a clear and substantial impact on important public policies or private sector decisions," in order to link the definition to pivotal science.
Response: The definition of influential scientific information in 30.2 of the final rule remains, "scientific information the agency reasonably can determine will have or does have a clear and substantial impact on important public policies or private sector decisions." The Agency will not be taking suggestions to modify the definition because this definition of influential scientific information is the same definition as in OMB Bulletin for Peer Review. In addition, the Agency does not believe this definition is expansive or vague. The EPA adopted this definition because it intended the scope to be consistent with how that term has been interpreted and applied in the context of peer review. Given that the definition is both established and has been routinely applied by the EPA, the EPA disagrees that the term is inherently too narrow or too broad. Rather than modify the proposed 40 CFR 30.2 definition of influential scientific information, the EPA is modifying 40 CFR 30.3 to clarify the Agency's intent that the requirements in 40 CFR Part 30 will apply to influential scientific information, unless the influential scientific information is exempted from peer review requirements as described in Section IX of the OMB Final Information Quality Bulletin for Peer Review. Consistent with this approach, the EPA is finalizing the definition of "influential scientific information" as proposed in the 2020 SNPRM. The EPA's Peer Review Handbook provides greater guidance on how the Agency identifies influential scientific information.
5.1.2 What is Covered by the Definition of Influential Scientific Information
Comment: Commenter (11381) provides that EPA defines influential scientific information without defining the predicate noun. The commenter notes the supplemental proposal definition approximates a key Information Quality Guidelines (IQG) definition, but that the supplemental proposal definition is narrower  -  it excludes financial or statistical information  -  without provision of a rational basis for the departure from the IQG. As such, the commenter asserts this leaves the Agency's definition ambiguous at best. The commenter states that the only subsets of information that assuredly lie within the definition are data and models, and thus leave the public and Agency staff confused. Furthermore, the commenter questions if the supplemental proposal includes only data and models, or all information ("any communication or representation of knowledge such as facts or data, in any medium or form") that is influential ("information the Agency can reasonably determine that dissemination ... will have or does have a clear and substantial impact on important public policies or important private sector decisions"). The commenter notes that because elsewhere the supplemental proposal focuses on data and models, this is not an insignificant concern and that, unless the matter is clarified, it will be the cause of wholly unnecessary and pointless argument.
Commenter (11390) notes that the defined terms are so broad they ostensibly could apply even to science that never actually impacts any Agency policy or regulation: The commenter states that if EPA finds science reasonably "will have" an impact in the future, that is enough to bar the Agency from considering key data in the present, whether or not such an impact ultimately occurs. Moreover, the commenter states that a wide range of EPA activities might be found to impact "private sector decisions" outside the scope of the Agency's own regulatory authority, making the potential reach of this provision virtually limitless.
Commenter (11399) provides that the definition provided in the supplemental proposal does little to clarify the meaning of "influential scientific information," which is troubling in a rule concerned with transparency. Thus, the commenter does not support this expansion to the proposed regulatory language.
Commenter (12645) asserts that a quick look at the recent Health Effects Institute (HEI) Review Committee reports shows a median of over 20 references that are cited by the reviewers. According to the commenter, the reviewers do not look at the raw data in order to write their influential analysis, nor would all cited references have such data available. Therefore, the commenter questions if EPA plans to tell HEI not to allow its Review committee to consider any literature for which the committee members have not personally evaluated the completed data, or at least are certain that the completed data are available? Similarly, the commenter notes the EPA's own analyses, such as the Integrated Science Assessments (ISA) reviewed by the Clean Air Science Advisory Committee (CASAC), often simply cites previous EPA documents, or other reviews such as those performed by the National Academy of Science (NAS), as foundations on which they then overlay the more recent scientific studies. The commenter questions whether the EPA now plans to remove these foundations, or to subject the literally thousands of papers that went into the weight of evidence for previous EPA analyses to considerations as to which are pivotal or influential.
Commenter (12727) states that there is not an adequate notice of the types of influential scientific information to which the rule could apply. The commenter notes that although some influential scientific information could have been indirectly excluded from use by the Agency by the Proposal  -- e.g., if destined to support a significant regulatory decision -- the supplemental proposal imposes an independent and immediate restriction on the development of all influential scientific information.
Commenter (12715) provides that the supplemental proposal's statement that the requirement only apply to studies that "could drive [EPA's] subsequent decisions," 85 Fed. Reg. at 15,399, is inconsistent with information EPA provided to staff of the House Committee on Science, Space, and Technology that for scientific evaluations designated as influential scientific decisions "every study considered by the Agency in writing a rule, regulation, risk assessment, and more would be subject to these data transparency requirements."
Response: The definition of influential scientific information in 30.2 of the final rule remains, "scientific information the agency reasonably can determine will have or does have a clear and substantial impact on important public policies or private sector decisions." This definition of influential scientific information is the same definition as in OMB Bulletin for Peer Review. The EPA adopted this definition because it intended the scope to be consistent with how that term has been interpreted and applied in the context of peer review. Given that the definition is both established and has been routinely applied by the EPA, the EPA disagrees that the term is inherently too narrow or too broad. The EPA's Peer Review Handbook provides greater guidance on how the Agency identifies influential scientific information. All Agency influential scientific information, besides going through peer review, will also be subject to public notice and comment.

However, in the 2020 SNPRM, the EPA did not incorporate the OMB Bulletin for Peer Review Section IX exemptions to peer review, including routine statistical information and financial information. Rather than modify the proposed 40 CFR 30.2 definition of influential scientific information, the EPA is modifying 40 CFR 30.3 to clarify the Agency's intent. As a result, the requirements in 40 CFR Part 30 will apply to influential scientific information, unless the influential scientific information is exempted from peer review requirements as described in Section IX of the OMB Bulletin for Peer Review. Consistent with this approach, the EPA is finalizing the definition of "influential scientific information" as proposed in the 2020 SNPRM. It is the Agency's intent to be consistent with the OMB Bulletin for Peer Review regardless of how commenters may interpret that intent or potential inconsistency.
 The application of this final rule to influential scientific information applies only during the develop of the study and then is narrowed even further as it only addresses dose-response data underlying the pivotal science used to develop that influential scientific information. To address the commenter's concern directly, though there may be "thousands of papers that went into the weight of evidence for previous EPA analyses," only a small number of studies will be consider pivotal science when applying the provisions of this final rule to either significant regulatory actions or influential scientific information. 
 
 In addition, the final rule focuses solely on dose-response data. The scope of dose-response data in this final rule is those studies consisting of the data integral to characterizing dose-response relationships. In some instances, this group will consist of a handful of studies. In other instances, where there are multiple toxicity endpoints, there may be more studies that are crucial to characterizing dose-response relationships. In some other cases, there may be a large number of studies that are used to characterize a dose-response relationship (e.g., where the dose-response is based on a meta-regression of epidemiology studies). However, not all of these studies would be considered pivotal science (see Section III.C.6 for the definition of "pivotal science").

Past briefings to Congress were intended to informational as of that moment during the Agency's deliberative process. The final rule, including its applicability, definitions, etc., considers all comments and represents the Agency's final decisions.
The EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions and finalizing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information.
And finally, this final rule does not regulate any entity outside the EPA (e.g. the commenters reference to HEI). Rather, the requirements modify the EPA's internal procedures regarding the transparency of pivotal science underlying significant regulatory actions and influential scientific information.
5.2 Expansion of Scope to Include Influential Scientific Information
 5.2.1 General Support/Dissent for Expansion
Comment: Several commenters (10865, 11388, 11407, 11895, 12702, 13788) generally oppose the expansion of coverage of the rule to include influential scientific information. Specific comment examples include:
 Commenter (11388) does not support the expansion due to the complexity and impracticality of including influential scientific information and instead recommends the final rule only apply to underlying studies, data, and models when promulgating significant regulatory actions.
 Commenter (11895, 11407) states that with the expansion, the supplemental proposal takes a bad proposal and makes it worse and that the expansion does nothing to address the crux of the problem: there are many valid and important scientific studies that cannot legally, ethically, or efficiently publicly release all their data.
 Commenter (10865) states that it is troubling that the scope of the SNPRM now includes "influential scientific information" in addition to significant regulatory actions.
 Commenter (12702) provides that the expansion broadens uncertainty and possible regulatory conflict.
 Commenter (13788) states that the expansion of the scope to include ISI is unwarranted and inconsistent with the IQA.
Conversely, several commenters (11397, 11401, 11499, 11500, 11922, 12402, 12612, 12699) generally support the expanded scope as defined. Specific comment examples include:
 Commenter (11397) asserts that the expansion is clearly in keeping with the goal of the rulemaking, and of Executive Orders 13777 and 13783, to apply transparency provisions not just when EPA is promulgating regulatory decisions or engaging in notice and comment rulemaking but also when the Agency is finalizing key scientific information upon which EPA may rely in the future. The commenter believes that the currently proposed extension of transparency to finalization of key scientific information  --  regardless of whether it is attached to a particular regulatory decision  --  is the correct approach.
 Commenter (11401) states that all scientific information used by EPA should adhere to the highest scientific standards.
 Commenter (11500) notes that the final regulation should make it clear that such policies or decisions include potential use of the information in public or private litigation. The commenter asserts that the need to apply the transparency rules to all key scientific information is highlighted by the EPA's current Integrated Science Assessment (ISA) for particulate matter (PM). The commenter notes that the EPA's CASAC did not support the ISA's findings that PM that have a diameter of less than 2.5 micrometers (PM2.5) is "likely to be causal" for cancer and that the PM that have a diameter of 10 micrometers and smaller (PM10) data are "suggestive" of causality, yet they remain in the final ISA, which can be relied upon in toxic tort litigation or other proceedings as official EPA findings.
 Commenter (11922) believes these expansions along with the SNPRM's proposed definitions and clarifications would enhance the transparency proposal's objectives to ensure that the data and models underlying the science supporting significant regulatory actions are publicly available in a manner to allow for analysis and validation. The commenter states that the more transparent the underlying scientific and regulatory science utilized in formulating policy decisions effecting regulated entities, the more confidence both the public at large and regulated entities will have in EPA and its implementation of policies and regulatory programs.
 Commenter (12612) notes that the expansion is an improvement that will increase the functionality of the Proposal in increasing EPA transparency.
 Commenter (12699) states that the expansion is an important change that indicates EPA's recognition that the science the Agency disseminates even outside the regulatory context can have a major impact. The commenter notes the definition of influential scientific information used in the supplemental proposal is drawn from OMB's "Final Information Quality Bulletin for Peer Review".
 Commenter (11499) supports retroactive application in stating that existing data and models should be covered if they underlie "pivotal regulatory science" in a rulemaking or "pivotal science" in the development of influential scientific information.
Commenter (13788) states that the expansion of the scope to include ISI is unwarranted and inconsistent with the IQA.
Response: Influential scientific information remains in the scope of the final rule. Because of the potential impact of the EPA's significant regulatory actions and influential scientific information on American lives and livelihoods, the American people deserve environmental decisions and policies that are based on the best scientific information. Only through continuous improvement to its procedures, especially those focused on transparency, can the EPA fully demonstrate that commitment. Where underlying dose-response data in pivotal science are available, subject matter experts can independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information.
The EPA is finalizing the applicability of this final rulemaking to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods. The EPA is not applying this rulemaking to permit proceedings, site-specific actions, or enforcement actions because these actions are typically focused on individual regulated entities.
5.2.2 Justification/Necessity for Expanding Scope
Comment: Commenters (10048, 10866, 11392, 11404, 11894, 12456, 12727, 12715) note that the supplemental proposal does not provide adequate justification for the expansion in scope to include influential scientific information. Specific comment examples include:
 Commenter (10048) states that the vague and broad scope is problematic and the supplemental proposal does not provide justifications for these changes, just as there is no justification for the proposal itself.
 Commenter (10866) notes the supplemental proposal fails to adequately justify the proposed rulemaking as it applies to "influential scientific information as well as significant regulatory actions."
 Commenter (11392) states that the supplemental proposal does not provide justification for the sweeping expansion, which makes it difficult for the public to assess the extent of the impact that the rule would have on the Agency's ability to consider the best available science. As such, the commenter asserts that EPA should remove influential scientific information from the scope of the rule. The commenter notes that, if EPA insists on maintaining influential scientific information within the scope of the rule, EPA must address why it decided to apply the rule to influential scientific information as well as significant regulatory actions and provide a detailed explanation of the factual evidence that it used to justify its decision.
 Commenter (11404) provides that the expansion is unnecessary, vague, and dangerous.
 Commenter (11894) states that the supplemental proposal does not in any way address either the legal support for this proposal or the policy need for it. Nor does it delineate its scope or how it would work.
 Commenter (12456) notes such a vague and broad scope is problematic, and the supplemental proposal does not provide justification for these changes.
 Commenter (12727) asserts that the closest the Agency comes to an explanation is a passing reference to OMB implementation updates recommending that "[a]gencies should prioritize increased access to the data and analytic frameworks (e.g., models) used to generate influential information." The commenter contends that the provisions in the proposed rule that would allow the EPA to disregard data and analytic frameworks in developing influential information are inconsistent with the OMB implementation updates. As such, the commenter provides that absent any rationale for expanding the scope of its rule to influential scientific information, finalizing the supplemental proposal would be arbitrary. 
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. OMB defines both significant regulatory actions and influential scientific information and has provided guidance as to how to identify and manage these actions because of the potential impact of these actions on American lives and livelihoods. This narrow but impactful scope of Agency actions is an important first step towards creating greater transparency.
These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. The EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information. The EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions and finalizing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information.
5.2.3 Application Guidance for EPA Programs
Comment: Commenter (12406) supports the inclusion of "influential scientific information", but believes that each program office at EPA should establish criteria and guidance to determine the circumstances under which the rule will be applied to "influential scientific information" within the context of each statutory and regulatory scheme, subject to public notice and comment. The commenter states this would reduce the risk and uncertainty of an overly expansive definition that might needlessly slow implementation within the context of some statutes. For example, the commenter notes that applying the Strengthening Transparency rule to Toxics Substances Control Act (TSCA) Section 5 new chemicals program may render review of premanufacture notices impossible within the 90-day statutory timeframe. For these reasons, the commenter recommends that EPA clarify applicability with subsequent criteria and guidance.
Response: In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. The EPA intends to issue implementation guidelines and statute-specific rulemakings, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. In this action, the EPA is finalizing a rule of internal agency procedures, including how the Agency will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information for independent validation.
5.2.4 Public Comment on Expanded Scope
Comment: Commenters (11361, 11377, 11396, 11492, 11920) state that the expansion to include "influential scientific information" will undercut EPA's ability to fulfill its mission to protect human health and the environment. Commenter (11377) asserts that systematically and arbitrarily favoring fully public data for even more of the Agency's work only further imperils citizens whose health depends on EPA's use of the best science, not just the most public.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. The EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.

Expanding this final rule to include influential scientific information will not undercut the Agency's missions but rather by having a better understanding of the underlying data, this final rule will enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information.

The EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions and finalizing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information.
Comment: Commenters (11393, 11407) state that if EPA refuses to consider a valid scientific study when finalizing "influential scientific information," the public might never be informed, have the opportunity to comment, or bring a legal challenge.
Response: The EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions and finalizing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available for independent validation.
As stated in section 30.5, "The Agency will identify the pivotal science that was given lesser consideration and provide a short description of why lesser consideration was given." This will ensure that the public is properly informed and has the opportunity to comment.
Comment: Commenters (9493, 11894, 12727) contend that the expansion of the proposal requires additional comment to fully address the new reach of the rule. Specifically:
 Commenter (9493) provides that due to the sprawling set of documents that play critical roles in protecting human health and the environment listed in the Agency's most recent Peer Review Agenda, additional comment is necessary to fully address all of these materials.
 Commenter (11894) asserts that EPA must afford the public notice and opportunity to comment on findings on the rationale concerning the actual effects of its supplemental proposal as it is presently unavailable to commenters.
 Commenter (12727) provides that commenters cannot anticipate the reach of the supplemental proposal or account for any potential overlap with the original due to a lack of evaluation on the breadth of the rule in regards to influential scientific information.
Response: The EPA disagrees with these comments. The SNPRM clearly states the potential new scope of the rule, provides background documentation and definitions (e.g. what is influential scientific information) to allow the public to understand the scope and rational and asks the public to comment on various options presented.
5.3 Determining What Constitutes Influential Scientific Information
Comment: Commenters (9362, 11390, 11404, 11892, 12430, 12465, 12688, 12720) contend that the supplemental proposal states the applicability to "influential scientific information" but provides no clarification as to what information the Agency will consider to fall under this category. Specific comments include:
Commenter (9362) questions how "the Agency reasonably can determine" whether a scientific study "will have or does have a clear and substantial impact on important public policies and private sector decisions."
Commenter (11390) provides that the supplemental proposal does not explain how EPA will determine particular science has such a "clear and substantial impact"; there are no factors for the Agency to consider when making such a determination, nor examples of scientific information that would be encapsulated within the definition. 
Commenter (11404) notes that determining a level of "influence: and deeming what knowledge qualifies as "pivotal" is too subjective. According to the commenter, science is science; it should be evaluated as such and sufficient methods of evaluation already exist.
Commenter (11892) states that it is unclear how "influential scientific information" will be used differently than has already been done by EPA in the past, while maintaining consistency with the OMB guidance on peer reviews and the Data Quality Act. Additionally, the commenter notes it is unclear how EPA will make the decision on whether something is "influential scientific information" and what the detailed criteria would be for that evaluation. The commenter suggests that the final rule make this clear.
Commenter (12430) asserts that the definition of influential scientific information offers no parameters for determining whether scientific information could have "a clear and substantial impact" or which hypothetical "public policies or private sector decisions" should be considered "important." Accordingly, the commenter states that the breadth of the definition would empower the administrator to make subjective, ad hoc -- and potentially inconsistent or politically motivated -- determinations about whether theoretical "policies" and "decisions" would be "important" and thus would justify the application of the supplemental proposal.
Commenter (12465) provides that the supplemental proposal is unclear what transparent criteria for the dissemination of "influential scientific information" would meet a "novel, controversial, or precedent-setting" determination. Additionally, the commenter states that scientific advancements are invariably "novel, controversial, or precedent-setting" which would tend to discourage advancement of science and is counter to basic scientific principles. The commenter contends that it is also not clear how the scope of the regulation interplays with scientific information that is outside of the scope initially but later becomes within the scope. For example, the commenter notes that a series of smaller studies outside of the scope of the rule would not be held to the transparency requirements, however, if they are appropriately scientifically rigorous, and are at some future point incorporated into a larger activity, they would be precluded from inclusion.
Commenter (12688) notes that the supplemental proposal sets out no test, or even parameters or factors to guide the Agency in determining what information might have a "clear and substantial impact," thus amplifying all the concerns the commenter had with the Proposal.
Commenter (12720) states that the definition of "influential scientific information" is overly vague, as it does not describe how, when, or why the Agency makes such a determination about any such information. Furthermore, the commenter notes the SNPRM does not define what a "clear and substantial" impact is and does not specify if and how such a determination is made or documented. Additionally, according to the commenter the definition is not specific to EPA's research review context and is vague and overly expansive in not describing the types of "public policies" and "private sector decisions" are affected by the nature of influence of the science itself, rather than EPA's determinations about that scientific content within the larger context of all available research. Notably, the commenter asserts EPA does not restrict "influential scientific information" to include only "publicly available" scientific information, because a large portion of important science may or may not be publicly available. As such, the commenter provides that the inconsistency between what EPA currently considers to be influential science on its own web site and how it is defined in the SNPRM signals EPA's lack of adequate advance consideration to define this term.
Response: The definition of influential scientific information in 30.2 of the final rule remains, "scientific information the agency reasonably can determine will have or does have a clear and substantial impact on important public policies or private sector decisions." This definition of influential scientific information is the same definition as in OMB Bulletin for Peer Review. The EPA adopted this definition because it intended the scope to be consistent with how that term has been interpreted and applied in the context of peer review. 

The interpretation of the term "influential" is consistent with OMB's government-wide information quality guidelines (IQG) and the IQG of the Agency. At EPA, scientific and technical work products that will have or do have a clear and substantial impact on important public policies or private-sector decisions would be considered influential. The following factors (See Chapter 3.2 EPA' Peer Review Handbook, 4[th] Edition) are considered when determining whether a product is likely to be influential:
 Establishes a significant precedent, model or methodology.
 Is likely to have an annual effect on the economy of $100 million or more, or adversely affect in a material way the economy; a sector of the economy; productivity; competition; jobs; the environment; public health or safety; or state, tribal or local governments or communities.
 Addresses significant controversial issues.
 Focuses on significant emerging issues.
 Has significant cross-agency and/or interagency implications.
 Involves a significant investment of agency resources.
 Considers an innovative approach for a previously defined problem, process, or methodology.
 Satisfies a statutory or other legal mandate for peer review.
The Agency disagrees with commenters that the definition, use or application of the term influential scientific information is vague or unclear.
Comment: Commenter (12715) provides that the current supplemental proposal states EPA would continue to use standard processes for identifying, evaluating, and reviewing available data, models, and studies," and that the requirement that studies have data and models available for independent validation would only apply to studies that "could drive [EPA's] subsequent decisions." The commenter asserts the distinction does not appear to be logical or defensible. Using human health risk assessment as an example, the commenter notes that the elements of the scientific evaluation other than the quantitative basis of the toxicity factor are just as important. Additionally, the commenter states that it is not logical that EPA would exclude studies from consideration in the quantitative part of its scientific evaluation but would consider these same studies in other equally important components of the scientific evaluation.
Response: The final rules defines pivotal science as "the specific dose-response studies or analyses that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information." The rule also states, "This action establishes how the Environmental Protection Agency (EPA) will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information. When promulgating significant regulatory actions or developing influential scientific information for which the conclusions are driven by the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect, the EPA will give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation."
 
Based on the comments on the 2018 proposed rule and the 2020 SNPRM, including that the number of studies that would be subject to the rule might hinder rulemaking, the EPA determined that the Agency should pursue an incremental approach to maximizing transparency in the science that it relies upon by focusing the final rule requirements on dose-response data and, in particular, only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). The EPA considered commenters' assertions that the scope of the 2018 proposed rule would be so broad as to make implementation infeasible. The 2018 proposed definition of "dose-response data and models" would apply to dose-response data [and models] "used to characterize the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and the magnitude of a predicted health or environmental impact." This relationship of the dose-response data to the magnitude of a predicted health or environmental impact would require the consideration of an array of studies beyond those that characterize dose-response relationships, including, for example, studies that inform the dose-response modeling (e.g., benchmark response selection); studies that identify data for toxicokinetic adjustments that inform calculation of a human-equivalent point of departure (POD); and studies that inform the selection of uncertainty factors. The number of studies that are used to establish the relationship between dose-response data and models and the magnitude of a predicted health or environmental impact can potentially be very large. This may make implementing the rule as proposed infeasible for at least some significant regulatory actions and influential scientific information. While transparency in EPA decision making is the purpose of this action, transparency cannot be effectively accomplished if the means to that transparency are not feasible. Rather than having this final rule apply to all the studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact, the EPA is balancing transparency and feasibility by focusing on those studies that describe the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect. Specifically, the scope of dose-response data in this final rule is those studies consisting of the data integral to characterizing dose-response relationships. In some instances, this group will consist of a handful of studies. In other instances, where there are multiple toxicity endpoints, there may be more studies that are crucial to characterizing dose-response relationships. In some other cases, there may be a large number of studies that are used to characterize a dose-response relationship (e.g., where the dose-response is based on a meta-regression of epidemiology studies). However, not all of these studies would be considered pivotal science (see Section III.C.6 of the final rule for the definition of "pivotal science").
Based on comments and other considerations, the EPA is concentrating its efforts in the final rule to increase transparency on dose-response data, as the dose-response data are discrete and the dose-response assessment is a well-defined and impactful step in the quantitative assessment of risk. This final rule provides an important step in furthering the progress toward maximizing transparency and will provide insight for future statute-specific requirements. 
Comment: Commenter (9455) states that the only measurable criterion of monetary cost in the supplemental proposal is that "influential scientific information which sways public policies and private enterprise decisions and highly influential scientific information which leads to large policies and decisions that would cost $500 million in a given year," fails EPA's fundamental purposes and negates environmental laws. Instead, the commenter provides that bins of scientific information and science should be defined by numbers of lives saved or lost, numbers of cases of chronic conditions avoided or incurred, costs of medical care avoided or incurred, acreage of land contamination avoided or incurred, and other measures of success or failure of policies and decisions.
Response: There are many ways that scientific information can be defined and put into bins, but with respect to this final rule and with regard to influential scientific information, the interpretation of the term "influential" is consistent with OMB's government-wide information quality guidelines (IQG) and the IQG of the Agency. At EPA, scientific and technical work products that will have or do have a clear and substantial impact on important public policies or private-sector decisions would be considered influential. The following factors (See Chapter 3.2 EPA' Peer Review Handbook, 4[th] Edition) are considered when determining whether a product is likely to be influential:
 Establishes a significant precedent, model or methodology.
 Is likely to have an annual effect on the economy of $100 million or more, or adversely affect in a material way the economy; a sector of the economy; productivity; competition; jobs; the environment; public health or safety; or state, tribal or local governments or communities.
 Addresses significant controversial issues.
 Focuses on significant emerging issues.
 Has significant cross-agency and/or interagency implications.
 Involves a significant investment of agency resources.
 Considers an innovative approach for a previously defined problem, process, or methodology.
 Satisfies a statutory or other legal mandate for peer review.
Comment: Commenter (12645) states that studies can only be considered "scientifically influential" in retrospect, i.e., this term is more suitable to the history of science. The commenter posits that if the original Atomic Energy funded studies of radioactive particles had not been confirmed in many ways, we would not have had such an interest in fine particles. So, the commenter queries, aren't these confirmatory studies scientifically influential? To the commenter, the definition given in the supplemental proposal is little more than handwaving and can be used by a variety of parties. Finally, the commenter states that citation of research within an EPA document on which a regulatory decision is based could also be used to define pivotal or influential science, and while imperfect, it is still far more understandable and accountable than any of the definitions in the supplemental proposal.
Response: The definition of influential scientific information in 30.2 of the final rule remains, "scientific information the agency reasonably can determine will have or does have a clear and substantial impact on important public policies or private sector decisions." This definition of influential scientific information is the same definition as in OMB Bulletin for Peer Review. The EPA adopted this definition because it intended the scope to be consistent with how that term has been interpreted and applied in the context of peer review. The intent of prospectively designating studies as influential scientific information is to gain from the additional attention and required peer review. Retrospective designation may be more accurate but provides no benefits to the Agency in terms of ensure a sounder more scientifically robust product.

The interpretation of the term "influential" is consistent with OMB's government-wide information quality guidelines (IQG) and the IQG of the Agency. At EPA, scientific and technical work products that will have or do have a clear and substantial impact on important public policies or private-sector decisions would be considered influential. The following factors (See Chapter 3.2 EPA' Peer Review Handbook, 4[th] Edition) are considered when determining whether a product is likely to be influential:
 Establishes a significant precedent, model or methodology.
 Is likely to have an annual effect on the economy of $100 million or more, or adversely affect in a material way the economy; a sector of the economy; productivity; competition; jobs; the environment; public health or safety; or state, tribal or local governments or communities.
 Addresses significant controversial issues.
 Focuses on significant emerging issues.
 Has significant cross-agency and/or interagency implications.
 Involves a significant investment of agency resources.
 Considers an innovative approach for a previously defined problem, process, or methodology.
 Satisfies a statutory or other legal mandate for peer review.
With regard to pivotal science, the Agency has considered comments and Part 30.2 of the final rule provides an final definition.
5.3.1 Limits to EPA's Application of the Rule
Comment: Commenter (12435) asserts that the supplemental proposal's coverage of influential scientific information essentially removes any limit to what information is applicable per the Agency's determination, so the effect is to allow the rule to apply at the Agency's discretion.
Response: The EPA disagrees with this comment. The definition of influential scientific information in 30.2 is "scientific information the agency reasonably can determine will have or does have a clear and substantial impact on important public policies or private sector decisions." This definition of influential scientific information is the same definition as in OMB Bulletin for Peer Review. The EPA adopted this definition because it intended the scope to be consistent with how that term has been interpreted and applied in the context of peer review. The interpretation of the term "influential" is consistent with OMB's government-wide information quality guidelines (IQG) and the IQG of the Agency. At EPA, scientific and technical work products that will have or do have a clear and substantial impact on important public policies or private-sector decisions would be considered influential. The following factors (See Chapter 3.2 EPA' Peer Review Handbook, 4[th] Edition) are considered when determining whether a product is likely to be influential:
 Establishes a significant precedent, model or methodology.
 Is likely to have an annual effect on the economy of $100 million or more, or adversely affect in a material way the economy; a sector of the economy; productivity; competition; jobs; the environment; public health or safety; or state, tribal or local governments or communities.
 Addresses significant controversial issues.
 Focuses on significant emerging issues.
 Has significant cross-agency and/or interagency implications.
 Involves a significant investment of agency resources.
 Considers an innovative approach for a previously defined problem, process, or methodology.
 Satisfies a statutory or other legal mandate for peer review.

The requirements of this final rule will apply to influential scientific information, unless the influential scientific information is exempted from peer review requirements as described in Section IX of the OMB Peer Review Bulletin.
 Chapter 6: Pivotal Science
The supplemental notice of proposed rulemaking (SNPRM) includes a definition for "pivotal science" at 40 CFR 30.2 that would pertain to the science underlying influential scientific information, which is not regulatory, and made conforming changes to the notice of proposed rulemaking (NPRM) regulatory text at 40 CFR 30.3, 30.5, 30.6 and 30.7. These proposed revisions were made to provide clarity on key terminology used in the regulatory text in the 2018 proposed rulemaking as well as in this supplemental proposal.
The SNPRM proposed "pivotal science" definition follows:
      Pivotal science means the specific scientific studies or analyses that underly influential scientific information.
Several comments were received regarding the definition of "pivotal science" and scope of coverage. In this section these comments are organized under the following sections:
Section 6.1: NPRM Pivotal Regulatory Science
 How Should the Proposed Rule Apply to Dose Response Data and Models Underlying Pivotal Regulatory Science If Those Data and Models were Developed Prior to the Effective Date?
 Could Prospective or Retrospective Application Introduce Bias Regarding the Timeliness and Quality of Scientific Information Available?
 How Does What this Rule Would Require Compare to Requirements of Environmental Statutes? 
 Recommended Changes to the Definition of Pivotal Regulatory Science at §30.2
Section 6.2: SNPRM Pivotal Science
 Need for Additional Clarity/Specificity
 Support for Expanded Scope
The specific comments provided in this Chapter reflect the comment letters that provide arguments supported by detailed rationale, examples, and references/documentation. The Docket ID numbers for these commenters are cited throughout this Chapter. In addition, some general comment letters may also include comments related to comments included and responded to in this Chapter. These comments are summarized in Appendix B. Note that points and issues raised in these general comment letters may also be raised by the more detailed comments summarized in this Chapter. The responses provided in this Chapter adequately address these more general comments.
6.1 NPRM Pivotal Regulatory Science 
6.1.1 How Should the Proposed Rule Apply to Dose Response Data and Models Underlying Pivotal Regulatory Science Developed Prior to the Effective Date?
6.1.1.1 Support Some Application of the Rule to Data and Models Developed Prior to the Effective Date
6.1.1.1.1 Recommend that EPA Revisit Regulatory Decisions Based on Studies Conducted in the Past that have had Major Impacts to Regulated Entities 
Comment: Commenter (3971) suggests that for regulations that have had major impact on the economic well-being of certain corporations or the national economy, the EPA should selectively revisit decisions of the past. The commenter states that many older studies that that the Agency relied upon as a basis for regulation were outdated, poorly conducted, and improperly interpreted. According to the commenter, the Agency often said that although the data were weak, it was all that was available, and suggested that action is warranted. The commenter notes that this bar may have been adequate in the past when we were dealing with major problems, such as a serious ambient air problem in cities, but that the trend toward invoking the Precautionary Principle in times of doubt has sometimes led to this Principle to be misapplied. The commenter contends that more studies and a proper interpretation of them will allow EPA to drift away from policy-driven regulation. 
 Response: EPA disagrees that this rule should require EPA to require the dose-response data underlying pivotal science to be publicly available for past rules. Such retroactive application would be time consuming and impracticable to do for all regulatory actions EPA has promulgated and influential scientific information EPA has developed. This final rule applies prospectively to significant regulatory actions and influential scientific information and has no retrospective effect on existing significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. 
6.1.1.1.2 Recommend that Studies Used as Pivotal Regulatory Science be Made Available Early in the Rulemaking Process
Comment: Commenter (6375) states that, in general, they did not believe that EPA needed to perform a retrospective review of all past final regulations to ensure compliance with the proposed rule when the rule is published. The commenter suggests that, at a minimum, however, the proposed rule should apply to any rule in the Advance Notice of Proposed Rulemaking (ANPRM) or proposed rule phase as of the effective date of this regulation and that enough time be available prior to any statutory or court-ordered deadlines to make public all data and models from dose-response studies as early as possible. The commenter states that this approach should apply to dose-response studies even if they were developed prior to the effective date of this rule. The commenter adds that, if there are instances when EPA cannot apply this rule, then the Agency should identify in the final rule which dose-response studies used as pivotal regulatory science have not been made publicly available and a schedule for making those studies available. 
Response: EPA agrees with the commenter that this rule should not apply retrospectively to past final regulations, but should apply to studies developed prior to the effective date of this rule. This final rule applies prospectively to significant regulatory actions and influential scientific information and has no retrospective effect on either significant regulatory actions or influential scientific information. For future significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. EPA further notes that it sees no need to include the proposed rule stage of significant regulatory actions in the regulatory text because as a practical matter proposed rules must comply with this final rule before being finalized. As a general matter, the EPA does not introduce the studies and analyses it relies on for a rulemaking at the final rule stage. The scientific basis for a rulemaking is provided for public review and comment in the public docket when the proposed rule is issued or, if subsequently added to the docket, through a separate opportunity for public comment. Advance notices of proposed rulemakings are not consistent with the purpose of this rule, given their preliminary nature and frequent focus on soliciting comments on a regulatory issue or approach. 
6.1.1.1.3 Recommend that Rule Apply to All Pivotal Regulatory Science Regardless of When Data and Models Developed
Comment: Commenter (6921) recommends that apply the enhanced transparency requirements included in the Proposed Rule to all pivotal regulatory science regardless of when the data and models were developed. The commenter asserts that, given that EPA generally relies on previously published research, applying the rule prospectively to any new study would undermine any effect from the rulemaking for years. According to the commenter, this would allow EPA to continue to use research that has not been replicated or validated. The commenter notes that, given the importance of EPA regulatory decisions that can impose millions, if not billions, of dollars on the regulated community and the American public, the EPA should make every effort to assure public access to all models and data used to support significant regulatory actions, regardless of when the study was first published or relied upon for regulatory decisions. The commenter asserts that the relative importance of the study to significant regulatory actions, rather than its age, should be the main determinant of the applicability of the enhanced transparency requirements.
Response: EPA agrees with the commenter. The final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data.
6.1.1.2 Oppose Application of the Rule to Data and Models Developed Prior to the Effective Date
Comment: Commenters (1012, 5170, 6373, 6448, 6916) take the position that this rule should not apply to data and models developed prior to the rule's effective date. Specific comments include:
Commenters (1012, 5170) assert that data and research that were conducted prior to a final date of this proposed rule should be exempted as the protocols were already approved. Commenter (5170) states that this approach would be similar to the manner in which pesticides that EPA approved for use in the past with less current research protocols have been and continue to be `grandfathered in', often for widespread use. 
Commenter (6373) states that it would be difficult, if not impossible, to apply the Rule's new requirements related to dose response assumptions and models to studies where the methodology was developed prior to the effective date of the Rule.
Commenter (6916) states that application of the proposed rule to dose response data and models developed prior to the effective date would have the potential effect, on Clean Air Act (CAA) rulemakings, of prohibiting the use of the well-developed dose response information resulting from two landmark studies, the Harvard Six Cities Study and the American Cancer Society Study of Particulate Air Pollution and Mortality, that have been called "the poster children of good-science reforms." The commenter expresses opposition to such an outcome, as it would remove from EPA's rulemaking toolbox the "best available science" on the link between air quality and health and is both unnecessary and unjustified. 
Response: This final rule applies prospectively to significant regulatory actions and influential scientific information and has no retrospective effect on either significant regulatory actions or influential scientific information. For future significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. 
Consistent with existing Agency practice, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information. The requirements of this final rule provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. 
Comment: Commenter (1012) states that data, studies and research that were initiated prior to the final date of this proposed rule should be excepted from this proposed rule as the protocols were already set before the rule was finalized. Similarly, commenter (6448) recommends that the provisions of this rule not apply to data and models currently underway. 
Commenter (6373) requests that EPA state whether the Rule will apply to scientific studies that have already commenced, where human participants have already signed data use agreements or other agreements that place limits on the use of their private data.
Response: For future significant regulatory actions and influential scientific information, this final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. Consistent with existing Agency practice, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information. The requirements of this final rule provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. 
6.1.2 Could Prospective or Retrospective Application Introduce Bias Regarding the Timeliness and Quality of Scientific Information Available?
6.1.2.1 Conditional Retrospective Application to Past Studies Used as Pivotal Regulatory Science
Comment: Commenter (6141) states that, in the event that EPA elects to apply the transparency rules retrospectively, the EPA should consider developing exemptions for those past studies for which it is not possible to release a dataset publicly for legitimate reasons pertaining to the use, for example, of human subject data. 
Response: EPA notes that this final rule applies prospectively to future significant regulatory actions and influential scientific information and has no retrospective effect on either significant regulatory actions or influential scientific information. That said, EPA agrees with the commenter that additional reasons for exemption from this rule are merited, and has amended 40 CFR 30.7 (proposed 40 CFR 30.9) to read, 
   "The Administrator may grant an exemption to this part for a study on a case-by-case basis if he or she determines that greater consideration is warranted because:
       technological or other barriers render sharing of the dose-response data infeasible; 
       the development of the dose-response data was completed or updated before the effective date of the final rule; 
       making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, confidential business information, or national security;
       a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis; or
       the factors used in determining the consideration to afford to the pivotal science indicate full consideration is justified."
Comment: Commenter (6141) states that instead of imposing an absolute ban on the use of past studies in all cases, EPA should develop reasonable rules and guidelines on when EPA may consider these past studies in future rulemakings. 
Response: EPA clarifies that the rule does not impose a ban on the use of any studies regardless of when they took place. Under the final rule no studies are categorically excluded from consideration, even studies where the underlying dose-response data are not available for independent validation. EPA will continue to assess the full body of science to evaluate the quality and transparency of available science for significant regulatory actions and influential scientific information using the factors outlined in § 30.5(b) of the final rule. The EPA will also consider the degree to which pivotal science is available and give greater consideration to pivotal science for which the underlying dose-response data are available in a manner sufficient for independent validation.
Comment: Commenter (6360) recommends that, if EPA elects a retrospective application, EPA should develop a process by which stakeholders could petition EPA based on objective criteria for the retrospective application of this rule to an existing administrative record for a final agency action. The commenter recommends that the criteria require a demonstration that the lack of transparency in the past undermines confidence in policies that significantly and adversely impact today's consumers, economy, or environment. Commenter states that this process could draw from the existing petition for rulemaking processes under the Administrative Procedures Act (APA) and requests for data correction under the Data Quality Act.
Commenter (6143) suggests that requirements apply on a case-by-case basis to retrospective assessment of research information to the extent that such actions are based or depend on research information.
 Response: This final rule applies prospectively to significant regulatory actions and influential scientific information and has no retrospective effect on either significant regulatory actions or influential scientific information. Such retroactive application would be time consuming and impracticable to do for all significant regulatory actions EPA has promulgated and influential scientific information EPA has developed. For future significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. 
6.1.2.2 Support for Retrospective Application to Past Studies Used as Pivotal Regulatory Science
Comment: Commenter (6362) states that the rule should generally apply prospectively to EPA decision making. The commenter also states that, in cases where EPA has developed analytical tools and models, e.g., ECOSAR, in the past that incorporate dose response data, it may be valuable to apply this rule retrospectively. The commenter notes that, in other cases, such as Integrated Risk Information System (IRIS) assessments, where the Agency has yet to articulate a periodic review schedule for updating scientific assessments dating back 10-20 years or longer, the EPA should develop appropriate mechanisms for application of the rule. 
Response: EPA agrees with the commenter that this rule should apply prospectively to EPA decision making. This final rule applies prospectively to significant regulatory actions and influential scientific information and has no retrospective effect on either significant regulatory actions or influential scientific information. For future significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. In the case of tools and models (e.g. ECOSAR) or IRIS assessments, this rule will only apply to those if they are used as pivotal science in future significant regulatory actions and influential scientific information.
Comment: Commenter (6449) states that in many cases the EPA relies upon toxicological assessments that have already been published by the Agency. The commenter recommends that the Rule make clear that for major or significant regulatory actions, the requirements of this rule will apply at the regulatory phase regardless.
Response: EPA clarifies that for future significant regulatory actions and influential scientific information, this final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. 
6.1.2.3 Oppose Retrospective Application
6.1.2.3.1 Regulatory Uncertainty
Comment: Commenters (6116, 6125, 6141, 6360, 6866, 6905, 6918) suggest that the EPA require only prospective application of the Proposed Rule and express concern that the Proposed Rule could increase regulatory uncertainty for industry.
Commenter (6125) states that even though limited by time and resources, our assessment shows that, if the proposed rule were to apply retroactively with few or no exceptions/exemptions, most of the EPA cancer classifications and reference values would be vacated. The commenter suggests that, given the magnitude of the issue, EPA must address the potential damages to public health and the environment before any final action. The commenter adds that, to issue a valid rule, EPA would have to analyze this issue on its own, including what studies the proposal will exclude, present the results for public comment, and explain why the benefits of the per se rule justified the public health sacrifice and regulatory confusion that it would cause. 
Commenters (6141, 6360) recommend that the EPA require only prospective application of the reforms. According to the commenters, a forward-looking process that does not disrupt any previously published proposed or final rule strikes a reasonable balance between EPA's positive efforts to increase transparency and the need for certainty, finality, and reasonable implementation burdens. Commenter (6360) states that reopening existing administrative records of final agency actions could result in significant unintended consequences that could undermine EPA's ability to implement the proposed rule. 
Commenter (6866) states that retrospective application would put in doubt all current regulations  -  rules that provide business with certainty about requirements and that protect the public  -  even though there is no reason to assume they are faulty. The commenter expresses that the EPA can always reopen rules on a case-by-case basis if there is reason to do so, and some environmental statutes require periodic reviews of the latest science. 
Commenter (6905) understands that this rule would only be used prospectively and urges EPA to retain that limitation. The commenter notes that the Proposed Rule, if implemented, would present increased litigation risk for states who rely on existing rules and standards set by the EPA and would increase regulatory uncertainty.
Commenter (6918) supports a forward-looking process in which EPA applies its new regulatory procedures to new proposed rules but does not disturb already-issued proposed or final rules. According to the commenter, a forward-looking process that does not disrupt any previously issued proposed or final rule strikes a reasonable balance between EPA's efforts to increase transparency and the need for certainty, finality, and reasonable implementation burdens.
Response: EPA agrees with the commenters, and this final rule applies prospectively to significant regulatory actions and influential scientific information and has no retrospective effect on either significant regulatory actions or influential scientific information. For future significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. Since this rule applies only prospectively to significant regulatory actions it is unnecessary to assess potential damages associated with a retrospective application of the requirements of this rule.
Comment: Commenter (6117) does not recommend any retroactive application of this regulation. The commenter states that the EPA has already embarked on a well-thought-out plan to require and promote research transparency, as described in the EPA's 2016 Plan to Increase Access to Results of EPA-Funded Scientific Research ("Plan"). The commenter provides that, although the preamble to the proposed regulation cites and "applies concepts and lessons learned" from the Plan, there are some concerning differences. The commenter states that the Plan indicates that it does not apply to "scientific research data collected before its implementation." The commenter states that, in contrast, the preamble and the proposed regulation suggest that the regulation may apply retrospectively.
Commenter (6141) suggests that EPA focus the new rules for scientific transparency on future studies rather than on past studies that are already designed and published with terms that may not fully comply with transparency requirements of the Proposed Rule. The commenter states that it should be noted that the epidemiologic science community has been making significant efforts in recent years to make data available where possible and to develop studies based on publicly available data where appropriate. The commenter contends that the Proposal would strengthen and expand upon this trend by further encouraging researchers to make increased efforts toward transparency so that the results of their research are eligible to be considered in future EPA rulemakings or other regulatory proceedings. 
Response: EPA disagrees with the commenters that all past studies should be exempt from the requirements of this rule. This final rule applies prospectively to significant regulatory actions or influential scientific information and has no retrospective effect on either significant regulatory actions or influential scientific information, but applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. Under the final rule no studies are categorically excluded from consideration, even studies where the underlying dose-response data are not available for independent validation. EPA will continue to assess the full body of science to evaluate the quality and transparency of available science for significant regulatory actions and influential scientific information. The EPA will also consider the degree to which pivotal science is available and give greater consideration to pivotal science for which the underlying dose-response data are available in a manner sufficient for independent validation.
6.1.2.3.2 Legal Considerations
Comment: Commenters (6117, 6133, 6137) assert that it is well established that agency rules cannot be applied retroactively unless Congress expressly grants the agency that power. The commenters cite Bowen v. Georgetown Univ. Hospital, 488 U.S. 204, 208 (1988). Commenter (6137) also cites Sierra Club v. Whitman, 285 F.3d 63, 68 (D.C. Cir. 2002) (referring to "unusual ability to implement rules retroactively"). Commenters (6133, 6137) contend that EPA has not identified any such congressional intention in any of the statutes at issue, and thus retroactive application of the Rule would be unlawful.
Commenter (6117) states that there is no specific grant of power to apply this proposed regulation retroactively. The commenter provides that the two bills upon which this proposed regulation is modeled, the Honest and Open New EPA Science Treatment (HONEST) Act of 2017 (H.R. 1430  -  115th Congress, 2017-2018), and the Secret Science Reform Act of 2015 (H.R.1030  -  114th Congress 2015-2016), were not enacted.
Commenters (6133, 6137) assert that the Supreme Court strongly disfavors retroactive application of rules and cite the following Court decision:
      "Retroactivity is not favored in the law. Thus, congressional enactments and administrative rules will not be construed to have retroactive effect unless their language requires this result. [] By the same principle, a statutory grant of legislative rulemaking authority will not, as a general matter, be understood to encompass the power to promulgate retroactive rules unless that power is conveyed by Congress in express terms. [] Even where some substantial justification for retroactive rulemaking is presented, courts should be reluctant to find such authority absent an express statutory grant." Bowen v. Georgetown Univ. Hosp., 488 U.S. 204, 208 - 09 (1988) (internal citations omitted).
Commenter (6133) states that the Proposal ignores an entire body of case law that has considered and roundly rejected both retroactivity in rulemakings and limiting data that underlies rulemakings to "publicly available data." That commenter adds that, in so doing, the Proposal is arbitrary and capricious, and should be rejected. The commenter notes that the suggestion in the Proposal, for example, that EPA may invoke the Proposal's approach to review all prior health and scientific studies underlying the national ambient air quality standards (NAAQS) is illegitimate, arbitrary and capricious, and contrary to caselaw. The commenter provides that Bowen and its progeny do not permit agency rules to have retroactive effect to disallow health studies and regulatory science generated prior to, or relied upon by EPA prior to, adoption of any final rule based on the Proposal. The commenter asserts that this caselaw does not entertain any such exception and accepting any such exception for these circumstances would circumvent the holdings and reasoning of this case law.
Response: EPA agrees with the commenters that this rule should not apply retroactively. This final rule applies prospectively to significant regulatory actions and influential scientific information and has no retrospective effect on either significant regulatory actions or influential scientific information. For future significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. EPA also notes that despite proposing to promulgate this rule under the substantive environmental statues EPA administers, EPA is finalizing the rule solely under its authority to promulgate procedural regulations governing its internal affairs (or "housekeeping authority").
Comment: Commenter (6133) states that, despite its assertion that the rulemaking is "intended" to apply prospectively, the Proposal contemplates prohibiting EPA -- or will prohibit EPA --  from relying on studies generated prior to rulemakings that fail to meet the Proposal's ill-defined criteria for "publicly available data." According to the commenter, this approach is arbitrary and capricious, runs counter to the specific language of many statutes the agency is tasked with administering, and would destroy the agency's ability to promulgate health-based standards to protect the American public using the best available science. 
Commenter (6137) states that the statutes upon which EPA relies do not support retroactive application. The commenter notes, for example, the CAA does not allow -- and, as the D.C. Circuit has held, certainly does not require, e.g., American Trucking, 283 F.3d at 372 -- applying the so-called transparency provisions of the Proposed Rule at all in rules subject to the procedural requirements of CAA§ 307(d). The commenter provides that such rules include NAAQS. The commenter states that, far from suggesting that EPA could lawfully reopen long-settled NAAQS to apply a new and novel standard of review, the CAA requires EPA to review and revise air quality criteria and NAAQS at least every five years and to "promulgate such new standards as may be appropriate." 42 U.S.C. § 7409(d)(1). The commenter asserts that this carefully chosen language confirms that Congress did not intend for EPA to apply rules like the Proposed Rule to undo existing NAAQS, but instead intended for regular reviews of scientific information to result in new NAAQS.
Commenter (6137) states that, even if EPA had statutory authority to apply the Proposed Rule retroactively, it could not do so rationally. The commenter cites Sierra Club, 285 F.3d at 68 (if EPA had authority to implement a rule retroactively, "retroactivity must be `reasonable,'"). According to the commenter, retroactive application of the Proposed Rule would undo well-settled rules in a tremendously unfair and irrational way and would lead to widespread confusion about countless environmental and public health policies and protections.
Response: This final rule applies prospectively to significant regulatory actions and influential scientific information and has no retrospective effect on either significant regulatory actions or influential scientific information. Such retroactive application would be time consuming and impracticable to do for all regulatory actions EPA has promulgated and influential scientific information EPA has developed. For future significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. EPA also notes that this final rule is relying exclusively on the Agency's housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. Some of these subsequent rulemakings will be substantive rules issued under the associated environmental statutes and will be subject to judicial review. By contrast, in this action, the EPA finalizes a rule of internal agency procedures, including how the Agency will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information for independent validation.
6.1.2.3.3 Retrospective Application May Not Be Practical
Comment: Commenters (2171, 2426, 6126, 6866, 6895) express that retrospective application of the proposed rule may not be feasible or practicable within a reasonable timeframe, both because of the volume of significant environmental rules already in place and because of the expertise that would be needed to reanalyze rules and supporting studies underpinning those rules. Commenter (2426) states that there was insufficient time and expertise to reanalyze the technical and legal aspects of these rules while maintaining oversight of ongoing agency business. 
Commenter (6866) states that a retroactive application would be practically impossible to achieve in a timely manner and would impose unnecessary burdens on the agency while achieving little. The commenter provides that data access and sharing plans developed by National Institutes of Health (NIH), NSF, and EPA for their own funded research have all been prospective.
Commenter (6895) states that applying the proposed rule retroactively to health studies, potentially initiated decades ago, would provide no additional protection and would be difficult and time consuming, if not impossible.
Commenter (6141) asserts that there are many reasons why it is not feasible or practicable for EPA to require retroactive application of its proposed transparency policy in the case of existing EPA regulations. 
Commenter (6126) states that EPA will not likely have the resources or capability to assess the entire suite of regulations and the evidence-base retrospectively. The commenter notes, for example, that the EPA is technically required to evaluate every regulation promulgated under the Resource Conservation and Recovery Act (RCRA) on a triennial basis; recent research suggests that statutory requirement has been largely unfulfilled due to a lack of resources to comply with the decades-old mandate. The commenter states that, in the absence of a major influx of new resources, EPA would be unlikely to engage in such meaningful and transparent reviews while also complying with statutory obligations.
Response: This final rule applies prospectively to significant regulatory actions and influential scientific information and has no retrospective effect on either significant regulatory actions or influential scientific information. Such retroactive application would be time consuming and impracticable to do for all regulatory actions EPA has promulgated and influential scientific information EPA has developed. For future significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. EPA also notes that this final rule does not require that EPA, a member of the public or other entity to independently validate a study through reanalysis before it can be considered to be pivotal science. 
Comment: Commenters (1449, 6141, 6907) state that there are situations where public access to data and analytic methods used in past published studies may infringe on legitimate confidentiality and/or privacy interests, and where exceptions from the requirement to provide full public access is justified and therefore appropriate. 
Commenters (6141, 6191) state that the cost and effort of redoing or updating these past studies also may be too onerous and costly in the case of large and complex existing data sets used in many cases. Commenter (6191) states that enforcement of data sharing for previously completed projects would introduce undue financial burden on principal investigators and may be largely impossible. The commenter provides that Institutional Review Board (IRB)-approved studies include provisions regarding disposal of data following study completion. The commenter states that researchers abiding by these provisions would have securely deleted and disposed of all raw data in the allotted time following project completion.
Commenter (6119) states that the requirements laid out in the rulemaking may not be practicable and/or feasible in the development of toxicity values, which rely heavily on epidemiology studies, case reports and clinical control settings. The commenter asserts that epidemiology studies and case reports are not reproducible as they come from uncontrolled public environmental settings. The commenter explains that uncontrolled public environmental settings are real life exposure scenarios and ultimately what "pivotal regulatory science" is trying to protect. The commenter adds that trying to reproduce the conditions in which a release is occurring or according to conditions 50 years ago is not practicable nor feasible. The commenter further adds that clinical controlled settings can also contain confidential data that cannot always be released for public viewing. 
Commenter (6909) states that this proposed rule could have the devastating effect of undermining the scientific bases underpinning many of EPA's key regulatory regimes if it is applied retroactively  -  as the language of the proposed rule indicates it may be. The commenter provides that many older epidemiological studies that have provided support for numerous of EPA's key CAA and Clean Water Act (CWA) programs utilized confidential health data that may not be possible to make public at this point. 
Response: This final rule applies prospectively to significant regulatory actions and influential scientific information and has no retrospective effect on either significant regulatory actions or influential scientific information. Such retroactive application would be time consuming and impracticable to do for all regulatory actions EPA has promulgated and influential scientific information EPA has developed. For future significant regulatory actions and influential scientific information, this final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. Under the final rule no studies are categorically excluded from consideration, even studies where the underlying dose-response data are not available for independent validation. EPA will continue to assess the full body of science to evaluate the quality and transparency of available science for significant regulatory actions and influential scientific information. The EPA will also consider the degree to which pivotal science is available and give greater consideration to pivotal science for which the underlying dose-response data are available in a manner sufficient for independent validation.
 Further, the EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
6.1.2.3.4 Retrospective Application is Complicated
Comment: Commenter (6362) states that retrospective application of any regulation (and its underlying scientific evaluations) is rife with complication, confusion, and significant ambiguity for EPA and stakeholders alike. Commenter states that, for example, it is unclear which administrative NAAQS records would be covered by the proposal and how far back it would apply.
Response: This final rule applies prospectively to significant regulatory actions and influential scientific information and has no retrospective effect on either significant regulatory actions or influential scientific information. Such retroactive application would be time consuming and impracticable to do for all regulatory actions EPA has promulgated and influential scientific information EPA has developed. For future significant regulatory actions (e.g. a future NAAQS regulation) and influential scientific information, this final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data.
6.1.2.3.5 Retrospective Application Would Undermine Existing Public Health Protections
Comment: Commenters (1012, 2171, 6191, 6895, 6907, 6909, 8281-PH6) oppose retroactive application because it could undermine existing public health protections.
Commenter (1012) states that retroactive application of this proposed rule would open a tremendously convoluted process that could undermine existing public health protections. Commenter (2171) states that this rule must not be applied retrospectively; such application would only serve to create confusion with the intent of undermining existing public health regulations. Commenter (6191) states that the retroactive imposition of this rule would invalidate well-conducted research to the detriment of public health. Commenter (6895) states that retroactive application of the Proposed Rule would be detrimental to public health. Commenter (8281-PH6) states that retroactive application would severely undermine existing public health and environmental protections that keep the public safe and healthy. Commenter (6907) expresses concern that, under the proposed rule, if applied retrospectively, older studies on pesticides may be not be considered. The commenter notes that this is of concern for older pesticides which are now unregistered where there are older studies that indicate that these pesticides pose harm to humans or the environment. 
Commenter (6909) states that, if EPA applies the proposed rule to studies retroactively, the core programs of the CAA and CWA may suddenly be left without any scientific basis that the agency will recognize. The commenter expresses that the prospect of the agency promulgating a rule that it can use to selectively hollow out core programs that not only protect public health but provide huge economic benefits (in the form of prevented lost workdays and hospital admissions, for example) is extremely troubling.
Response: This final rule applies prospectively to significant regulatory actions and influential scientific information and has no retrospective effect on either significant regulatory actions or influential scientific information. Such retroactive application would be time consuming and impracticable to do for all regulatory actions EPA has promulgated and influential scientific information EPA has developed. For future significant regulatory actions and influential scientific information, this final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. Under the final rule no studies are categorically excluded from consideration, even studies where the underlying dose-response data are not available for independent validation. EPA will continue to assess the full body of science to evaluate the quality and transparency of available science for significant regulatory actions and influential scientific information. The EPA will also consider the degree to which pivotal science is available and give greater consideration to pivotal science for which the underlying dose-response data are available in a manner sufficient for independent validation.
6.1.2.4 Bias in the Quality of Scientific Information Available
Comment: Commenters (0671, 4590, 6126, 6137, 6152, 6191, 6895) express concern that retroactive application could result in the exclusion of studies that currently represent the "best available science" and, thus, reduce the quality of scientific information available for EPA's regulatory decision-making. 
Commenter (6137, 6191) states that retrospective application of the provision would directly introduce bias regarding the quality of the scientific information available, as applying the rule would exclude most data/scientific information from consideration.
Commenter (4590) expresses concern that an incomplete pool of scientific evidence could skew regulatory decision-making toward studies that are less rigorous and trustworthy. The commenter states that, often, research that provides real insight into the impact of pollution on individuals and communities requires precisely the types of data most at risk of exclusion from EPA consideration under this rule. 
Commenter (0671) states that the proposed regulatory change should not cover past or future dose response data or models. The commenter asserts that the previous rules were created using the best available science when they were promulgated. According to the commenter, science doesn't disappear because the EPA now wants to apply an unreasonable standard of evidence.
Commenter (6152) states that it appears that EPA may prohibit the use of older scientific studies that were developed before this rule was proposed and therefore may not conform to the requirements in this proposal. The commenter asserts that this arbitrary exclusion and potential retroactive culling of studies would surely undermine the quality of science used in rulemakings. The commenter states that this would impact worker epidemiology studies that often take years and sometimes decades to complete.
Commenter (6895) opposes the rule as an impediment to examination of the current state of the science and strongly opposes any retroactive application of the rule to an existing record. The commenter states that such application will arbitrarily introduce bias into previous decisions to the detriment of public and environmental health. The commenter states that applying the proposed rule retroactively would de facto preclude EPA's reliance on previously validated scientific findings from published, peer-reviewed research.
Commenter (6373) states that both the prospective and retrospective application of the proposed rule requirements will impact the quality of the scientific information that would be available to EPA. The commenter asserts that the Rule would require researchers to spend significant extra time and funds preparing public-use data, which would likely compromise EPA's ability to review and incorporate the most up-to-date relevant epidemiological and toxicological evidence of the health effects of pollutants into its regulations.
Commenter (1012) states that retroactive or prospective application of this proposed rule would ultimately bias the quality of the scientific information available by only allowing studies that support industry positions, which could bias the EPA towards that position. 
Commenters (0569, 4590) express concern that the proposed language at subparts §30.2 and §30.9 could impact the quality of the science available because it could allow the EPA to selectively choose studies to meet its agenda. Commenter states that subpart §30.2 contains an a priori criterion of "pivotal regulatory science" while §30.9 gives the EPA Administrator discretion to issue exemptions from the policy on a case-by-case basis. Commenter states that these sections are vague on how and by what measures determinations would be made, opening the door for drawing upon studies driven by unknown interests or by political considerations rather than, as EPA's Mission Statement notes, "...the best available scientific information." Commenter (4590) states that empowering the Administrator with the discretion to implement case-by-case exemptions risks arbitrary policymaking, untethered from clearly established principles regarding when it is appropriate to rely on non-publicly available data as the best available scientific evidence.
Commenter (6362) states that bias should not be presumed. According to the commenter, if EPA uses a weight-of-the-evidence approach (as required under TSCA) and EPA has concerns about bias having been introduced, it can assess this using a sensitivity analysis by evaluating the impact of each study and/or model on the overall outcome of the analysis. The commenter asserts that bias should not be inferred if newer, more scientifically robust studies based on modern, up to date knowledge of biology and dose response are determined to be of better quality, relevance, and evidentiary value. 
Commenter (6375) states that, in general, if the right processes and systems are in place to manage the public sharing of dose-response studies, then no unintended bias will occur for any prospective application of the rule. The commenter recognizes the challenge this may present and recommends that EPA consider a process ensuring that new dose-response studies are identified and assessed when they are completed and not wait until the rulemaking process begins. The commenter states that the NAAQS process is an example where EPA could develop a continual review process. The commenter asserts that continual assessment of new dose-response studies to determine if they have the potential to be pivotal regulatory science is key to ensuring that this rule does not cause bias or otherwise slow the rulemaking process. According to the commenter, this will allow enough time for independent verification to be completed prior to the rulemaking process, thus minimizing unnecessary delays. 
Commenter (6449) states that there may be bias against the following: 
 high quality studies published in the past where underlying data is not available;
 studies published outside the U.S; 
 studies performed by authors or in a journal not informed by the rule; and
 proprietary models.
Commenter (6375) states that, regarding retrospective application, if EPA does not ensure that data and models from past dose-response studies are made publicly available when reviewing a previously finalized rule or creating a new rule, EPA could be giving the same weight to studies that may not have the same level of quality or veracity. The commenter notes that this could bias the rulemaking decisions toward results which do not represent the best available science. The commenter suggest that the Agency should release underlying data from older studies including methodologies so that strength and weaknesses of these data sets can be identified and that bias toward lower quality studies can be identified and constrained going forward. 
Commenter (9227) notes that EPA's proposal would regulate not the scientists, but EPA itself and that the EPA would "ensure" that data and models underlying scientific studies "pivotal" to regulatory action are publicly available simply by barring EPA's own use in regulatory actions of any studies for which the authors do not make the data and models publicly available. The commenter asserts that the "mechanism" mentioned in the preamble is not technical assistance or funding to encourage greater availability of data; it is simply the pressure generated by EPA's refusal to consider the results of a study if the authors do not release publicly the underlying data and models. The commenter states that most of the significant barriers to release data and models are not a matter of the researcher's preference, but rather take the form of legal and ethical constraints, significant costs, large time investments, or the loss of proprietary data critical to a researcher's future career prospects. The commenter suggests that while it seems plausible that having their research applied in a regulatory context would be viewed as an incentive by some, or perhaps many, researchers, there is no reason to believe that such an incentive would be sufficient to overcome the significant barriers to public release of data where those barriers exist. The commenter contends that the party most likely to be incentivized by EPA's proposed requirements is the regulated community which has vested financial interests in regulatory actions the agency may take -- a situation that almost certainly will lead to significant bias and conflicts of interests in the scientific evidence that the agency considers.
Response: This final rule applies prospectively to significant regulatory actions and influential scientific information and has no retrospective effect on either significant regulatory actions or influential scientific information. Such retroactive application would be time consuming and impracticable to do for all regulatory actions EPA has promulgated and influential scientific information EPA has developed. For future significant regulatory actions and influential scientific information, this final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data.
EPA disagrees with the commenters that this rule will result in the exclusion of studies from consideration for EPA significant regulatory actions and influential scientific information. 
Under the final rule no studies are categorically excluded from consideration, even studies where the underlying dose-response data are not available for independent validation. EPA will continue to assess the full body of science to evaluate the quality and transparency of available science for significant regulatory actions and influential scientific information. The EPA will also consider the degree to which pivotal science is available and give greater consideration to pivotal science for which the underlying dose-response data are available in a manner sufficient for independent validation.
EPA disagrees with the commenter that the Administrator's exemption outlined in proposed 40 CFR 30.9 will result in arbitrary policymaking. The EPA believes that retaining the Administrator's exemption provision because there are conditions under which compliance with the requirements in 40 CFR part 30 might be impracticable. For example, the underlying dose-response data for some studies, particularly older studies, may not be readily publicly available because of the technological barriers to data sharing (e.g., differences in data storage devices or data retention practices) that existed when they were developed, such as the age of the study. As a result, the EPA is finalizing the Administrator's exemption provision as proposed in the 2020 SNPRM, with additional conditions described here. Due to other changes in the final rule, the Administrator's exemption provision, which was previously in 40 CFR 30.9 in the 2018 proposed rule and the 2020 SNPRM, is now 40 CFR 30.7 in the final rule. To ensure that the Administrator's decision is appropriately transparent, in the final rule the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. This documentation would typically be provided as part of the proposed rulemaking given this will be part of the decision of what is the pivotal science for the rule. Regardless, the final rulemaking will provide clear documentation. The EPA is also including criteria that may be used by the Administrator when he or she is determining whether greater consideration should be afforded to pivotal science for which the underlying dose-response data are not available in a manner sufficient for independent validation. As a result, the Administrator may determine that greater consideration is warranted when:
 Technological or other barriers render sharing of the dose-response data infeasible; 
 The development of the dose-response data was completed or updated before the effective date of the final rule; 
 Making the dose-response data would conflict with laws and regulations governing privacy, confidentiality, CBI, or national security; 
 Third-party independent validation of the study's underlying dose-response data through reanalysis has been conducted; or 
 The factors in 40 CFR 30.5 used in determining the consideration to afford to the pivotal science indicate full consideration is justified. 
To assist the Administrator in determining whether to grant an exemption, the EPA program or Region responsible for the significant regulatory action or influential scientific information and public commenters can provide input when the Administrator is considering an exemption. The EPA will document the rationale for the Administrator's exemption in the significant regulatory action or influential scientific information. The EPA is confident that the above criteria provide sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
Finally, EPA disagrees with the commenters that by taking into consideration the availability of data and models underlying pivotal science, EPA will be biased against considering the best available science. The EPA does not agree that its approach will be biased. As discussed in the preamble, the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. When characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect, the EPA will identify pivotal science and give greater consideration to pivotal science for which the underlying dose-response data are available in a manner sufficient for independent validation. Including this review of dose-response data availability for pivotal science is critical to the EPA's progress toward increased transparency and providing increased opportunity for scientific reanalysis and review by independent third parties. This approach will result in significant regulatory actions and influential scientific information that are based on the highest quality, most relevant studies and ensure adequate human health and environmental protections, while maximizing transparency and maintaining consistency with the statutes the EPA administers. 
6.1.2.5 Bias in the Timeliness of Scientific Information Available
6.1.2.5.1 Analysis Paralysis and Data Dredging
Comment: Commenter (6446) expresses concern that there are no criteria or ad hoc methods that would prevent stalling vital decisions to protect public health and the environment by "analysis paralysis." The commenter contends that the proposed rule, if adopted, would inevitably delay setting protective standards through prolonged evaluation of the sufficiency of the public availability of research data and methods, rather than evaluation of the actual quality and import of scientific research. 
Commenter (6122) states that the decision to delay action in favor of more refined data or models is a decision in-of-itself, and often has real consequences for ecosystems and endangered species. The commenter contends that forcing EPA to consider virtually every possible model in any decision will not lead to better decision-making, but instead will lead to regulatory paralysis, and EPA will defer taking life-saving steps to address pollution. 
Response: The EPA disagrees with the commenters that requirements of this rule will result in any meaningful delay in promulgating regulations. While this rule does require the Agency to evaluate the availability of data and models for pivotal science, the incremental burden to the Agency to carry out these requirements is expected to be small given (1) the extensive scientific review EPA already conducts regularly and (2) that the requirement is limited to pivotal science (i.e., typically a small, though highly important, subset of all the studies the EPA would review). See section III.J of the rule preamble for further explanation.
Comment: Commenter (6897) states that the proposed rule would encourage data dredging and deconstruction that are the antithesis of good science. The commenter notes that epidemiological studies consider a huge number of variables and can relate them to a variety of outcomes in order to determine one significant association, such as the impact of poor air quality and asthma. The commenter explains that data dredging involves systematically retesting data sets such as those underlying epidemiologic studies through multiple models and statistical tests in order to produce desired outcomes. The commenter states that the proposed rule would allow regulated industries to request data and subject it to endless dredging to slow or entirely forestall regulation that would hurt their economic interests. The commenter adds that, similarly, data "deconstruction" is a term that has been used to describe a variety of sophisticated attacks on data, including reconsidering raw data through unreliable models and confounding data to obscure findings of harm. By explicitly requiring that the EPA consider a vast array of alternative models, which may be prepared by outside stakeholders, the commenter states that the proposed rule has the potential to buoy well-financed stakeholders who benefit from undermining good science and stalling sound policy by enforcing their ability to endlessly test and manipulate data.
Response: EPA disagrees with the commenter. Because of the potential impact of final significant regulatory actions, independent validation of studies that are pivotal to the EPA's final significant regulatory actions and influential scientific information will increase public confidence in the findings and conclusions presented in those studies. As noted in preamble to the final rule, the EPA's Science Advisory Board (SAB) noted that it "recognizes the importance of this rule and its purpose, establishing transparency of the influential scientific information used for significant regulations and enhancing public access to scientific data and analytical methods to help ensure scientific integrity, consistency and robust analysis. Strengthening transparency by improving access to data can lead to an increase in the quantity and the quality of evidence that informs important regulatory and policy decisions. The scientific community is moving toward adopting the precept of sharing accurate data and information to increase credibility, high-quality outcomes and public confidence in science. The SAB supports the adoption of this precept." The scientific community has also embraced greater data transparency, as evidenced by data sharing and availability requirements for many high-impact journals and the emergence of organizations, such as the Center for Open Science, and international initiatives like Findable, Accessible, Interoperable, and Reusable (FAIR) data principles; Facilitate Open Science Training in European Research (FOSTER); and Guidelines for Transparency and Openness Promotion (TOP) in Journal Policies and Practices that incentivize greater transparency in research.,
6.1.2.5.2 Barrier to Timely EPA Rulemaking
Comment: Commenter (1012) states that retroactive or prospective application of this proposed rule would ultimately bias the timeliness of the scientific information available by introducing procedural hurdles to the Agency's review process which could bias the EPA towards industry positions. 
Commenter (6373) states that the prospective and retrospective application of these requirements will impact the timeliness of the scientific information that would be available to EPA. The commenter states that, even if it were feasible to recreate past studies so that they use a new data set that may be made public, it would be neither time-efficient nor cost-effective to repeat these studies merely to create a public data set for an "independent validation" in order that they be considered in the regulatory process.
Commenter (6119) states that it is their assessment that the proposal could further delay development and update of toxicity values or reference values published in the Comprehensive Environmental Response, Compensation and Liability Act (CERCLA), Toxic Substances Control Act (TSCA), CWA, Safe Drinking Water Act (SDWA) and other federal acts authorized by EPA. The commenter notes that states, EPA and the public are already struggling to respond to emerging contaminants for which toxicity values and risk-based standards have yet to be developed. There are currently more than 129 million organic and inorganic substances identified with a chemical abstracts service number in the open scientific literature. Yet, the commenter notes that only a handful have been studied and even fewer with "dose response data and models" (reference values (reference dose and/ or reference concentrations)) used for "pivotal regulatory science" as noted in the proposed rule. In Alaska, for example, the commenter states that a cleanup level for sulfolane has yet to be set, awaiting a National Toxicology Program (NTP) study on the public health impacts of long-term exposure to the environmental contaminant.
Commenter (6119) recommends that, if finalized, the rule should be applied only going forward and not retroactively due to the disruption that would cause. The commenter states that making it retroactive would lead to a lengthy and expensive hiatus in environmental management (protection and permitting) while the past decisions are explored, and the public provided notice.
Commenter (1012) states that retroactive application of this proposed rule would cause needless delays in the already beleaguered NAAQS review process. Commenter (6125) asserts that the proposed rule would likely impair significantly the Agency's ability to meet regulatory deadlines for the NAAQS or other programs.
Commenter (1012) states that retroactive application of this proposed rule would open a tremendously convoluted process that could undermine existing public health protections and set the Agency up for years of legal challenges. Commenter (5170) states that retroactive application of this proposed rule would make the Agency vulnerable to an onslaught of legal challenges while public health is held in limbo.
Commenter (6116) states that this proposal would slow the EPA's work to a near standstill or leave it open to criticism in terms of credibility, leaving states with data gaps that they cannot fill. The commenter adds that delays and lack of data due to convoluted, ill-defined requirements to achieve "independent validation" will cause harm to tribal governments who need quick, reliable information to address their environmental protection needs. 
Commenter (6373) states that the Rule would require researchers to spend significant extra time and funds preparing public-use data, which would likely cause critical delay in the publication of findings and compromise EPA's ability to review and incorporate the most up-to-date relevant epidemiological and toxicological evidence of the health effects of pollutants into its regulations. 
Response: EPA notes that this final rule applies prospectively to significant regulatory actions and influential scientific information and has no retrospective effect on either significant regulatory actions or influential scientific information. Such retroactive application would be time consuming and impracticable to do for all regulatory actions EPA has promulgated and influential scientific information EPA has developed. For future significant regulatory actions and influential scientific information, this final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data.
Further, EPA disagrees with the commenters that requirements of this rule will result in any meaningful delay in promulgating regulations. While this rule does require the Agency to evaluate the availability of data and models for pivotal science, the incremental burden to the Agency to carry out these requirements is expected to be small given (1) the extensive scientific review EPA already conducts regularly and (2) that the requirement is limited to pivotal science (i.e., typically a small, though highly important, subset of all the studies the EPA would review). See section III.J of the preamble for further explanation.
Comment: Commenter (6119) states that the proposal could further delay development and update of toxicity values or reference values published in the CERCLA, TSCA, CWA, SDWA and other federal acts authorized by EPA. The commenter asserts that states, EPA and the public are already struggling to respond to emerging contaminants for which toxicity values and risk-based standards have yet to be developed. The commenter provides that, while there are currently more than 129 million organic and inorganic substances identified with a chemical abstracts service number in the open scientific literature, only a handful have been studied and even fewer with "dose response data and models" used for "pivotal regulatory science" as noted in the proposed rule. According to the commenter, unnecessarily restricting the data currently in practice or potentially used for regulatory development would do little to rectify this situation in a timely or "transparent" manner. 
Response: EPA disagrees with the commenter that requirements of this rule will result in any meaningful delay in promulgating regulations. While this rule does require the Agency to evaluate the availability of data and models for pivotal science, the incremental burden to the Agency to carry out these requirements is expected to be small given (1) the extensive scientific review EPA already conducts regularly and (2) that the requirement is limited to pivotal science (i.e., typically a small, though highly important, subset of all the studies the EPA would review). Further, EPA notes that this final rule does not restrict the science that EPA can consider in its final significant regulatory actions and influential scientific information. Under the final rule no studies are categorically excluded from consideration, even studies where the underlying dose-response data are not available for independent validation. EPA will continue to assess the full body of science to evaluate the quality and transparency of available science for significant regulatory actions and influential scientific information. The EPA will also consider the degree to which pivotal science is available and give greater consideration to pivotal science for which the underlying dose-response data are available in a manner sufficient for independent validation. See section III.J of the preamble for further explanation.
Comment: Commenter (6137) asserts that the provisions in the proposed rule related to independent peer review of all pivotal regulatory science used to justify regulatory decisions is inconsistent with the provisions in the Office of Management and Budget (OMB) Bulletin that state agencies should "ensure peer review does not unduly delay the release of urgent findings." The commenter asserts that, if EPA wants to independently peer review all pivotal science, then it must clearly outline how it will ensure the process will not lead to undue delay. According to the commenter, the EPA has failed to do so. The commenter notes, for example, there is nothing in the Proposed Rule outlining how EPA will conduct an independent review of the studies underlying the NAAQS standard, which they state will require review of hundreds if not thousands of science documents, within the NAAQS review cycle. The commenter provides that the standard review period in the independent peer review process (i.e., from submission of a manuscript to final review) is time intensive. The commenter states that, if it can take months to review one manuscript, it is difficult to envisage how EPA could subject every piece of pivotal regulatory science to the same standard of review and still complete scientific assessments in a timely manner.
Commenter (6137) also asserts that the absence of a provision in the proposed rule to account for time-sensitive health and safety disseminations is inconsistent with such provisions in the OMB Peer Review Bulletin. The commenter contends that this encompasses most of the studies that EPA intends to review, including but not limited to, IRIS assessments, TSCA risk evaluations, and National Emissions Standards for Hazardous Air Pollutants (NESHAP) risk assessments. The commenter states that, in each of these cases, EPA utilizes scientific data to make safety determinations for chemical pollutants and impacts on human health and the environment, and thus, timely review is tantamount to protecting public health. The commenter argues that the Proposal adds another layer of review  -  one that is wholly unnecessary and duplicative  -  that is antithetical to the time sensitive nature of these reviews.
Response: EPA disagrees with the commenters that requirements of this rule will result in any meaningful delay in promulgating regulations. While this final rule requires the Agency to evaluate the availability of dose-response data for pivotal science, the incremental burden to the Agency to carry out these requirements is expected to be small given (1) the extensive scientific review EPA already conducts regularly and (2) that the requirement is limited to pivotal science (i.e., typically a small, though highly important, subset of all the studies the EPA would review). See section III.J of the preamble for further explanation. The EPA further notes that the requirements of this final rule do not apply to influential scientific information or pivotal science that is part of a time-sensitive health or safety dissemination. See 40 CFR 30.3 "The provisions of this part do not apply to physical objects (like laboratory samples), drafts, and preliminary analyses, and influential scientific information or pivotal science that meet one or more of the exemptions identified in Section IX of the OMB Final Information Quality Bulletin for Peer Review." Implementation guidlines and statute-specific transparency regulations or programmatic regulations are forthcoming.
Comment: Commenter (6147) states that the requirement to review past decisions may delay or halt the normal process of EPA decisions. The commenter provides that much of the early data may not be easily obtained and there may be no new data. The commenter expresses that the debate over what is "sound science" has become politicized and has raised a barrier to releasing information in a timely manner. The commenter contends that this attempt to focus on the uncertainties that are inherent in any complex analysis tends to detract from the purpose of seeking a solution to environmental problems. According to the commenter, while it is important to assure the public that regulatory decisions are based on the best available science, it is also important to seek resolution and begin the activities that will restore or protect the environment.
Response: This final rule applies prospectively to significant regulatory actions and influential scientific information and has no retrospective effect on either significant regulatory actions or influential scientific information. Such retroactive application would be time consuming and impracticable to do for all regulatory actions EPA has promulgated and influential scientific information EPA has developed. For future significant regulatory actions and influential scientific information, this final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. EPA disagrees with the commenters that requirements of this rule will result in any meaningful delay in promulgating regulations. 
Comment: Commenter (6446) expresses concern that a related likely consequence of the proposed rule, even if it contains ad hoc mechanisms for exceptions, is a reduction in EPA's ability to respond quickly to emerging challenges, when data and models take time to be made publicly available and/or redacted and otherwise prepared in a format appropriate for public review. Commenter (6903) states that the Proposed Rule would be a barrier to effective and timely EPA rulemaking and would make it nearly impossible for the Agency to expediently respond to existing and emerging issues and uphold its statutory mandates to protect the public's health.
Response: EPA disagrees that this rule will reduce EPA's ability to respond quickly to emerging challenges. The rule contains no requirement that dose-response data underlying pivotal science be made publicly available. Instead, it governs internal agency procedures for how the EPA will give greater consideration to studies where the underlying dose-response data are available in a manner sufficient for independent validation when promulgating significant regulatory actions or developing influential scientific information for which the conclusions are driven by the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect.
6.1.2.5.3 Other Comments Regarding Prospective or Retrospective Application
Comment: Commenter (0559) encourages EPA to clarify how historical and new studies will be treated under the proposed rule, including how model uncertainty will treat studies demonstrating low-dose effects.
Commenter (6191) states that proposed rule does not describe the process for retrospective application within the text of the rule.
Response: This final rule applies prospectively to significant regulatory actions and influential scientific information and has no retrospective effect on either significant regulatory actions or influential scientific information. Such retroactive application would be time consuming and impracticable to do for all regulatory actions EPA has promulgated and influential scientific information EPA has developed. For future significant regulatory actions and influential scientific information, this final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. 40 CFR Part 30.5 discusses uncertainty, "The EPA shall also describe critical assumptions and methods used in its dose-response assessment and shall characterize the variability and uncertainty of the assessment. The EPA shall evaluate the appropriateness of using default assumptions on a case-by-case basis. The EPA shall clearly explain the scientific basis for critical assumptions used in the dose-response assessment that the EPA relied on for the significant regulatory action or influential scientific information."
Comment: Commenter (6373) asserts that EPA should state whether it will decline to consider contemporaneous and future scientific studies that rely in part on past scientific work for which certain data were not or are not currently available.
Response: Under the final rule no studies are categorically excluded from consideration, even studies where the underlying dose-response data are not available for independent validation. EPA will continue to assess the full body of science to evaluate the quality and transparency of available science for significant regulatory actions and influential scientific information. The EPA will also consider the degree to which pivotal science is available and give greater consideration to pivotal science for which the underlying dose-response data are available in a manner sufficient for independent validation.
6.1.3 How Does What this Rule Would Require Compare to Requirements of Environmental Statutes?
Comment: Commenter (0671) states that the proposed regulatory change will force the EPA to ignore the best available science on dose response and models, which is counter to the intent and text of the CAA and amendments, and which will introduce bias by preventing the EPA from using what are often the most relevant and well-designed studies.
Response: Under the final rule no studies are categorically excluded from consideration and the EPA will continue to assess the full body of science to evaluate the quality and transparency of available science for final significant regulatory actions and influential scientific information. The EPA will also consider the degree to which pivotal science is available and give greater consideration to those studies with greater data and model availability. Further, as discussed in the final rule preamble in section III.A, this internal procedural rule does not interpret or apply any provisions of the statutes that EPA administers, and the EPA is maintaining language in 40 CFR 30.3 stating that the statutes EPA administers, or their implementing regulations, will control in the event of any conflicts with the requirements of this rule. When implementing this procedural rule in forthcoming statute-specific transparency regulations or programmatic regulations, the Agency will continue to comply with and abide by the requirements in those statutes and implementing regulations, including regulatory deadlines.
Comment: Commenters (6117, 6133, 6137) assert that it would be unlawful for this rule to have a retrospective effect. 
Response: See discussion and response in section 15.2 of this Chapter (Could Prospective or Retrospective Application Introduce Bias Regarding the Timeliness and Quality of Scientific Information Available?) subsection B.3 (Oppose Retroactive Application; Legal Considerations) of this Chapter (15. Strengthening Transparency in Regulatory Science  -  Pivotal Regulatory Science).
Comment: Commenter (6133) states that the Proposal conflicts with the statutes that EPA administers. The commenter provides that the Proposal unlawfully restricts EPA's consideration and use of "dose response data and models that underlie" what the Proposal calls "pivotal regulatory science." 83 FR 18770/2.
      Pivotal regulatory science' is the studies, models, and analyses that drive the magnitude of the benefit-cost calculation, the level of a standard, or point-of-departure from which a reference value is calculated. In other words, they are critical to the calculation of a final regulatory standard or level, or to the quantified costs, benefits, risks and other impacts on which a final regulation is based. Id. 
Commenter (6133) asserts that, by restricting EPA's implementation of its federal organic statutes and the APA in this fashion, and by defining "pivotal regulatory science" in this manner, the Proposal violates federal laws. The commenter states that the Proposal does so by requiring EPA to implement federal laws based on the Proposal's criteria and concept of "pivotal regulatory science," rather than on the congressional criteria and requirements in federal statutes that contradict, disallow, or fail to include those criteria and concepts in the Proposal.
Response: EPA disagrees that this final rule conflicts with the statutes EPA administers. As a threshold matter, and as discussed in the final rule preamble in section III.A, this internal procedural rule does not interpret or apply any provisions of the statutes that EPA administers. Furthermore, 40 CFR 30.3 states that the statutes and regulations will control in the event of any conflicts, and EPA clarifies that the requirements in this final rule set the overarching structure and principles for science transparency in the EPA's final significant regulatory actions and influential scientific information. The EPA intends to develop internal implementation guidance and promulgate either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule, which will clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes EPA administers.
Comment: Commenter (6133) complains that EPA not only fails to provide a passable definition for its invented term, "pivotal regulatory science," the agency also fails to provide its rationale for limiting the scope of the rule to so-called "pivotal regulatory science." The commenter asks (1) why is the "public availability of science and data in a manner sufficient for independent validation" any less important or necessary or justified when the science is not "pivotal" or "critical" to a regulatory decision, (2) why should not all science, studies, data and information considered by EPA meet the standards for transparency, verifiability, independent validation, and trustworthiness that are the abiding concerns of the Proposal, and (3) why is it not arbitrary and capricious for EPA to continue to consider science and data that are unavailable and insufficient for independent validation in areas outside the reach of the Proposal? The commenter states that EPA offers no explanation for this disparate treatment; the agency's reasoning, such as it is, is entirely conclusory. The commenter provides that, by way of explanation for the limitation, EPA only suggests that the imposed standards "are of paramount importance when the government relies on science to inform its significant regulatory decisions." According to the commenter, this explanation is hopelessly circular and ultimately incoherent. The commenter states that EPA does not explain why it believes this explanation to be true. The commenter adds that the Proposal just substitutes the word, `paramount,' for the word, `critical,' that it substitutes for the word, `pivotal.' The commenter asserts that this failure to thoroughly explain both the term "pivotal regulatory science" in a way that meaningfully defines the scope of the regulation, and the rationale behind limiting the application only to pivotal (critical, paramount) science, makes it impossible for interested parties to comment fully and meaningfully on the Proposal. The commenter suggests that, should EPA intend to finalize this unlawful proposal, the EPA first must withdraw the Proposal, then issue a supplemental proposal with the necessary definitions and explanations. 
Response: EPA agreed that there was value in issuing a supplemental proposal to make clarifications, modifications and additions to certain provisions in the rulemaking and therefore issued a supplemental notice of proposed rulemaking in March 2020. In the supplemental, EPA clarified, that "EPA would continue to use standard processes for identifying, evaluating, and reviewing available data, models, and studies. When the Agency has potentially identified multiple key studies or models of similar quality that could drive its subsequent decisions, the Agency will investigate the availability of the underlying data." EPA has limited the primary requirements of this rule to pivotal science, which is one or two studies, for two primary reasons. First, although this rulemaking does not require reanalysis of studies, it is informative to evaluate published findings on the patterns of such attempts since journal data sharing policies have been adopted. Lewandowsky et al. (2020) evaluated the cost-effectiveness of reanalysis studies under various scenarios and concluded that reanalysis studies are most cost-effective when they are focused on studies of the greatest interest to the scientific community (in this study, the number of citations was a surrogate for interest) (Ref. 43). This finding is consistent with results in other studies that found and encouraged narrowing the focus of attempted reanalysis studies to those studies of greater significance (Refs. 70, 72, 73, 74). Consistent with the recommendations of these studies and commenters and the EPA's commitment to cost-effective government practices, it is logical to narrow the applicability of this rule to those studies that are most pivotal or "key" to the development of final significant regulatory actions and influential scientific information. Second, though it was never the Agency's intent, the EPA finds that implementing the data and model availability requirements of this final rule for all the studies EPA considers in final significant regulatory actions and influential scientific information may be infeasible.
Finally, EPA disagrees that the Agency should not consider science where the underlying data and models are not publicly available. The EPA will continue to consider all peer-reviewed science evaluated in the EPA's influential scientific information and final significant regulatory actions, consistent with existing study quality assessment factors and the statutory mandates specific to that rulemaking.
Comment: Commenter (6133) states that the EPA does not and cannot identify any statutory basis -- in federal environmental statutes, the APA or otherwise -- to apply the Proposal's approach. The commenter provides that EPA's preambular explanation says that "pivotal regulatory science" is "critical to the calculation of a final regulatory standard or level, or to the quantified costs, benefits, or risks and other impacts on which a final regulation is based." According to the commenter, the Proposal nowhere explains what these "other impacts" are. The commenter states that the Proposal does not limit or bound these "other impacts," nor link them to the sentence's incoherent notion of what is "critical" and what is not. Commenter states that the preambular gloss is inconsistent with the proposed CFR definition. According to the commenter, the former says "pivotal regulatory science" is critical to vague "other impacts" on which a final rule is based. The commenter states that the proposed CFR definition, by contrast, says "pivotal regulatory science" "drive[s] the requirements and/or quantitative analysis of EPA final significant regulatory decisions." The commenter contends that the Proposal does not square the contradictions between science that drives a final rule's requirements and science that is "critical" to "other impacts" in a final rule. 
Response: The EPA intends to promulgate regulations under the environmental statutes EPA implements to further clarify how the Agency will apply the definition of pivotal science in specific programs authorized under those statutes (e.g., CAA, CWA, SDWA, RCRA, FIFRA, TSCA, Emergency Planning and Community Right-to-Know Act (EPCRA)). The specific criteria for determining pivotal science may necessarily be specific to the authorizing statute, as well as the final significant regulatory action or the influential scientific information. The EPA intends to explain in each final significant regulatory action and for each influential scientific information how the pivotal studies were identified.
6.1.4 Recommended Changes to the Definition of Pivotal Regulatory Science at §30.2
6.1.4.1 Pivotal Regulatory Science 
6.1.4.1.1 Support for the Proposed Definition
Comment: Commenters (6150, 6180) support the Proposed Rule's definition of pivotal regulatory science as referring to the specific studies or analyses that drive the requirements and/or quantitative analysis of EPA's significant regulatory decisions. Commenter (6150) states that, as noted in the Preamble, this includes scientific studies, models, and analyses that drive the magnitude of the benefit-cost calculation, the level of a standard, or point-of-departure from which a reference value is calculated.
Commenter (6375) concurs with the general definition of "pivotal regulatory science" in the Proposed Rule. Commenter states that the definition of "pivotal regulatory science" could be further clarified, however, to include examples that illustrate situations in which the definition would and would not apply. The commenter suggests that examples where the rule would apply should include:
 Studies cited as the basis for Causal or Likely Causal determinations in a NAAQS Integrated Science Assessment, if they are not part of a generalized weight-of-evidence approach that incorporates a broader literature base;
 Studies EPA uses for its mortality or morbidity projections in a NAAQS Health Risk and Exposure Assessment, or the specification of a definitive concentration-response function upon which projections are based; and
 Studies EPA cites as a basis for quantifying a NAAQS.
Response: EPA acknowledges the commenters' support for the definition of pivotal regulatory science, and notes that the term has been combined with the term "pivotal science," proposed in the 2020 supplemental notice of proposed rulemaking. This final rule establishes how the EPA will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information. An integrated science assessment would be considered influential scientific information, but the final rule focuses on only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). These are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information.
6.1.4.1.2 Do Not Support the Proposed Definition
General Comments
Comment: Commenter (0671) states that EPA should not attempt to define the concept of "pivotal regulatory science," because the premise of this proposed regulatory change is flawed.
Commenter (3971) suggests that EPA assemble small groups of dedicated scientists in order to produce a definition of "pivotal regulatory science" and how to implement such definition.
Response: When promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data, the Agency shall follow best practices to evaluate potential links between exposure to a pollutant, contaminant, or substance and the effect and the nature of the dose-response relationship. The EPA will use the following factors to assess the quality of studies identified in the systematic review: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. The EPA will rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence of a relationship between exposure and effect, the EPA will identify those studies based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both of an EPA significant regulatory action or influential scientific information will be identified as pivotal science. The EPA intends to issue implementation guidelines and statute-specific rulemakings that will further describe these criteria and how the EPA will identify pivotal science in its assessments and rulemakings. 
Definition Not Adequate and Lacks a Scientific or Legal Foundation
Comment: Commenters (1011, 1272, 4881, 6114, 6116, 6131, 6133, 6144, 6361, 6362, 6373, 6909, 6924, 8281-PH20) express concern that the term "pivotal regulatory science" is not adequately defined. Commenter (4881) states that a central element of the proposed rule is the vague definition of "pivotal regulatory science" to which the rule would apply. Commenter (6114) states that it is difficult to understand why the agency would publish a rule with such an obvious flaw, as this could leave the interpretation of this term up to the courts, which may not turn out the way the EPA intended. Commenter (6133) states that the term "pivotal regulatory science" is perhaps the most vague, unexplained and internally inconsistent term used in the Proposal. Commenter (6144) states that the term used in the proposed rule, "pivotal regulatory science" has no precise meaning in science and seems to be contrived to cover an arbitrary set of information without clear articulation and basis. Commenters (6114, 6116, 6131, 6361) state that although the term "pivotal regulatory science" is used several times in the notice and appears to be a focus for this rule, it is not adequately defined in the Federal Register notice.
Commenter (1272) states that the Food and Drug Administration (FDA) describes "regulatory science as a science-based decision-making process needed to fulfill the responsibilities of a public health agency", while the dictionary describes pivotal as "of crucial importance in relation to the development or success of something else". The commenter provides that, using these two in conjunction, pivotal regulatory science would mean any using science-based decision making whenever there is importance in the developing regulation, which they state that one could argue is every decision the EPA makes.
Commenters (1011, 6133, 6362) express concern that, in the proposed rule, EPA variably defines pivotal regulatory science as "the studies, models, and analyses that drive the magnitude of the benefit-cost calculation, the level of a standard, or point-of-departure from which a reference value is calculated," and "the specific scientific studies or analyses that drive the requirements and/or quantitative analysis of EPA final significant regulatory decisions." Commenter (1011) states that these designations are vague and without further guidance for what constitutes pivotal regulatory science, EPA will introduce uncertainty into its decision-making process. The commenter adds that it is unclear why EPA is choosing to create a new class of scientific information and holding it to a standard of data access that the rulemaking does not require for "non-pivotal" regulatory science. 
Commenters (4839, 4881, 6924) express concern that "pivotal regulatory science" is not a phrase defined in or widely used in scientific literature.
Commenter (4839) states that the draft regulation embraces a policy agenda that lacks a foundation in scientific research, peer review and application to regulatory policy. The commenter asserts that the proclamation of new terms such as "pivotal regulatory science" is more rhetorical than real and demonstrates no connectivity to probabilistic risk assessment or other well-established methodologies that guide public health and environmental policies. The commenter contends that the term "pivotal regulatory science" is un-peer reviewed and lacks transparency regarding key decision factors used to weigh the role and utility of data used in linear or non-linear dose response functions, cost benefit analysis, identification of policy options or regulatory impact analysis. The commenter adds that pivotal regulatory science is presented as a cornerstone "to change agency culture and practices," an agenda that embodies a political objective rather than a scientific purpose. 
Commenter (6924) states that "pivotal regulatory science" is not scientific terminology and is not used in the scientific literature. According to the commenter, the proposed rule's definition appears to attempt to delineate a smaller subset of relevant studies that would be "pivotal regulatory science", but the term is not useful because the entire body of relevant scientific evidence would meet the definition of pivotal regulatory science. The commenter adds that this is because a properly conducted assessment considers the entire body of evidence in the evaluation, integration and development of conclusions. The commenter provides that, for example, though there may be one study or several studies that contribute to the numerical calculation of a point of departure, the rest of the body of scientific evidence is used to make determinations that also are important in the magnitude of the benefit-cost calculation, including: (1) uncertainty factors used in calculations of risk; (2) whether particular sub-populations are susceptible and need greater protections; (3) the modeling of dose-response; (4) the likely occurrence and severity of potential health effects; (5) population exposure levels, etc. The commenter adds that a typical EPA assessment draws upon hundreds to thousands of studies, and according to this definition all the underlying data of those studies would need to be made publicly available.
Commenters (4881, 6114, 6116, 6131, 6361) assert that although the term "pivotal regulatory science" is used several times in the notice and appears to be a main focus for this rule, it does not appear anywhere else in the legal records, court decisions, or rules that address environmental protection or the role of the EPA. 
Commenters (6116, 6131, 6361) state that, with no grounding in the law, it is unclear what EPA is trying to achieve by designating certain research as "pivotal regulatory science."
Commenter (6133) states that the term has no statutory basis in any statute cited by EPA, or otherwise. Commenter states that, beyond having no statutory underpinning, the meaning of the phrase is neither self-evident nor adequately defined in the Proposal. 
Response: EPA disagrees that the term pivotal science (which encompasses the proposed term pivotal regulatory science in this final rule) is inadequately defined. 40 CFR Part 30.2 clearly defines pivotal science as meaning "the specific dose-response studies or analyses that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information."
In the 2020 SNPRM, the EPA introduced the term "pivotal science," defined in proposed 40 CFR 30.2 as "the specific scientific studies or analyses that underly [sic] influential scientific information." This term was proposed as a parallel to "pivotal regulatory science," defined in 40 CFR 30.2 of the 2018 proposed rule as "the specific scientific studies or analyses that drive the requirements and/or quantitative analysis of EPA significant regulatory decisions." The EPA received comment on the use of "regulatory" in "pivotal regulatory science." Some commenters contended that there is no such thing as science that is regulatory; rather, there is science used to support regulation. Some commenters also noted that the terms "pivotal science" and "pivotal regulatory science" have similar scopes. The EPA acknowledges that no scientific study is inherently regulatory; rather, the EPA uses science to inform its significant regulatory actions. In order to increase the clarity of this final rule, to take into account the similarities between the two definitions, and to more accurately describe the science that the EPA uses, the EPA is removing the term "pivotal regulatory science" and combining the definitions of "pivotal science" and "pivotal regulatory science" under the single term "pivotal science" in 40 CFR 30.2. The EPA is responding to comments on both terms together.
The EPA also acknowledges that the term "regulatory science" may be confusing because it suggests either the term refers to a scientific discipline of regulatory decision-making (akin to FDA), or that some science the EPA considers is inherently regulatory. Neither of these interpretations reflects the Agency's intent in defining this term. The EPA considers the breadth of scientific evidence in its rulemakings; while this scientific evidence informs policy decisions, the EPA's consideration of the science does not make it "regulatory science." To reflect this fact, in the final rule the EPA is changing the proposed term "regulatory science" to "science that serves as the basis for informing a significant regulatory action."
In this final rule, rather than considering all studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact as "pivotal science," the EPA is balancing transparency and feasibility by focusing on those studies that inform the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect. Thus, "pivotal science" includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). These are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. In the context of this rule, studies informing the dose-response modeling (e.g., benchmark response selection), studies identifying data for toxicokinetic adjustments, or studies informing the selection of uncertainty factors do not drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. Future statute-specific rulemakings may interpret "pivotal science" more broadly.
Definition Does Not Clarify Pivotal Regulatory Science Applicability 
Comment: Commenters (6114, 6116, 6131, 6361) state that it is impossible to determine exactly what the ramifications of this rule would be. 
Commenters (6114, 6116, 6131) state that, for example, it is unclear whether the term "pivotal regulatory science" would apply to studies or modeling conducted by industry groups, meaning that the research performed by these groups could be held to a lower standard than research performed by reputable scientists around the world. 
Commenter (6133) states that EPA's choice to modify "regulatory science" with the adjective "pivotal" does nothing to clarify the scope of scientific studies and information encompassed by the Proposal. The commenter states that it is arbitrarily vague, and unexplained under the Proposal which science would be considered "pivotal," and under what conditions. The commenter adds that, because the term was created out of thin air to serve EPA's purposes and has no statutory grounding or intuitive meaning, this ambiguity-ridden definition is woefully inadequate. The commenter further adds that it is also arbitrary and capricious and an abuse of EPA's discretion and that EPA is well aware of the insufficiency of the definition, as is evident in the agency's solicitation of comments on the definitions of "pivotal regulatory science" and "dose response data and models" within the Proposal.
Response: In the 2020 SNPRM, the EPA introduced the term "pivotal science," defined in proposed 40 CFR 30.2 as "the specific scientific studies or analyses that underly [sic] influential scientific information." This term was proposed as a parallel to "pivotal regulatory science," defined in 40 CFR 30.2 of the 2018 proposed rule as "the specific scientific studies or analyses that drive the requirements and/or quantitative analysis of EPA significant regulatory decisions." The EPA received comment on the use of "regulatory" in "pivotal regulatory science." Some commenters contended that there is no such thing as science that is regulatory; rather, there is science used to support regulation. Some commenters also noted that the terms "pivotal science" and "pivotal regulatory science" have similar scopes. The EPA acknowledges that no scientific study is inherently regulatory; rather, the EPA uses science to inform its significant regulatory actions. In order to increase the clarity of this final rule, to take into account the similarities between the two definitions, and to more accurately describe the science that the EPA uses, the EPA is removing the term "pivotal regulatory science" and combining the definitions of "pivotal science" and "pivotal regulatory science" under the single term "pivotal science" in 40 CFR 30.2. The EPA is responding to comments on both terms together.
The EPA also acknowledges that the term "regulatory science" may be confusing because it suggests either the term refers to a scientific discipline of regulatory decision-making (akin to FDA), or that some science the EPA considers is inherently regulatory. Neither of these interpretations reflects the Agency's intent in defining this term. The EPA considers the breadth of scientific evidence in its rulemakings; while this scientific evidence informs policy decisions, the EPA's consideration of the science does not make it "regulatory science." To reflect this fact, in the final rule the EPA is changing the proposed term "regulatory science" to "science that serves as the basis for informing a significant regulatory action."
The EPA notes that implementation guidelines and statute-specific transparency regulations or programmatic regulations are forthcoming. At that time, the agency will be better able to determine and assess potential ramifications of implementing this procedural rule.
Comment: Commenter (6909) states that EPA has not made it clear in the proposed rule how or even whether the rule would apply in a variety of frequently encountered situations. The commenter notes that, for example, EPA is openly unsure whether the rule would apply to a study underlying a guidance document; whether it would apply to a study that merely affirms an existing EPA standard; whether it would apply to a study underlying an individual party adjudication, or an enforcement action, or a site-specific permitting action. 
Response: The EPA has considered the public comments and finalized the scope of this rule to apply to significant regulatory actions and influential scientific information. The EPA notes that implementation guidelines and statute-specific transparency regulations or programmatic regulations are forthcoming.
Comment: Commenter (6909) states that many of the central terms that will dictate how and to what studies the rule would apply are undefined or unclear.
Response: The EPA agrees that some terms were unclear. The final rule, in response to comments, defines the following terms: "data," "dose-response data," "independent validation," "influential scientific information," "pivotal science," "publicly available," "reanalyze," "science that serves as the basis for informing a significant regulatory action," and "significant regulatory actions." Each definition is discussed further in the preamble to the final rule.
Definition Allows for Limitless Discretion
Comment: Commenter (4881) states that there is no useful guidance in the language of the proposed rule for an EPA Administrator to apply in deciding which issues fit and which do not fit the fluid definition of "pivotal." According to the commenter, this ambiguity invites intrusion of political preference and prejudice into the essential scientific process of public health risk assessment. 
Commenter (6137) states that the EPA's Proposed Rule lacks the requisite standards and principles that are the hallmark of lawful agency decision-making. The commenter asserts that the vague definitions proposed in 40 CFR §30.2 invite limitless agency discretion to decide when the Rule's requirements apply. The commenter notes that the Proposed Rule defines "pivotal regulatory science" as studies that "drive the requirements and/or quantitative analysis" of EPA's action, but this lacks any understandable meaning. The commenter states that there is no standard governing what science does or does not "drive" the regulatory action. According to the commenter, the lack of regulatory standards here means that EPA staff will impermissibly determine applicability of the rule "based upon their own unwritten personal standards." White v. Roughton, 530 F.2d 750, 754 (7th Cir. 1976). The commenter adds that these provisions allow for arbitrary application of the Rule, rendering the Proposed Rule arbitrary and thus unlawful. 
Commenter (6373) states that the Rule does not clarify who will be determining which studies are considered "pivotal regulatory science" or at what stage or stages of EPA's review process this determination will be made. According to the commenter, this is particularly concerning given the former EPA Administrator's directive last year to remove or refuse to consider for membership former and current recipients of EPA grants from the Clean Air Scientific Advisory Committee (CASAC) and other advisory committees. Commenter states that, unless EPA has in place a fair review process for determining "pivotal regulatory science," the Rule can be misused to disregard or discredit relevant studies.
Response: The EPA notes that implementation guidelines and statute-specific transparency regulations or programmatic regulations are forthcoming. The EPA has clarified in this final rule that in general, the EPA will rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify a subset of those studies based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints that would inform a dose-response assessment for those effect endpoints and drive the requirements and/or quantitative analyses of an EPA final significant regulatory action or influential scientific information will be identified as pivotal science. The EPA disagrees that designating a set of key studies as "pivotal science" will necessarily be biased or without accountability. The EPA follows an objective, unbiased process for identifying and evaluating scientific studies and already identifies key or pivotal studies in some of its actions (e.g., IRIS assessments).
6.1.4.1.3 Recommendations for Specific Changes to the Definition
Comment: Commenter (0567) recommends the EPA delete the words "final" and significant" in the proposed definition of pivotal regulatory science. The commenter interprets that the Agency identifies pivotal regulatory science as the key scientific data and methods used to translate research results into application in the regulatory process. The commenter asserts that there is no reason not to apply appropriate processes in all regulations. 
Commenter (4932) recommends that EPA alter the definition of "pivotal regulatory science" by striking the word "final" from the proposed definition to make the Transparency Proposal fully consistent with Executive Order 12866. The commenter notes that "regulatory action" under Executive Order 12866 means "any substantive action by an agency (normally published in the Federal Register) that promulgates or is expected to lead to the promulgation of a final rule or regulation, including notices of inquiry, advance notices of proposed rulemaking, and notices of proposed rulemaking." The commenter adds that the necessary (and seemingly unintended) implication of including the word "final" in the definition is that the transparency objective (i.e., the disclosure of pivotal regulatory science) does not apply to advanced notices of proposed rulemaking or proposed rulemaking. According to the commenter, the public must have access to pivotal regulatory science at these earlier steps in the rulemaking process in order to assess effectively the merit of the Agency's regulatory science. The commenter contends that disclosure of pivotal regulatory science when final agency action is issued is too late in the rulemaking process as any opportunity for public comment will have ended. 
Commenter (6155) states that the following language from EPA's regulations adopted under the Data Quality Act may help further clarify the terms "pivotal regulatory science" and "significant regulatory actions" as defined in §30.2 of the proposed rule. The commenter provides that EPA will generally consider the following classes of information to be influential, and, to the extent that they contain scientific, financial, or statistical information, that information should adhere to a rigorous standard of quality: 
 Information disseminated in support of top Agency actions (i.e., rules, substantive notices, policy documents, studies, guidance) that demand the ongoing involvement of the Administrator's Office and extensive cross-Agency involvement; issues that have the potential to result in major cross-Agency or cross-media policies, are highly controversial, or provide a significant opportunity to advance the Administrators priorities. Top Agency actions usually have potentially great or widespread impacts on the private sector, the public or state, local or tribal governments. This category may also include precedent-setting or controversial scientific or economic issues.
 Information disseminated in support of Economically Significant actions as defined in Executive Order 12866, entitled Regulatory Planning and Review (58 FR 51735, October 4, 1993), Agency actions that are likely to have an annual effect on the economy of $100 million or more or adversely affect in a material way the economy, a sector of the economy, productivity, competition, jobs, the environment, public health or safety, or State, Tribal, or local governments or communities.
 Major work products undergoing peer review as called for under the Agency's Peer Review Policy. Described in the Science Policy Council Peer Review Handbook, the EPA Peer Review Policy regards major scientific and technical work products as those that have a major impact, involve precedential, novel, and/or controversial issues, or the Agency has a legal and/or statutory obligation to conduct a peer review. These Major work products are typically subjected to external peer review. Some products that may not be considered "major" under the EPA Peer Review Policy may be subjected to external peer review but EPA does not consider such products influential for purposes of these Guidelines.
 Case-by-case: The Agency may make determinations of what constitutes "influential information beyond those classes of information already identified on a case-by-case basis for other types of disseminated information that may have a clear and substantial impact on important public policies or private sector decisions.
Commenter (6356) suggests EPA remove the word "significant" from the proposed definition of "pivotal regulatory science" or it should simply remove the term from the final rule and replace it with the definition of another proposed term "regulatory science" -- from which it also should delete the word "significant." According to the commenter, it does not appear that both terms are necessary, and they make cryptic distinctions between data and assumptions that drive the agency's regulatory decisions.
Commenter (6362) states that, assuming the intent is to define and distinguish the subset of scientific studies and analyses that form the scientific foundation for EPA's regulatory decisions from the larger universe of all the scientific information reviewed and considered by the agency, a more precise word than "pivotal" would be "material." The commenter provides that those scientific studies and analyses that are material to its regulatory decision must be or be made publicly available in a manner sufficient for independent validation.
Commenter (6449) states that it is not clear whether "pivotal regulatory science" means studies and analyses performed by the EPA, or those studies published by others in peer-reviewed journals that might support EPA's analysis. The commenter suggests that EPA consider that pivotal studies may also be those that determine qualitatively the weight of evidence (WOE) that a specific toxic endpoint is of relevance to human health or the species of concern. The commenter contends that selection of the appropriate toxic endpoint based on WOE after systematic review of the literature is as important as appropriate quantitative analysis. 
Response: The EPA has removed the word "final" from the definition, so that pivotal science applies to EPA significant regulatory actions. The EPA also notes that this action also requires the EPA to identify and make publicly available the science that serves as the basis for informing a significant regulatory action at the proposed or draft stage to the extent practicable. Pivotal science is defined as the specific dose-response studies or analyses that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. Therefore pivotal science is not limited only to the science developed by the EPA.
6.1.4.1.4 Need More Clarity in the Definition
Comment: Commenter (1011) states that they do not interpret the proposed rule's classification of pivotal regulatory science as a change to the current understanding of what constitutes "best available science". The commenter urges EPA to continue to use the best available science when building the foundation for regulatory actions. The commenter asserts that the studies used by EPA should adhere to good laboratory and field practices, with results being peer-reviewed and any associated methods, models, and data made available for further review and analysis. The commenter recommends that, in cases where patient confidentiality or other circumstances prevents the dissemination of data, EPA should continue to follow the guidelines set forth in the OMB's Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies. These guidelines appropriately state that the worthwhile goal of allowing public access to data and methods "does not override other compelling interests such as privacy, trade secrets, intellectual property, and other confidentiality protections." The commenter further recommends that, if any aspect of the proposed rule is intended to alter EPA's definition and use of the best available science, the Agency should issue further guidance clarifying these changes and allow for public comment and consultation with the scientific community prior to implementation.
Commenter (6448) states that it is unclear how "pivotal regulatory science" differs from "best available science", if at all, and when well understood "best available science" would become "pivotal regulatory science." The commenter contends that the introduction of an additional ill-defined term would only lend towards confusion and inconsistency in regulatory decisions. The commenter recommends that EPA reconsider this approach and utilize the existing "best available science" standard along with solicitation of public comment and consultation with the EPA SAB and/or broader scientific community. 
Commenter (6126) suggests that EPA reduce the vagueness and ambiguity in the definitions. The commenter asserts that the lack of clarity about the definitions cited here limits the ability to interpret intent and potential impact of the proposal. The commenter states that, after EPA clarifies definitions, a new round of public comment will likely be needed to address the basic questions raised by EPA in the proposed rule on other matters. The commenter asserts that "pivotal regulatory science" is a new term, undefined and unused in the existing text of Title 40 of the Code of Federal Regulations. The commenter notes that the meaning appears to hinge on whether a study or analysis "drives" EPA's actions. According to the commenter, this appears to be a dichotomous designation ("drives"/ "doesn't drive"), which may not reflect how many individual studies or analyses are factored into the development of regulation. The commenter adds that, in practice, the use of evidence involves different types of use considerations., The commenter states that further clarification is needed about how EPA intends to interpret this term. 
Response: This purely internal procedural rulemaking is not intended to modify the Agency's interpretations of "best available science." "Pivotal science" is a statutorily neutral term and therefore is more generally applicable. The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. The EPA will then identify and consider "pivotal science in accordance with the provisions of this rule," unless the implementation of the rule conflicts with statutory requirements and associated implementing regulations. It should also be noted that under the final rule no studies are categorically excluded from consideration, even studies where the underlying dose-response data are not available for independent validation. EPA will continue to assess the full body of science to evaluate the quality and transparency of available science for significant regulatory actions and influential scientific information. The EPA will also consider the degree to which pivotal science is available and give greater consideration to pivotal science for which the underlying dose-response data are available in a manner sufficient for independent validation.
6.1.4.1.5 "Drives the Requirements" Phrase
Comment: Commenter (2169) states that, because there is no mechanism in the regulation for assuring that the science drives the decision, there is no means for judging the term pivotal. The commenter asserts that, without transparent consideration of all the information available and public comment on whether all pertinent information was considered, it is pointless to promulgate rules on the availability of the information.
Commenter (6133) states that the use of the phrase "drive the requirements" within the CFR definition is particularly incoherent. The commenter asks the following questions: 
 What does "drive the requirements" mean? 
 Does the definition apply only to scientific studies that were outcome determinative? 
 Does it encompass any scientific study that was considered in making the requirements? 
 What about studies that were useful but not determinative? Something else entirely? 
 Can more than one study be "pivotal" to the regulatory decision, or does the term "drive the requirements" imply that only one study could be "pivotal" to a given decision? 
 Are most of the studies used by EPA considered to "drive the requirements" or is this term limited in some fashion, unrevealed to the public? 
 Will EPA "know it when it sees it," making it up as the agency goes along?
Commenter (6137) asserts that the Proposed definition of "pivotal regulatory science", as studies that "drive the requirements and/or quantitative analysis" of EPA's action, lacks any understandable meaning. The commenter notes that there is no standard governing what science does or does not "drive" the regulatory action. According to the commenter, the lack of regulatory standards here means that EPA staff will impermissibly determine applicability of the rule "based upon their own unwritten personal standards." White v. Roughton, 530 F.2d 750, 754 (7th Cir. 1976). The commenter contends that these provisions allow for arbitrary application of the Rule, rendering the Proposed Rule arbitrary and thus unlawful.
Response: The EPA clarifies that "pivotal science" includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). These are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. In the context of this rule, studies informing the dose-response modeling (e.g., benchmark response selection), studies identifying data for toxicokinetic adjustments, or studies informing the selection of uncertainty factors do not drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. With regard to the meaning of "drive the requirements and/or quantitative analysis," these are the studies that are integral to quantitatively characterizing dose-response relationships for the toxicity endpoints that underlie the requirements or analyses of EPA significant regulatory actions or influential scientific information. The EPA may further interpret the meaning of "drive," and describe the process for designating key studies as pivotal science in subsequent implementation guidelines and/or statute-specific rulemakings.
6.1.4.1.6 Significant Regulatory Decisions
Comment: Commenter (6909) states that there are numerous ways in which the proposed rule is unacceptably vague. The commenter notes that many of the central terms that will dictate how and to what studies the rule would apply are undefined or unclear, including "pivotal regulatory science". The commenter provides that, even though the proposed rule offers a limited definition of "pivotal regulatory science," that definition hinges on the phrase "significant regulatory decisions." The commenter states that there is no explanation of how EPA would decide which regulatory decisions are significant and which are not. 
Commenter (6137) argues that the EPA's Proposed Rule lacks the requisite standards and principles that are the hallmark of lawful agency decision-making. The commenter notes that the Proposed Rule defines "regulatory science" as "scientific information . . . that provide the basis for EPA final significant regulatory decisions." The commenter adds that there are no standards or definitions for what subset of "regulatory decisions" should be deemed "significant." According to the commenter, these provisions allow for arbitrary application of the Rule, rendering the Proposed Rule arbitrary and unlawful.
Response: The EPA notes that the operable definition in the final rule is for "significant regulatory actions," not "significant regulatory decisions." Significant regulatory actions are defined in the final rule as "final regulations determined to be `significant regulatory actions' by the Office of Management and Budget pursuant to Executive Order 12866." The EPA disagrees that this term is vague, arbitrary, or unlawful.
6.1.4.2 Pivotal Regulatory Science and Regulatory Science
Comment: Commenters (1427, 6133, 6362) express that the proposed definitions of pivotal regulatory science and regulatory science lack clarity and overlap.
Commenter (1427) states that the proposed definitions of pivotal regulatory science and regulatory science are not sciences, overlap each other and obscure any legitimate need to distinguish between regulation and pivotal regulation. The commenter clarifies that, for example, there is no difference between chemistry and regulatory chemistry. According to the commenter, if this regulation is ultimately deemed necessary, the EPA might want to define pivotal regulation.
Commenter (6362) states that the proposed definitions of "pivotal regulatory science" and "regulatory science" can be interpreted as simultaneously referencing something identical as well as one being a subset of the other. The commenter asserts that, therefore, the definitions are vague and need clarification.
Commenter (6133) states that, as proposed, the terms "regulatory science" and "pivotal regulatory science" are vague, even incoherent terms with definitions that lend no assistance to commenters in understanding the Proposal. The commenter argues that the terms lack statutory authority and are vague, inconsistent, unexplained, and otherwise arbitrary and capricious.
Commenter (6133) expresses concern that the Proposal is internally confusing through the workings of its proposed regulatory text. Commenter states that EPA alternates between explaining the Proposal in terms of "regulatory science" and "pivotal regulatory science." The commenter cites, for example, in proposed §30.1, the Proposal "directs EPA to ensure that the regulatory science underlying its actions is publicly available . . . " and, later, in proposed §30.3, the Proposal indicates that the provisions apply "to dose response data and models underlying pivotal regulatory science that are used to justify significant regulatory decisions." The commenter adds that, in the subsequent section, proposed §30.4, the Proposal references "all studies (or regulatory science) relied upon . . .." According to the commenter, the Proposal is arbitrarily vague and incoherent concerning whether "regulatory science" that is relied upon is the same as "pivotal regulatory science," or whether it is a new category of science entirely. The commenter asks: (1) if this phrasing implies that the definition of "regulatory science" does not already include science that is "relied upon"? and, if so, (2) does EPA mean that the phrase, "provides the basis," is not synonymous with "relied upon"? The commenter states that the Proposal provides no answers to these questions.
Commenter (6133) expresses concern that the Proposal makes no attempt to clarify how "pivotal regulatory science" is distinct from the separately defined, "regulatory science," a term integral to proposed §30.1, which states the Proposal's very purpose. The commenter provides that the definition of "regulatory science" is almost identical to that of pivotal regulatory science, with the exception that "regulatory science" encompasses information that "provide the basis for EPA final significant regulatory decisions," while "pivotal regulatory science" "drives the requirements." The commenter states that the phrase "provides the basis" in the proposed "regulatory science" (§30.2) definition could mean that science was one of many studies considered, that it was the bedrock study upon which regulation was grounded, that EPA relied on the study, or that the study was critical to EPA's determination. According to the commenter, the Proposal nowhere addresses or explains whether or how these possible meanings are distinct from the possible meanings of the "drive the requirements" phrase of the "pivotal" definition. The commenter concludes that it is entirely unclear from these definitions what makes science that "provides the basis" distinct from science that "drive the requirements," and that neither of these terms meaningfully distinguishes "pivotal" regulatory science from ordinary regulatory science.
Response: Through its 2020 supplemental notice of proposed rulemaking and this final rule, the EPA has sought to increase the clarity surrounding "science that serves as the basis for informing a significant regulatory action" (proposed as regulatory science) and "pivotal science" (proposed as pivotal regulatory science). The EPA clarifies that the two terms are distinct; while "pivotal science" is a subset of "science that serves as the basis for informing a significant regulatory action," the two sets are not identical. The EPA has further clarified the use of these two terms within the regulatory text, and is finalizing the rule so that "science that serves as the basis for informing a significant regulatory action" is only defined in 40 CFR 30.2 and used in 40 CFR 30.4.
The EPA disagrees that the proposed definition for regulatory science was without meaning, but in response to comments is altering the final definition to increase clarity. For example, the EPA notes that the proposed definition for "regulatory science" combined both general categories of scientific information, such as assessments and models, with specific examples of EPA scientific products, such as criteria documents and regulatory impact analyses. The EPA acknowledges that this may increase confusion and is therefore limiting the final definition to general categories. As such, the EPA is altering the definition of "science that serves as the basis for informing a significant regulatory action" in 40 CFR 30.2 to mean "studies, analyses, models, and assessments of a body of evidence that provide the basis for EPA significant regulatory actions." Examples of models include those used in regulatory impact analyses. Examples of assessments of a body of evidence include risk assessments, hazard identifications, IRIS assessments, and criteria documents.
Other commenters expressed confusion over the scope of what constitutes science that serves as the basis for informing a final significant regulatory action, as defined in the proposed rule. One commenter asserted that the phrase "provides the basis" means that science that serves as the basis for informing a final significant regulatory action could be all the science considered, relied upon, and included in the administrative record of a rulemaking by the EPA. The EPA agrees with this and clarifies in the final rule that the scope of science that serves as the basis for informing a significant regulatory action is equivalent to the science included in the public docket as part of a rulemaking, but not all of that body of science would typically be considered "pivotal science."
On the other hand, "pivotal science" includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). These are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. In the context of this rule, studies informing the dose-response modeling (e.g., benchmark response selection), studies identifying data for toxicokinetic adjustments, or studies informing the selection of uncertainty factors do not drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. Future statute-specific rulemakings may interpret "pivotal science" more broadly.
6.2 SNPRM Pivotal Science
6.2.1 Need for Additional Clarity/Specificity 
Comment: Commenters (10866, 10972, 11332, 11360, 11381, 11391, 11484, 11576, 11894, 12645, 12688, 12715, 12720, 12727, 14397) express that the proposed rule, and specifically the terms "pivotal regulatory science" and "pivotal science," are unclear, as EPA did not detail a list of criteria or specific process that will be used to determine which studies fall under these categories. Commenters (11391, 12720, 12727) also note that this need for additional specificity was identified by the SAB. Specific areas pointed out by commenters as needing clarification include the following:
 How will EPA determine whether science is "pivotal" or "scientifically influential" and how this will be communicated to the public (9362, 11360, 11332, 12645, 12715, 12720, 12727). Commenter (12645) argues that this can only be accurately known in retrospect.
 Who determines whether a study is "pivotal" or "scientifically influential" and whether this is considered final or there will be opportunity to revisit the determination if underlying circumstances change (12720).
 How to interpret the word "underly," as different studies and analyses may inform key scientific information "to differing degrees, depending on study quality, regulatory context, and EPA interpretation of scientific studies" (12720, 12727). 
Response: EPA has described how it will identify pivotal science in the preamble in Section III.C of the final .rule preamble. Specifically, "pivotal science" includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate points of departure (PODs)). EPA has finalized the definition of pivotal science using "drive" instead of "underly". Pivotal science are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. In the context of this rule, studies informing the dose-response modeling (e.g., benchmark response selection), studies identifying data for toxicokinetic adjustments, or studies informing the selection of uncertainty factors do not drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. 
EPA will identify the pivotal science at the proposed rule stage for significant regulatory actions and for influential scientific information when it is submitted for peer review. In both instances, there will be opportunity for public comment on the designation of a study as pivotal science. 
EPA will continue to identify influential scientific information consistent with the OMB Bulletin for Peer Review. To align this rule with the definition of influential scientific information in the OMB Bulletin for Peer Review, the requirements in 40 CFR Part 30 apply to influential scientific information, unless the influential scientific information is exempted from peer review requirements as described in Section IX of the OMB Bulletin for Peer Review.
Comment: Commenters suggest the following revisions to the rule in order to address perceived issues with clarity and/or interpretation of pivotal science:
1) Related to 40 CFR 30.3.
          Clarifying the language in § 30.3, as the comma after "data and models" could be interpreted to mean that the proposed rule would only apply to data and models, whereas commenter (12699) prefers the more expanded interpretation that includes all pivotal science and pivotal regulatory science.
2) Related to the definitions of pivotal science and pivotal regulatory science.
 Using the citation of research within an EPA document on which a regulatory decision is based as an indicator of "pivotal science" or "influential scientific information" (12645).
 Designating information as "pivotal" if it comes from a peer-reviewed scientific publication (9362).
 Defining "pivotal regulatory science" as "the study (or studies if necessary) on which the regulatory limit is based," and requiring EPA to provide a transparent explanation of how and why the selected study or studies were chosen over others (12720).
3) Related to 40 CFR 30.5
 Giving more weight to publicly funded studies than to private studies which use proprietary data and giving peer-reviewed information the highest weighting to avoid favoring privately funded research (9362).
 Establishing criteria for selecting studies for decision-making based on (1) representativeness of the broader body of evidence; (2) usefulness; (3) past efforts to reanalyze, reproduce, and confirm the findings, and (4) public availability of data and methods (11921)
 Deleting the phrase "other things equal" in § 30.5 due to its subjectivity (9362). 
 Clearly stating in § 30.5 that the rule applies to EPA's IRIS program (11499, 12397). 
 Deleting the statement "nonparametric models that incorporate fewer assumptions" in § 30.6 on the grounds that parametric models may be more appropriate in many cases (9362).
 Adding language in § 30.6 "strengthening the concepts that EPA decisions should be evidence-based, pertinent to the issue being addressed and fully transparent, including disclosure of areas of uncertainty and potential confounding of the data" (12397).
Response: Responses to these comments follow.
1) Related to 40 CFR 30.3:
EPA has revised 40 CFR 30.3 to make it clear that the rule applies to science that serves as the basis for informing a significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science .
2) Related to the definitions of pivotal science and pivotal regulatory science:
EPA disagrees that pivotal science should be defined simply based on whether a study has been peer-reviewed or whether it is cited in an EPA document used to support a significant regulatory action. Pivotal science is the subset of the science supporting a significant regulatory action or influential scientific information that drives the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. 
See the description in Section III.E of the preamble of how EPA will identify studies as pivotal science.
3) Related to 40 CFR 30.5:
The EPA will use the following factors to assess the quality of studies identified in the systematic review: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. The EPA will rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence of a relationship between exposure and effect, the EPA will identify those studies based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both of an EPA significant regulatory action or influential scientific information will be identified as pivotal science.
In 40 CFR 30.5 EPA has identified the factors it will consider in determining the degree of consideration to afford pivotal science for which the dose-response data are not available for independent validation. EPA considered the comments and included factors based on usefulness and quality.
EPA has incorporated the language from proposed 40 CFR 30.6 into final 40 CFR 30.5. EPA notes that the language in 40 CFR 30.5 requires a consideration of both parametric and non-parametric models so that EPA can consider both in determining the consideration to give pivotal science for which the dose-response data are not available for independent validation. The commenter has not provided convincing rational on why non-parametric models should not be considered, particularly given that this factor is only one of several considerations in 40 CFR 30.5.
"Other things equal" has been deleted from 40 CFR 30.5.
All IRIS assessments are influential scientific information and thus subject to this rule. Adding a statement that IRIS assessments are subject to the rule would be superfluous.
6.2.2 Scope of Pivotal Science
6.2.2.1 Scope is Too Broad
Comment: Commenters (11390, 11576, 12435, 12546, 12688, 12702, 12727) note that because of the expanded scope of the rule, the lack of clear criteria contained in the proposed rule, and the fact that properly conducted science considers "the entire body of evidence," nearly any study would meet the definition of "pivotal science" or "pivotal regulatory science."
Response: EPA narrowed the scope of the rule from data and models to dose-response data. As EPA has explained in the preamble rather than considering all studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact as "pivotal science," the EPA is balancing transparency and feasibility by focusing on those studies that inform the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect.
6.2.2.2 Support for Expanded Scope
Comment: Commenters (11499, 11500, 11915, 12399) support the expanded scope of the rule to include all "influential scientific information" and "pivotal science."
Response: EPA notes the commenters support of EPA's approach.
Comment: Commenter (11499) supports retroactive application in stating that existing data and models should be covered if they underlie "pivotal regulatory science" in a rulemaking or "pivotal science" in the development of influential scientific information.
Response: This rule applies to dose-response data, regardless of when the dose-response data were generated, that are considered to be pivotal science in a significant regulatory action proposed after the effective date of publication of this rule or in influential scientific information submitted for peer review after the effective date of this rule.

 Chapter 7: Publicly Available
In the notice of proposed rulemaking (NPRM), the EPA used the term "publicly available" but did not define it at 40 Code of Federal Regulations (CFR) 30.2 or in the preamble to the 2018 NPRM. Given its use at 40 CFR 30.1, 30.5 and 30.9, the supplemental notice of proposed rulemaking (SNPRM) proposes a new definition of "publicly available."
As noted in SNPRM preamble, the EPA received a large number of comments stating that the approach in the 2018 NPRM would likely preclude the use of valid data and models from consideration as pivotal regulatory science, because the proposed requirement to make data and models publicly available in a manner sufficient for independent validation would prevent the use of data and models that include ccconfidential bbusiness iinformation (CBI), proprietary data, personally identifiable information (PII) that cannot be sufficiently deidentified to protect the data subjects, as well as many older studies. Based on a consideration of these comments, the EPA is proposing a modified version of the 2018 NPRM regulatory text at 40 CFR 30.5 called Option 1. Option 1 would allow Agency consideration of studies where there is tiered access to data and models that have CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects.
The supplemental proposal also proposes an alternative approach to 40 CFR 30.5 called Option 2. Under Option 2, when promulgating significant regulatory decisions or finalizing influential scientific information, the Agency will, other things equal, give greater consideration to studies where the underlying data and models are available in a manner sufficient for independent validation either because the information is publicly available or available through tiered access when the data include CBI, proprietary data, or PII; and appropriate techniques have been used to reduce the risk of re-identification.
Several comments were received that address Options 1 and 2 together as well as separately. Comments that encompass both Options 1 and 2 are covered under the topic of Broad Issues. Thus, these comments are organized under the following topics:
Section 7.1: Publicly Available, Tiered Access
 40 CFR 30.2 Definition of Publicly Available
 40 CFR 30.5: Broad Issues
 40 CFR 30.5: Option 1
 40 CFR 30.5: Option 2
 Ensure Over Time More Availability of Data
The specific comments provided in this Chapter reflect the comment letters that provide arguments supported by detailed rationale, examples, and references/documentation. The Docket ID numbers for these commenters are cited throughout this Chapter. In addition, some general comment letters may also include comments related to comments included and responded to in this Chapter. These comments are summarized in Appendix B. Note that points and issues raised in these general comment letters may also be raised by the more detailed comments summarized in this Chapter. The responses provided in this Chapter adequately address these more general comments.
7.1 Publicly Available, Tiered Access 
7.1.1 40 CFR 30.2 Definition of Publicly Available
The supplemental notice of proposed rulemaking (SNPRM, or supplemental proposal) rule adds the following definition to section 30.2: 
      Publicly available means "lawfully available to the general public from federal, state, or local government records; the internet; widely distributed media; or disclosures to the general public that are required to be made by federal, state, or local law. The public must be able to access the information on the date of publication of the proposed rule for the significant regulatory action or dissemination of the draft influential scientific information for public review and comment."
7.1.1 Clarification
Comment: Commenter (11401) supports the clarity achieved by defining publicly available.
Conversely, commenters (9587, 11381, 12720) suggest the EPA clarify the definition of publicly available. Commenter (9587) notes that the EPA would be well-served by a systematic reform to provide clear definitions of all terms relevant to policymaking and suggests that the EPA, following government best practices in the Department of Defense, institute a formal ontology to establish terminological exactitude in all policymaking.
Commenter (11381) states that the definition of publicly available in the supplemental notice of proposed rulemaking (SNPRM) is unclear. The commenter notes that it is different from the Information Quality Guidelines (IQG) term dissemination in potentially confusing ways, apparently because the Agency is trying to make the term perform dual purposes. The commenter states that, under the IQGs, dissemination is triggered only by agency action. Information an agency does not disseminate is not covered, and thus need not meet information quality standards. The commenter states that the term does not apply to what information an agency might consider, evaluate, or analyze. The commenter states that, in contrast, publicly available appears to act as both a substitute for dissemination and a filter governing what information the EPA will consider, evaluate, or analyze. The commenter states that the SNPRM uses the term publicly available in a manner resembling dissemination, but it also uses publicly available as a screen on what the Agency will consider to be pivotal regulatory science and/or pivotal science. 
Commenter (12720) states that the SNPRM's definition of "publicly available" data is vague in not making it clear whether the information itself would be actively made available to members of the public by data holders in government sources, media sources, or other online sources, or whether data subject to public distribution from government sources through legal approaches, including data requests submitted to EPA via the Freedom Of Information Act (FOIA) regardless of actual distribution, would also be subject to this definition. 
Response: The term "publicly available" is not meant to convey that the EPA is responsible for disseminating dose-response data considered pivotal science under the final rule. Therefore, the definition of "disseminate" in the IQG is not relevant.
The final rule is limited to dose-response data and does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. The final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science. EPA does not intend for the term "publicly available" to resemble "dissemination."
Comment: Commenter (11479) states that, in this definition [publicly available], the EPA wrongly applies a catch-all approach to transparency, as a recent paper in Environmental Health Perspectives points out:
      "Strategies designed to serve some goals of the open science movement may be inadequate for achieving other goals. For example, efforts to provide access to all the data underlying a study may be helpful for enabling other scientists to reanalyze the results and identify weaknesses that could undermine its reproducibility. However, providing all that data may do little to help citizens or policymakers understand the overall significance of the study or the crucial limitations or weaknesses associated with it...Additional strategies -- such as effective two-way communication between the scientists who designed the study and those using the information -- might be needed to clarify the information needed by decision makers." 
Response: The EPA agrees that subject matter experts are key to effectively determining if the results of a study can be produced when the study data are reanalyzed. For that reason, "independent validation" was defined in 40 CFR 30.2 as "the reanalysis of study dose-response data by subject matter experts who have not contributed to the development of the study to evaluate whether results similar to those reported in the study are produced" (emphasis added). However, the EPA maintains that underlying dose-response data should be available to a wide audience. Therefore, the EPA is also maintaining the definition of "publicly available." 
The final rule is limited to dose-response data and does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. The final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science. 
Comment: Commenter (12731) states that, in general scientific usage, "publicly available" means the data, methods, results and summary evaluation are available in a usual form (e.g., a peer-reviewed scientific publication or a technical report). The commenter states that the definition proposed by EPA goes well beyond the generally accepted scientific definition and is likely to mislead the public into believing that the Agency's definition is the scientific norm.
Response: As discussed in the preamble to the SNPRM, the EPA relied upon the definition of "publicly available" found in 16 CFR 313.3. There does not appear to be a published standard definition of "publicly available" in the scientific community so the definition from 16 CFR 313.3 was utilized. 
Further, in this final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. As noted by commenters, the step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
Comment: Commenter (10020) states that the proposed definition is ambiguous, misleading and unworkable. The commenter asserts that the definition must incorporate contrasting definitions to clarify its meaning and make the proposed new definition workable. The commenter suggests the proposed new definition of "publicly available" should include a contrasting definition similar to the definition of "nonpublic personal information" found in 16 CFR section 313.3(n). The commenter states that, without a contrasting definition or definitions as are found in section 313.3, the proposed new definition is devoid of meaning. 
Commenter (10020) does not agree with the proposal's suggestion that its definition of "publicly available" is "similar" to the definition of "publicly available" at 16 CFR section 313.3. The commenter states that 16 CFR section 313.3 was carefully crafted to include a contrasting definition of "nonpublic personal information" which your new proposed definition does not. The commenter states that the contrasting definition gives clarity and meaning to the definition of "publicly available". 
Commenter (10020) states that the proposal's stated reason for 16 CFR section 313.3 as the basis for your new proposed definition is that "the meaning of information that is available to the general public should not vary." The commenter states that proposed new definition varies "the meaning of information that is available to the general public" contrary to your announced intention, which is quite correct, that the meaning in the Code of Federal Regulations "should not vary." The commenter states that the new proposed definition of "publicly available" ought to simply and expressly incorporate 16 CFR section 313.3(p) by reference. The commenter states that the proposed new rule defining "publicly available" would serve no purpose since there is already a perfectly workable definition of "publicly available" found in 16 CFR section 313.3(p). 
Response: The EPA disagrees that a contrasting definition is needed or that incorporation of 16 CFR 313(p) by reference is needed. Section 30.2 of the final rule provides the following definition:
Publicly available means "lawfully available to the general public from federal, state, or local government records; the internet; widely distributed media; or disclosures to the general public that are required to be made by federal, state, or local law. The public must be able to access the information on the date of publication of the proposed rule (or, as appropriate, a supplemental notice of proposed rulemaking, or notice of availability) for the significant regulatory action or on the date of dissemination of the draft influential scientific information for public review and comment."
The final rule is limited to dose-response data and does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. The final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
The EPA has finalized the approach that will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. Restricted or tiered access in this final rule means that the underlying dose-response data are be available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
7.1.1.2 FOIA
Comment: Commenters (11192, 11361, 12720, 12464, 12715) express concern that the proposal's definition of "publicly available" in the SNPRM includes information available through the FOIA, which means the studies and data may not be immediately or easily available to the public. The commenters assert that the FOIA is not a reasonable tool for making data publicly available because the use of such a data release mechanism cannot ensure equitable access, could involve significant costs, would complicate the existing system due to an anticipated surge in FOIA requests for underlying data and models used in pivotal regulatory science, and/or would involve long processing times with comparatively short public comment periods for EPA regulatory proposals.
Commenters (11192, 11361) state that, while most government records are legally available through the FOIA, anyone who has filed a FOIA request knows that the actual availability of information can be extraordinarily laborious and expensive to obtain. Commenter (12720) states that the FOIA process could involve significant costs. Commenter (12715) states that, although the data may technically be "available," it is likely to be difficult and costly to obtain. 
Commenter (12720) states that the FOIA system at EPA and other federal agencies is widely known to be dysfunctional and slow, with thousands of pending requests in the backlog each year. Commenter (12464) states that once someone files a FOIA request, it may take months or years for an agency to provide all requested documents, and frequently, an agency will deny a request or miss statutory deadlines to respond, meaning that the requestor must resort to the courts to obtain the desired documents. 
Commenters (12720, 12715) express concern that the FOIA process involves long processing times and comparatively short public comment periods for EPA regulatory proposals. 
Commenter (12715) states that the data often may not be received by the requestor until after the comment period ends, if at all, in which case, it is unlikely that interested parties would be able to obtain the information in a manner timely enough to allow for their own review and comment within any timeframe designated by EPA.
Response: EPA agrees that FOIA mechanisms do not achieve the goals of this rule. Availability via FOIA mechanisms does not mean that these data are immediately or easily available to the public. Some commenters cited the EPA's Freedom of Information Act Annual Report Fiscal Year 2019 (2020), which lists a median response time for "expedited processing" of FOIA requests by the EPA as 493 days (Ref. 42). Also, FOIA applies to federal agencies and cannot be used to compel other entities to release information. 
The concern regarding the long processing time for FOIA requests has been addressed in the final rule by the addition of the following sentence to the definition of publicly available: "The public must be able to access the information on the date of publication of the proposed rule for the significant regulatory action or dissemination of the draft influential scientific information for public review and comment."
Comment: Commenter (12720) states that the EPA has not explained how its SNPRM would affect the existing FOIA activities at the Agency, despite the potential for major disruption of the process if members of the public are expected to use FOIA requests in order to obtain "information legally available from government sources" as it pertains to this proposal. The commenter states that the EPA has not described any modification of the online FOIA request and response system to accommodate a significant volume of requests for all "data and models" underlying specific scientific studies. 
Response: The final rule will not affect the Agency's existing FOIA activities. The final rule does not require EPA to obtain, store or publish dose-response data. The rule merely provides that "[t]he public must be able to access the information on the date of publication of the proposed rule for the significant regulatory action or dissemination of the draft influential scientific information for public review and comment." If the dose-response data underlying pivotal science is not available, those studies will receive lesser consideration under section 30.5 of the final rule.
Comment: Commenter (12720) asserts that there are major flaws which arise from EPA's proposed definition of "publicly available" in the context of FOIA. The commenter states that it is not clear that the EPA would itself be the custodian of "data and models" subject to this rule, nor how the EPA would ensure sufficient data security, integrity, and sharing mechanisms. The commenter states that it is also not clear how members of the public could access "information legally available from government sources" given the volume and complexity of the underlying "data and models" that fall under the scope of this proposal. The commenter asks if members of the public will be expected to identify specific studies for release under FOIA and how would the EPA evaluate such requests. The commenter states that it is likely that, upon receipt of such a request, the EPA would need to contact and potentially negotiate a public data release with study researchers and actual study participants, in the case of research analyzing human exposures and health responses. The commenter states that it is far from clear that such a FOIA request could be fulfilled by EPA, given existing legal restrictions on data sharing that are characteristic of many studies involving human subjects. The commenter notes that the EPA's own interpretation states that FOIA "does not require agencies responding to requests to conduct research or analyze data." 
Response: The final rule will not affect the Agency's existing FOIA activities. The rule merely provides that "[t]he public must be able to access the information on the date of publication of the proposed rule (or, as appropriate, a supplemental notice of proposed rulemaking, or notice of availability) for the significant regulatory action or on the date of dissemination of the draft influential scientific information for public review and comment." The final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. This is a rule of internal procedure that does not impose requirements on any party other than the EPA. This rule imposes no costs on researchers or their institutions, and the EPA will consider and evaluate all relevant and appropriate science in its significant regulatory actions and influential scientific information.
Section 30.4 of the final rule provides that the EPA "shall clearly identify all science that serves as the basis for informing a significant regulatory action. The EPA shall make all such science that serves as the basis for informing a significant regulatory action publicly available to the extent permitted by law." Members of the public will not need to submit FOIA requests to gain access to the studies EPA relies on. 
The EPA recognizes that researchers and other third parties may voluntarily consider the EPA's position on data availability, as described in this rule, as they make their own decisions about how to conduct research and the extent to which they make data and models available. Researchers may choose to make more data and models available, but the EPA recognizes that these parties will weigh their own benefits and costs and make choices that they deem appropriate.
Comment: Commenters (11192, 11361) state that studies originating outside of the government would not be subject to FOIA, so there would need to be another system for creating public access. The commenters state that there is currently no central place for academics to publish their data, let alone to publish data with tiered access. 
Response: The final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which data and models are available. This is a rule of internal procedure that does not impose requirements on any party other than the EPA. This rule imposes no costs on researchers or their institutions, and the EPA will consider and evaluate all relevant and appropriate science in its significant regulatory actions and influential scientific information. The EPA recognizes that researchers and other third parties may voluntarily consider the EPA's position on data availability, as described in this rule, when making their own decisions about how to conduct research and the extent to which they make data and models available. Researchers may choose to make more dose-response data available, but the EPA recognizes that these parties will weigh their own benefits and costs and make choices that they deem appropriate. Further, the EPA recognizes that there are currently many options related to data sharing mechanisms and repositories for researchers. The rule does not prescribe a type of mechanism, but just requires the EPA to determine whether dose-response data is available in a manner sufficient for independent validation.
Comment: Commenters (12720, 12464, 12715) express concern that the proposal's definition of "publicly available" in the SNPRM could systematically favor the EPA's use of industry-submitted data. 
Commenter (12720) states that, in 2019, the EPA finalized an update to its FOIA regulations. The commenter states that, under this rule, the Administrator and other officials are allowed to review all materials that fit a FOIA request criteria and can decide whether to release or withhold a record or a portion of a record on the basis of responsiveness or under one or more exemptions under the FOIA, and to issue no records responses. The commenter states that, given the vague language in this proposal and existing FOIA regulations at EPA, there is a significant potential for political interference and arbitrary determinations within EPA for public release of data and models. 
Commenter (12464) notes that the definition provides that "publicly available" means, among other things, "lawfully available to the general public from federal, state, or local government records." The commenter states that this means that data submitted by industry when applying to state or federal agencies for a permit or approval will be treated as "publicly available" for purposes of the proposal, even if members of the public could obtain access to those data only by filing a FOIA or state public records law request. 
Commenter (12715) states that the limitations on the actual ability for requestors to obtain data and documents upon which EPA relies increase the likelihood that when EPA adopts regulations or standards, it will consider studies or data that are not in any practical manner available to the public, while at the same time failing or refusing to consider peer-reviewed studies that have been published in the open literature but which have private underlying data. 
Response: These comments are addressed in the preamble of the final rule:
      Several commenters contended that the proposed definition of "publicly available" would introduce a bias favoring industry data submitted to the EPA. They asserted that industry-generated studies submitted to the EPA pursuant to FIFRA would be considered publicly available because they could be obtained by the public in response to a Freedom of Information Act (FOIA) request. However, this does not mean that these are immediately or easily available to the public. Some commenters cited the EPA's Freedom of Information Act Annual Report Fiscal Year 2019 (2020), which lists a median response time for "expedited processing" of FOIA requests by the EPA as 493 days (Ref. 42). The EPA finds that such comments have merit and is modifying the definition in the final rule to add the following at the end of the definition: "The public must be able to access the information on the date of publication of the proposed rule (or, as appropriate, a supplemental notice of proposed rulemaking, or notice of availability) for the significant regulatory action or on the date of dissemination of the draft influential scientific information for public review and comment."
7.1.1.3 Other
Comment: Commenters (11576, 12720) share the concerns that the Science Advisory Board (SAB) outlined with respect to what "available" means in the context of the supplemental notice:
      A question to be answered is whether making the scientific papers reporting these studies available without charge makes the studies "available," or whether all data from every measurement taken as part of the study need to be available to anyone to analyze. At one
end of this range of interpretation the requirement is easily implementable. On the other end of the spectrum, meeting the requirement would be enormously expensive and time consuming at best and could be expected to result in the exclusion of much of the scientific literature from consideration (the machine data may no longer be available and/or the researchers may no longer be alive or in a position to assemble the data).
Response: In this final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. As noted by commenters, the step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
The EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. The final rule includes sufficient flexibility in terms of the allowance for restricted access, considerations in 40 CFR 30.5, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data to minimize potential unintended consequences
Comment: Commenter (12712) asserts that the default rule for all work funded by the EPA must be that relevant code and data are available online or on request. The commenter states that the EPA must be clear that making data available means that the actual final dataset used in statistical analysis is made available. 
Response: EPA has implemented the public access requirements for EPA researchers, grantees and contractors as required by the February 22, 2013, OSTP memorandum "Increasing Access to the Results of Federally Funded Scientific Research" and the 2016 EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research." EPA researchers, grantees, and contractors are required to make peer-reviewed journal articles and the underlying data and models resulting from EPA-funded research accessible to the public.
The EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research" requires the data underlying a journal article resulting from research funded by EPA be made publicly accessible. However, the rule requires the EPA to evaluate the availability of the analyzable dataset, not the more limited dataset that accompanied the publication of the study or model. For this rulemaking EPA will not identify a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. The step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
Comment: Commenter (12406) states that the proposed definition regarding "publicly available" does not include any reference to the concept of tiered access to CBI and PII as a permissible and appropriate means by which information could be deemed within the definition of publicly available. The commenter suggests this definition should be revised as follows:
      Publicly available means lawfully available to the general public from federal, state, or local government records; the internet; widely distributed media; in designated reading rooms; available through tiered access when the data includes CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects; or disclosures to the general public that are required to be made by federal, state, or local law. 
Response: The EPA does not agree that restricted access should be a part of the definition of "publicly available" and therefore, EPA has not made the suggested edits to that definition. Restricted access is covered in other provisions of the rule. The EPA has finalized the approach that will give greater consideration to pivotal science whose for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. Restricted or tiered access in this final rule means that the underlying dose-response data or models are be available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (12720) notes that the SAB report finds that requiring the identification of all studies and regulatory science supporting regulatory actions and making them available for reanalysis will be a complex process. The commenter contends that identifying and making "pivotal science data or studies" available could present challenges if some studies were considered in the regulatory decisions but were not used to determine the point of departure or reference dose or other regulatory/technical level. The commenter states that it is not clear how much information and which studies should be included in the requirement to identify and make studies available.
Response: In the final rule, pivotal science "includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). These are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information." The final rule does not require EPA or any other entity to conduct independent validation before EPA can use pivotal science. Nor does the final rule require EPA to obtain, store or publish dose-response data.
Comment: Commenter (12720) suggests that the rule could benefit from use of the term "analysis dataset" to define data that should be made publicly available. The commenter states that this term refers to data that have been collected and processed (e.g., cleaned and transformed) for analysis. The commenter states that, in this way, the public could understand which studies/data were considered and used without expending the resources to gather and disseminate all available data. The commenter states that the level of detail required to allow the public to transparently reach the same conclusion(s) as the Agency will differ among individuals who seek the data. The commenter states that some people may wish to verify that the EPA has selected the right "pivotal studies." The commenter states that, however, a more expansive scope would increase the reporting burden in a way that may make the proposed rule untenable.
Response: In this final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. As noted by commenters, the step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information. 
7.1.2 40 CFR 30.5: Broad Issues
7.1.2.1. Retroactive Application of the Rule
7.1.2.1.1 General 
Comment: Commenters (11393, 11395, 11407, 11895, 12412, 12474, 13549) generally oppose the proposal that the rule would apply regardless of when the data and models were generated. 
Commenters (11393,11395, 12412, 12435, 12474, 13549) state that many environmental statutes require the EPA to promulgate standards and protections based on the best available science, and then reanalyze those decisions at set points in the future. The commenters state that the SNPRM, and its proposal to apply regardless of when the data and models were generated, suggests that as the EPA reexamines standards and protections, the Agency would potentially exclude from consideration studies that led it to enact those standards and protections in the first place. The commenters state that it would be nonsensical and unlawful for EPA to overturn or weaken a standard or protection not based on any new science, or on a concern that the results were invalid, but rather because that study has not publicly released all its underlying data. 
Commenter (11401) states that, while it is reasonable not to apply the new standard to existing rules, it would be a mistake to perpetuate outdated, substandard scientific studies into the future. The commenter states that these standards should apply to any science that a new rulemaking is using, regardless of when the science was generated; perpetuating non-transparent, irreproducible studies potentially in perpetuity should not be the goal of this rulemaking. 
Commenter (11894) states that the supplemental proposal attempts to have it both ways by maintaining that the rule would apply retroactively while requesting comments on whether this should apply only to data and models that are generated after the effective date of the rulemaking.
Response: The EPA is maintaining that the rule provisions apply equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that are unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator, as detailed under 40 CFR 30.7. One of the criteria the Administrator may use in his/her determination to grant a waiver is that the development of the dose-response data was completed or updated before the effective date of the rule. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
The considerations in 40 CFR 30.5, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data are intended to minimize potential unintended consequences, such as the potential skew toward newer, domestic, and/or industry-funded studies suggested by the commenters. In addition, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. The EPA believes the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making.
7.1.2.1.2 Planning
Comment: Commenters (10339, 11890, 12430, 12720, 12715) express concern that the rule and SNPRM fail to explain how historical data, which may have been collected under different policies and procedures to ensure confidentiality, will be treated. Commenter (11890) states that older records may have lost underlying data due to record retention schedules.
Commenter (12430) states that the EPA is in the process of adopting, and has recently adopted, multiple rules that will have a significant negative impact on air quality and human health, and it supports these rulemakings with information that may not meet the proposed standards for public availability. The commenter states that, for example, in the Safer Affordable Fuel-Efficient Vehicles Rule Part One: One National Program, the EPA notes that some information relied upon in the rulemaking is not publicly available, e.g., CBI. The commenter states that the EPA has similarly relied upon information not publicly available in the "Safer Affordable Fuel-Efficient Vehicles Rule for Model Years 2021-2026 Passenger Cars and Light Trucks." The commenter states that the EPA does not explain the extent to which the SNPRM would prevent the Agency from relying upon non-public information in other concurrent or near-term rulemakings. The commenter states that it is an open question, then, whether the EPA would need to amend its open rulemakings to eliminate the non-public information that it is proposing to limit itself from considering if the SNPRM is finalized  -  or whether the EPA would simply find reasons to exempt its preferred studies from the SNPRM's requirements under its arbitrary exemption provision. The commenter states that retroactive application is even more problematic than prospective-only application, but either approach would abrogate EPA's duties to protect public health and the environment.
Commenter (12720) asserts that the retrospective application of data transparency standards is a monumental challenge. The commenter states that a large amount of work would be required to locate, curate and retrospectively make datasets available for public access. The commenter states that this requirement could adversely affect the ability to move this program forward in a meaningful capacity. The commenter states that, for example, the EPA could decide not to apply the proposed rule and its specific requirements retrospectively given the potential difficulty accessing, reviewing and making data available that were not originally intended to be disseminated in such a manner as defined in a future rule. The commenter states that the EPA could consider designating a "start date" and begin collecting and releasing pivotal study data at that time. The commenter states that, when the EPA updates an existing risk assessment after the start date, the EPA could collect and release pivotal data. The commenter states that it is true that a substantial amount of work would be required in order to make any data and models underlying studies available to the public, but the EPA has not identified a responsible party for this work, nor has it made clear the timelines, electronic data sharing mechanisms, or public reporting of such availability would be achieved, archived, and maintained over time. The commenter states that it is not clear how such a "start date" for future applicability of data requirements could be determined or fairly applied, and in many cases such as the national ambient air quality standards (NAAQS) science reviews, the assessments rely on a foundation of scientific evidence that stretches back many years, if not decades. The commenter states that, as a result, determination of any "starting date" would be arbitrary and deprive the EPA of the best available science.
Further, the commenter (12720) contends that EPA has not acknowledged that there are regular review cycles for different environmental rules, and that new requirements for all "data and models" to be considered by the Agency in development of "influential scientific information and/or pivotal regulatory science" will substantially complicate existing science review processes at the Agency. The commenter contends that the supplemental proposal makes no reference to the fact that EPA is constantly re-reviewing scientific evidence, and that the body of science remains relevant regardless of when it was generated. Additionally, the commenter states that EPA has not explained why retroactive application is warranted, even as the Agency proceeds with a number of influential science products that rely on studies that do not meet the requirements of the supplemental proposal.
Commenter (12715) states that the proposal at 15,399 and 15,403 is unclear whether studies with data and models generated before the effective date of this proposed rule would be subject to the 40 CFR 30.5 criteria.
Commenter (11392) asserts that the supplemental proposal's retroactive application could have radical consequences and that EPA must provide greater clarity on what the implications would be for past studies that underpin the Agency's regulatory standards.
Response: The EPA is maintaining that the rule provisions apply equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that are unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator, as detailed under 40 CFR 30.7. One of the criteria the Administrator may use in his/her determination to grant a waiver is that the development of the dose-response data was completed or updated before the effective date of the rule. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
The considerations in 40 CFR 30.5, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data are intended to minimize potential unintended consequences, such as the potential skew toward newer, domestic, and/or industry-funded studies suggested by the commenters. In addition, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. The EPA believes the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making.
7.1.2.1.3 Costs
Comment: Commenters (11890, 12731) express concern regarding the costs associated with any retrospective application of the rule.
Commenter (11890) states that scientists who volunteer to provide data would need funding to respond to requests from EPA. The commenter states that this rule should not apply retroactively. The commenter states that archived materials were likely collected under grants that have long expired or are funded by grants that can't be used to fund such activities. 
Commenter (11892) states it is difficult to envision how the supplemental proposal could be applied retroactively to the science developed through long term experience in the implementation of various regulations and standards. The commenter asserts that the exclusion of research that was accepted as scientifically valid in the past could only result in the need to duplicate such research, adding needless cost and delay to the process of decision making. Furthermore, according to the commenter, there are already EPA mechanisms for revising regulations and standards based on new data and analyses.
Commenter (12731) refers to the SAB report which concluded, "the retrospective application of modern transparency standards is a challenge. A large amount of work would be required to locate, curate and retrospectively make datasets available for public access. This requirement could adversely affect the ability to move this program forward in a meaningful capacity."
Response: The EPA has identified some incremental costs that the Agency may incur as a result of this final rule. As stated in Section III.A.2, the EPA will continue its current practice of conducting extensive review of scientific studies during the development of significant regulatory actions and influential scientific information. The additional procedures required by this rule only apply to pivotal science, which is a subset of the total number of studies that the EPA would evaluate. Given the costs of the current robust process for identifying and reviewing scientific studies and documentation that are existing Agency practice, as well as that the determination of dose-response data availability is limited to pivotal science underlying significant regulatory actions and influential scientific information, the EPA anticipates that the incremental costs of this rule will be small. The Agency may also incur other administrative costs to perform analyses and evaluations to support activities such as exemption decisions made by the Administrator, and documenting these or other decisions made pursuant to the requirements of the final rule. Again, the Agency anticipates that the incremental costs for these activities will be small relative to current administrative costs for developing significant regulatory actions or influential scientific information. Finally, this final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
7.1.2.1.4 Consent Needed
Comment: Commenters (11361, 12430, 12720, 12731, 12727) express concern that it may not be possible to obtain consent from study authors and/or data owners of historical studies. 
Commenter (11361) states that any study that includes primary collection of human health data would need to have obtained informed consent from subjects to participate in the study. The commenter states that consent agreements are created to be very specific, scope- and time-limited agreements. The commenter states that, for any study that did not originally make its data public--which describes the vast majority of studies conducted--participants haven't consented to their data being made public, even if a de-identification process has taken place. The commenter states that it could be an enormous undertaking to try to track down participants years after a study concludes to attempt to gain consent in order to comply with this rule. The commenter states that, since many consent agreements are time-limited, researchers may not have the consent of participants to even attempt to contact them after the conclusion of the study. The commenter states that the EPA is leading researchers into a legal, ethical, and financial quagmire by retroactively dismissing or devaluing existing studies that do not comply with these unrealistic requirements.
Commenter (12430) states that participation in epidemiological studies is often contingent on keeping subject data confidential, consequently, studies that have already been completed may contain data that was collected contingent on confidentiality. The commenter states that researchers cannot now make that data publicly available, in response to the SNPRM, without violating state and/or federal laws (e.g., California Information Practices Act, the Health Insurance Portability and Accountability Act) or locating research participants and requesting their consent to the release (an expensive and time-consuming process that may frequently result in participants declining to give their consent). 
Commenter (12720) expresses concern that the proposal does not explain how permissions will be obtained from study authors and research participants. The commenter states that this shortcoming raises numerous questions, including the length of time that will be allowed for study participants to consider and respond to any requests for data release. 
Commenters (12731, 12727) refer to Dr. Janice Chambers' comment in response to EPA's charge questions that even a "tiered access" approach to disclosing data would not be possible to apply retroactively to studies carried out in the past. The commenters state that, based on her personal experience, Dr. Chambers opined that "many of the older epidemiology studies would not have considered this Option in their Informed Consent Forms" and that the researchers who conducted those studies (assuming they are even available) would therefore not have consent to share the data for reanalysis. The commenters note that participants in past health studies may either have died or be unable to provide consent due to age or illness -- and that even participants who are reachable may be unwilling to release subsets of their data even to "safe harbor data archives."
Commenter (11900) states that a participant's right to privacy is established during informed consent to participate in a study. The commenter states that informed consent is an ethical and legal requirement of human subject research. The commenter states that consent documents describe the nature of the research, the risks and benefits associated with participation, and how the researcher will protect the confidentiality of the research records. The commenter states that, up to the present, nearly all human environmental health studies have guaranteed in their informed consent not to share identifiable study data outside the research team. The commenter states that all human subjects data is potentially identifiable, even after overt identifiers are removed; thus, researchers will be put in the position of having to decide whether placing data in a restricted-access enclave -- subject to access requirements that are not yet specified -- violates their pre-existing pledge to participants. 
Response: The final rule does not require EPA or any other entity to obtain, store or publish dose-response data. As provided in 30.5(f), "[w]here the Agency is making dose-response data publicly available, it shall do so in a fashion that is consistent with law, protects privacy, confidentiality, CBICBI, and is sensitive to national security." Dose-response data is considered "publicly available in a manner sufficient for independent validation" when it includes the information necessary for the public to understand, assess, and reanalyze findings and may include, for example:
 Data (data would be made available subject to access and use restrictions);
 Associated protocols necessary to understand, assess, and extend conclusions;
 Computer codes and models involved in the creation and analysis of such information;
 Recorded factual materials; and
 Detailed descriptions of how to access and use such information. 
The provisions of this section apply to dose-response data underlying studies that are pivotal science, regardless of who funded or conducted the studies. The Agency shall make all reasonable efforts to explore methodologies, technologies, and institutional arrangements for making such data available before it concludes that doing so in a manner consistent with law and protection of privacy, confidentiality, national security is not possible. Further, an exemption under 30.7 may be granted so that pivotal science where the underlying data are not available for independent validation may be used and granted greater consideration under 30.5.
Comment: Commenters (11890, 12418) argue that, to protect privacy, the rule should not be retroactive. Commenter (11890) states that new technology has enabled re-identification of study participants and studies considered de-identified 20 years ago would be vulnerable to breaches in privacy today. 
Commenter (12418) agrees with the modification in the supplemental proposal that all studies completed before the effective date of this proposed rule are considered eligible for EPA rulemaking regardless of whether they contain data or models that are publicly available. The commenter states that the EPA must acknowledge that protecting the privacy of human participants in studies which have been used to support regulations in the past is an essential justification for prohibiting application prior to the effective date of this rulemaking. The commenter states that confidentiality of participant information is a scientific imperative of human health research that this rule must address more directly. 
Response: The EPA is maintaining that the rule provisions apply equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that are unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator, as detailed under 40 CFR 30.7. One of the criteria the Administrator may use in his/her determination to grant a waiver is that the development of the dose-response data was completed or updated before the effective date of the rule. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
The considerations in 40 CFR 30.5, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data are intended to minimize potential unintended consequences, such as the potential skew toward newer, domestic, and/or industry-funded studies suggested by the commenters. In addition, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. The EPA believes the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making.
7.1.2.1.5 High Quality Research
Comment: Commenters (10973, 11361, 12617, 12731, 13788, 14397, 11900) object to the exclusion of older research simply because it does not have full data sets available. Commenter (11361) states that, by retroactively applying this rule, the EPA nearly guarantees that most existing public health studies would be stricken from consideration. 
Commenter (12731) asserts that the Agency's proposed solutions do nothing to allow rigorous consideration of the vast body of historical studies that document the harm that pollutants cause to human health and welfare. The commenter states that, while the EPA acknowledges objections to the original proposal's exclusion of "many older studies" that include valid data and models (85 FR 15401-02), the SNPRM does not respond to these concerns. The commenter states that, as the EPA admits, there are compelling reasons why it may be impossible or infeasible to provide access to data and models underlying older studies. 
Commenter (14397) states that this proposal could allow the EPA to retroactively dismiss studies that have been used for decades when setting or renewing NAAQS and related regulations. The commenter states that these studies could include seminal publications regarding lead and mercury, and particulate matter., The commenter states that it should be noted that the EPA is not suggesting that past studies are invalid or reached erroneous conclusions, and indeed many of these studies have been reviewed, replicated, and validated by further independent studies. The commenter states that the EPA simply wants these studies to be dismissed because properly confidential data cannot be released. The commenter states that this approach would force the EPA to arbitrarily make public health and environmental decisions based on incomplete or improperly weighted information.
Response: The EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. As described in 40 CFR 30.5 of the final rule, the EPA will rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies -- based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. Further, because study quality factors (including soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review) would have already been evaluated at an earlier stage in the assessment process, the studies evaluated with these additional factors are presumed to be of the highest quality available.
Comment: Commenter (12423) understands that the EPA interprets the proposed rule as not retroactive, but asserts that interpretation is misleading. The commenter states that any future regulations or policies would need to adhere to this proposed rule, including the review of existing regulations and policies. The commenter states that, without the effective ability to tier access these data under the proposed rule, an important study would be either discounted or deprioritized, depending on whether the modified proposal (tiered access) or alternative proposal (weighted studies) for public data is applied. The commenter states that actions and decisions made by EPA then could not make use of this scientifically valid and critical study without an exemption by the Administrator.
Commenters (12403, 11900) express concern that retroactive application of the proposed rule would be detrimental to public health.
Response: This final rule applies prospectively to significant regulatory actions and influential scientific information developed after the effective date of the final rule and has no retrospective effect on existing significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. 
The EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. Where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies -- based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. Further, because study quality factors (including soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review) would have already been evaluated at an earlier stage in the assessment process, the studies evaluated with these additional factors are presumed to be of the highest quality available.
Comment: Commenter (12455) provides that the supplemental proposal needlessly complicates their use of pesticide reviews, which are traditionally based on CBI studies. According to the commenter, it is unclear whether older studies, when they come up for review, will be completely excluded from consideration as they would not qualify under the supplemental proposal. Commenter (12715) states that Option 1 is fatally flawed in that it continues to unlawfully and arbitrarily exclude from consideration valid scientific studies where the underlying data is unavailable, regardless of whether the studies have been peer reviewed or would be considered part of the "best available science" the Agency is commanded to consider.
Response: The EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. With this final rule, the EPA is maintaining language in 40 CFR 30.3 stating that the statutes that the EPA administers, or their implementing regulations, will control in the event of any conflicts with the requirements of this rule. The Agency will continue to comply with and abide by the requirements in those statutes and implementing regulations, including regulatory deadlines. 
7.1.2.2 Best Available Science
7.1.2.2.1 Legal Concerns
Statutes
Comment: Commenters (11360, 11390, 11393, 11396, 11403, 11407, 11489, 11491, 11895, 11911, 12412, 12474, 12702, 12731, 13549, 13788, 12715) assert that the proposal violates statutes that require the EPA to base its decisions on the best available science. 
Commenter (11360) states that the proposed rule and the SNPRM in effect bar the EPA from utilizing scientific information that rests on difficult-to-release information. The commenter states that the best available science must serve as the foundation for EPA's regulatory and policy actions. The commenter states that Congress intentionally and wisely embedded peer-reviewed research in the foundation of the Clean Air Act (CAA), including requiring regular reviews of the science, explicitly recognizing that the EPA needs the most current, peer-reviewed data to protect the public health. The commenter states that these expectations also are reflected in other public health laws that support the EPA's mission, including the Toxic Substances Control Act (TSCA).
Commenters (11393, 11407, 11895, 12412, 12474, 13549) assert that the EPA must base its decisions on the best available science and that anything less is nonsensical, arbitrary, and in violation of the environmental laws the EPA is charged with implementing. The commenters state that the SNPRM instead provides for ways the EPA would exclude and lessen consideration of the best available science. 
Commenter (11390) asserts that weighting, discounting, or excluding relevant studies from consideration based not on the quality of the science but on the public availability of underlying data directly contradicts the EPA's statutory mandates and core mission. The commenter states that the EPA itself has defined "best available science" and similar statutory standards without reference to the public availability of data; for example, the EPA has explained that:
      Use of best available science involves the use of supporting studies conducted in accordance with sound and objective scientific practices, including, when available, peer reviewed science and supporting studies and data collected by accepted methods or best available methods (if the reliability of the method and the nature of the decision justifies use of the data).
Commenter (11489) asserts that science does not depend on the public availability of underlying data to indicate quality and reliability of evidence and public availability of research data is not a proxy for the reproducibility of science. The commenter states that, if this rule is made final, the EPA will fail to meet a key component in its enabling legislation that requires the agency use the "best available science" in its regulatory decisions.
Commenter (11491) states that the EPA is often legally obligated to base its rules on the best available science, and the Agency does not have the authority to modify statutes requiring the Agency to use the best available science and health information.
Commenter (11911) asserts that ignoring valid and potentially valuable information in rulemaking with profound public health and economic consequences is a steep cost to pay  -  and it is at odds with the Agency's mandate to use the "best available" science. 
Commenter (12731) asserts that, like the original proposal, the SNPRM lacks any demonstration that the public unavailability of a study's underlying data or models necessarily -- or even likely -- renders the study invalid or unreliable. The commenter notes that comments on the original proposal demonstrate that mechanisms are already in place to validate studies for which disclosure of underlying data and models is either illegal or impracticable. The commenter contends that the EPA's automatic exclusion from consideration of (or, at least, diminished reliance upon) studies based solely on the public unavailability of underlying data or models would result in the Agency failing to consider all available science -- and even the best available science in some cases -- in direct contravention of statutory requirements. The commenter states that, under the differential weighting approach, the EPA would not be able to use the best available science and consider all available information, in violation of the EPA's statutory obligations. 
Commenter (13788) states that, unlike the 2018 proposal, the SNPRM does not include any acknowledgement that the EPA must use the "best available science" for regulatory actions -- but it must.. The commenter notes that, under the Administrative Procedure Act (APA) and CAA, agency actions that are arbitrary, capricious, an abuse of discretion, or otherwise not in accordance with law are to be held unlawful and set aside. The commenter states that agencies are required to "articulate a satisfactory explanation for [the] action including a `rational connection between the facts found and the choice made.'" The commenter states that, here however, the EPA is proposing to exclude or devalue scientific studies based on criteria that are unrelated to scientific value or merit, and that would eliminate the Agency's ability to rely on research that may be the best available science.
Commenter (11576) expresses concern with respect to the practicability of requiring all information to be made available, given unequivocal statutory deadlines embedded within environmental statues. 
Response: The final rule provides that if implementing the rule results in any conflict between this rule and the environmental statutes that the EPA administers, and their implementing regulations, this rule will yield and the statutes and regulations will be controlling. The requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
Case Law
Comment: Commenters (11390, 11403, 11479, 11495, 12720) refer to court cases and assert that the EPA must use the best available science regardless of whether the data are publicly available. 
Commenter (11390) asserts that the "best available science" and similar standards set out in environmental statutes require the use of scientific information and analysis that follow standards accepted by the scientific community, not information and analysis based upon the public availability of data. The commenter states that the proposed requirements ignore well-established indicia of reliable scientific research, such as peer review. The commenter states that proposal would force the EPA to ignore high quality science and that this controverts the EPA's statutory obligations and will impair the EPA's ability to protect public health and the environment. The commenter contends that public availability of data is a criterion often irrelevant to scientists' standards for determining whether research represents the best science. The commenter asserts that the proposal's emphasis on public availability contradicts the EPA's own statutory mandates and is inappropriate, given the courts' agreement that publicizing the data underlying studies upon which EPA relies "would be impractical and unnecessary." 
Commenter (11403) states that the EPA has long relied on peer-reviewed studies in its rulemaking, even where the underlying data may not be publicly available. The commenter states that, as the proposed rule notes, the EPA has successfully defended that exact approach before the D.C. Circuit, which has recognized that "[i]f EPA and other governmental agencies could not rely on published studies without conducting an independent analysis on the enormous volume of raw data underlying them, then much plainly relevant scientific information would become unavailable to EPA for use in setting standards to protect public health and the environment...."
Commenter (11495) states that, in at least two instances, the D.C. Circuit Court of Appeals has recognized that studies for which the underlying data are not publicly available may constitute the "best available science." The commenter states that, in American Trucking Associations, Inc. v. EPA, the Court held that the CAA did not require the EPA to make underlying data public where EPA relied on the study itself and not the raw underlying data. The commenter states that the Court agreed with EPA's argument that requiring the Agency to obtain and publicize the relevant data "would be impractical and unnecessary." The commenter states that in Coalition of Battery Recyclers Association v. EPA, the court again upheld the EPA's reliance on non-publicly available data, reasoning that the EPA relies on studies and their results rather than the raw data underlying those results.
Commenter (11479) states that records released through a FOIA request confirm that EPA staff are aware that the Agency does not need to have access to the underlying raw data, codes, etc. to promulgate regulations. The commenter states that during the development of this proposed rule, Dr. Nancy Beck, the Deputy Assistant Administrator for the Office of Chemical Safety and Pollution Prevention, emailed colleagues "for awareness" that in the 2002 court decision American Trucking Association v. EPA, it became evident that, if the EPA were required to have access the raw data of peer-reviewed scientific studies, the Agency would not be able to use a significant amount of important scientific literature. The commenter states that the courts, through that decision and through Coalition of Battery Recyclers Association v. EPA, have made it clear that the EPA is not required to obtain or analyze raw data in order to use independent scientific studies that are published in peer-reviewed journals to regulate.
Commenter (11495) states that the EPA has already established an array of rigorous review techniques that go beyond the typical scientific journal peer review process to ensure that the best available science is used. The commenter states that imposing this new public availability of data obligation negates the purpose of these "rigorous reviews" and the already-strict review imposed by the scientific community. The commenter states that courts have held that agencies "cannot ignore available...information." The commenter states that plaintiffs and petitioners can establish a violation of a statutory "best available science" requirement imposed on the Agency by "point[ing] to any scientific evidence that the agency failed to consider." The commenter states that the Ninth Circuit Court of Appeals has also held that "the best available data requirement . . . prohibits [an agency] from disregarding available scientific evidence that is in some way better than the evidence [it] relies on." The commenter states that, under the SNPRM, the EPA would not consider some scientific studies that are potentially more accurate and robust than the studies it does consider based upon the arbitrary criterion of the public availability of the study's underlying data.
Commenter (12720) states that, in its SNPRM, the EPA proposed to categorically ignore and exclude all peer-reviewed research with non-public underlying data, without individually considering each study or offering specific reasons for not relying on that study. The commenter states that the SNPRM, by barring consideration of foundational scientific research premised upon non-public data, would result in EPA "fail[ing] to consider an important aspect of the problem." The commenter states that there is no evidence of a court supporting an agency's decision to exclude entire categories of evidence, or studies or information based on categorical prohibitions like the ones in the SNPRM, without considering the source and offering specific reasons for not relying on the study. The commenter states that, both the EPA and the courts have indicated in API and Coalition of Battery Recyclers, that a rule like the one the EPA is currently proposing is not required by the CAA and would be both impractical and unnecessary. The commenter states that this SNPRM runs counter to the D.C. Circuit's decision in API and would render the EPA's regulatory actions based on the SNPRM arbitrary and capricious and an abuse of EPA's discretion. The commenter states that SNPRM's blanket rule would represent a significant and unlawful departure from D.C. Circuit rulings on agencies' limited discretion to choose the sources it will consider and ignore. 
Response: The EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. 
The final rule provides that if implementing the rule results in any conflict between this rule and the environmental statutes that the EPA administers, and their implementing regulations, this rule will yield and the statutes and regulations will be controlling. The requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information and do not interpret provisions of the statutes referenced by commenters. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information.
7.1.2.2.2 Informed Decision-Making
Comment: Commenters (11360, 11393, 11399, 11473, 11576, 11895, 12403, 12412, 12427, 12434, 12443, 12455, 12474, 12546, 12731, 13549) object to the SNPRM because the rules will lead the EPA to make decisions based on incomplete and/or improperly weighted information, rather than the best available information. 
Commenters (11393, 11895, 12412, 12474, 13549) contend that the SNPRM does not address the fact that many past and ongoing public health studies simply cannot lawfully, ethically, or efficiently publicly release the underlying data. The commenters state that the SNPRM does not claim that such studies are invalid or reached incorrect conclusions. The commenters state that some of these past studies have been reviewed, replicated, and validated by additional independent studies. The commenters state that, even if applied only to future research, the SNPRM would limit participation, limit the information that can be collected, and waste limited resources.
Commenter (11399) states that additional "independent validation" that goes above and beyond the peer review process and the EPA existing science advisory infrastructure is a new procedural hurdle that will limit the universe of scientific studies that the EPA may consider. The commenter states that creating such an impediment undermines the EPA's legal obligation to use the best-available science to guide its decision-making processes and portends a weakening of the scientific underpinnings of health-based environmental regulations. 
Commenter (11473) expresses concern that discounting evidence from the decision-making process on the basis that some data are confidential runs counter to the EPA stated mission to reduce environmental risks...based on the best available scientific information. The commenter states that the EPA 2016 document on transparency stated that "Classified or otherwise protected EPA-funded scientific research will not be made publicly available."
Commenters (11576, 12405, 12427) refer to a joint statement by a group of scientists and editors-in-chief at leading scientific journals in response to the proposed rule: "It does not strengthen policies based on scientific evidence to limit the scientific evidence that can inform them; rather, it is paramount that the full suite of relevant science vetted through peer review, which includes ever more rigorous features, inform the landscape of decision making. Excluding relevant studies simply because they do not meet rigid transparency standards will adversely affect decision-making processes."
Commenter (12403) asserts the rule would compromise the EPA's decision-making by excluding from consideration or devaluing well-conducted peer reviewed studies that cannot comport with data availability requirements. The commenter states that it also runs the risk of regarding small, poorly designed studies while penalizing larger and more representative research studies.
Commenter (12434) states that the proposal would exclude validated and qualified science that has undergone rigorous peer review if the data were not publicly available. The commenter states that, consequently, the EPA would necessarily rely on scientific studies with publicly available data regardless of whether these studies were the most robust or best suited to inform decision making. The commenter states that decisions could be based on science that does not represent the latest scientific consensus, but by science arbitrarily selected by the Administrator, a political appointee. 
Commenter (12443) asserts that, as written, this rule would result in excluding many well-designed studies that contain private medical information that cannot ethically be made publicly available, because to do so would jeopardize the privacy of study participants' medical data and possibly other information. The commenter states that researchers should not be forced to choose whether to protect the privacy of participants' data or to make their findings applicable to the EPA. The commenter states that, if the EPA were to ignore findings from studies with private data, many high-quality research findings could not be considered, which would weaken the basis of EPA decision-making. The commenter states that there is no scientific basis for discounting such findings, and valuable research would be suppressed, harming the public health.
Commenter (12455) states that the EPA's intent to have all decisions based on publicly available data as stated in the SNPRM could harm federal decision-making. The commenter states that there are many qualified research entities that conduct studies where the report may not be available to the public. The commenter states that, although the concept of having studies be publicly available so peer review can validate data has the veneer of transparency, it clearly subverts the ability of the researcher using the data to determine the validity of data, study, or methods used.
Commenter (12546) notes that the EPA is proposing to downgrade the consideration of studies that do not make underlying data and models publicly available. The commenter states that the EPA itself acknowledges that potential studies for downgrade or elimination are "high-quality" and that scientific findings and research are valid whether or not data are publicly available, a reiteration of the conclusion put forth in the EPA's 2016 Plan. The commenter states that, if this model is pursued, the EPA can base new and renewed standards on lower-quality science, rather than the "best available science," and "adequate information." The commenter states that the omission of high-quality studies because they do not have available the underlying data could greatly impact the strength of systematic reviews and meta-analyses needed to show the harms of an exposure on a health outcome, and this could lead to conclusions that the evidence-base lacked sufficient weight.
Commenter (12731) refers to the SAB report which noted that "[i]t is not clear: (1) how many of the studies the EPA currently relies upon to take important regulatory actions would meet the public disclosure standards stated in the proposed rule, and (2) whether EPA has assessed the feasibility of making underlying information from the studies publicly available, or what the impact of precluding those studies would be on EPA's decision making and its ability to protect public health/environment." The commenter states that the EPA thus continues to disregard the likelihood that its rule would severely erode the best available science available to the agency, which it recognizes "must serve as the foundation of [its] regulatory actions." The proposal remains arbitrary in ignoring this obvious -- and now expanded -- counterproductive effect.
Response: The EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. The final rule does not require EPA or any other entity to conduct independent validation before EPA can use pivotal science. Nor does the final rule require EPA to obtain, store or publish dose-response data.
The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Comment: Commenters (10040, 11479, 11491, 11907, 11926, 12457, 12458, 12716, 12720, 12715) object to the rules because they would eliminate important health evidence collected under assurance of confidentiality and undermine the EPA regulatory decisions.
Commenter (10040) states that, by not allowing the use of science grounded in datasets that use confidential, protected health data from individuals who have consented on the condition that their data remain confidential, this proposal stands to severely undermine the scope of research available to inform the EPA as it works to make scientifically informed regulatory decisions.
Commenter (11479) notes that one Option offered in the new notice would allow the EPA, if considering a study whose underlying data are not made available, to place less weight on the results "to the point of entirely disregarding them," even if the research and results were strong and compelling, met the highest scientific standards, and had undergone extensive scientific peer review. The commenter states that, in other words, the EPA would give the most weight to studies with publicly available data rather than to studies that provide the strongest scientifically supportable evidence. The commenter states that, because there are many environmental health studies relying on medical data and other private information, the rule's restrictions would mean that the best available science could be excluded.
Commenter (11491) states that neither of the Options provided in the SNPRM, regarding the need to both use and maintain the confidentiality of certain information relied upon by the Agency in its decision-making process, adequately addresses the constraints that the rule would put on EPA's ability to rely on the best available science. The commenter states that the rule would constrain the Agency's ability to use the best available science and thereby undermine the quality and defensibility of its final actions. The commenter refers to the EPA's SAB report which states: 
      There is minimal justification provided in the Proposed Rule for why existing procedures and norms utilized across the U.S. scientific community, including the federal government, are inadequate, and how the Proposed Rule will improve transparency and the scientific integrity of the regulatory outcomes in an effective and efficient manner. It is plausible that in some situations, the Proposed Rule will decrease efficiency and reduce scientific integrity, determining if in fact that will be the case requires a thorough and thoughtful examination that is currently absent in the Proposed Rule. Moving forward with altered transparency requirements beyond those already in use, in the absence of such a robust analysis, risks serious and perverse outcomes.
Commenter (11362) contends that the EPA must prioritize public health in its decision making, therefore data from public health and epidemiological studies must be taken into consideration for influential scientific information and regulatory actions and should be exempt from this rule if passed into law. 
Commenter (11907) states that this rule will likely reduce or severely restrict the EPA's use of epidemiology studies that are based on anonymous or confidential PII that can contribute to identifying sources of air, water, or ground pollution that can lead to chronic human health conditions and terminal diseases, not only within Native American communities, but within all communities throughout the United States.
Commenter (11926) asserts that the proposal and SNPRM would seriously impact the EPA's ability to utilize the best available science in formulating new regulations or renewing existing ones. The commenter states that, by mainly considering studies with publicly available data or those posted to a special database that allowed public access to sensitive data by special arrangement, the rule would mean little weight would be given to studies with strictly confidential non-public data. The commenter states that this would eliminate important health evidence collected under assurance of confidentiality because publication of the data would be a breach of confidentiality. Ignoring the results of such studies, many in the most exposed and most affected populations, would mean that regulations and policies essential to protect public health would never come into existence. 
Commenter (12457) contends that the proposed revisions could result in the failure to properly consider important pivotal science and research findings during the policy making process. The commenter states that limiting information that is utilized for policy setting is not the most prudent course of action as policy makers often need to consider a range of information and data in order to set sound policy. 
Commenter (12716) asserts that, to have access to the best available science, studies must protect the privacy of those affected by environmental risks. The commenter states that the EPA's proposal requires access to underlying data and, consequently, will limit the types of scientific studies that will be conducted and used to inform regulatory decision-making. The commenter notes that the EPA recognizes the need to balance the interests of making information available to the public and protecting other interests, like privacy.
Commenter (12720) expresses concern that the EPA's proposal does not clarify how fundamental patient privacy and confidentiality issues can be overcome. The commenter notes that the SAB points out that such hurdles will prove insurmountable in some cases, depriving the EPA of the best available science to inform critical safeguards to limit pollution and protect public health. 
Commenter (13788) asserts that the EPA's real intention in the SNPRM, as in the proposal, is to limit the Agency's access to public health science, not to ensure the quality of the studies the Agency relies on. The commenter states that it is made clear by the Agency's statement that if "multiple high-quality studies exist but only two studies have data and models that are available for independent validation and reanalysis, EPA would only include those two studies as pivotal regulatory science and/or pivotal science." The commenter states that, by acknowledging that the highest quality studies will be ignored if confidential data and models are not made available for validation and reanalysis, the EPA is admitting that this rule is not concerned with improving the quality of studies relied on by the Agency. The commenter states that the SNPRM is instead focused on information availability, even if it comes at the expense of the quality of scientific work. The commenter states that the best studies may not rely on available underlying data -- precisely because they are human health studies that requires the maintenance of the privacy of the study participants  -- and in those situations this rule would clearly prioritize data availability over quality.
Commenter (12715) states that the issue of protecting confidentiality of data has been well recognized. The commenter states that this process demonstrates that, even after going through an arduous, time consuming process, the removal of personal identifying information would be difficult to achieve. The commenter states that, therefore, many valid, peer reviewed epidemiological studies would be excluded from consideration, or arbitrarily given less weight, because the data would not be publicly available. 
Commenter (12458) asserts that giving greater weight to studies where the underlying data and models are available or made available through tiered access would disproportionately exclude research that relies on personal health data, including research that draws crucial connections between environmental pollution and human health. The commenter states that such research could include the kind of critical work on the human health impacts of water and air pollution that has contributed to regulations that have saved thousands of lives and trillions of dollars. The commenter states that giving greater weight to studies where the underlying data is available will increase the relative weight of industry-funded research and research using non-human subjects, which are less likely to foreground questions about human health. The commenter states that, for these reasons, the SNPRM and rule would actively interfere with EPA's mission to protect public health both in rulemaking and in other Agency activities. 
Response: The final rule does not require EPA or any other entity to conduct independent validation before EPA can use pivotal science. Nor does the final rule require EPA to obtain, store or publish dose-response data. The EPA shall give greater consideration to pivotal science where the underlying dose-response data are publicly available in a manner sufficient for independent validation. The Agency shall also give greater consideration to pivotal science based on dose-response data that include CBI, proprietary information or PII if these data are available through restricted access in a manner sufficient for independent validation. For pivotal science where there is no access to dose-response data, or access is limited, the Agency may still consider these studies but will give them lesser consideration unless the Administrator grants an exemption under 40 CFR 30.9. The Agency will identify the pivotal science that was given lesser consideration and provide a short description of why lesser consideration was given.
7.1.2.2.3 Public Health
Comment: Commenters (10040, 11377, 11378, 11390, 11491, 11926, 12443, 12458) object to the rule because it would undermine the Agency's ability to execute its mission to protect public health, and its obligation to use best available science. Commenter (10040) states that these regulatory decisions have profound implications for the lives, health and well-being of communities nationwide. Commenter (11377) asserts that systematically and arbitrarily favoring fully-public data for even more of the Agency's work only further imperils citizens whose health depends on the EPA's using the best science, not just the most public. Commenter (11926) asserts that ignoring the results of such studies, many in the most exposed and most affected populations, would mean that regulations and policies essential to protect public health would never come into existence. Commenter (12731) states that, in November 2019, the editors of the nation's leading science journals issued a statement underscoring that the notion of discounting studies based on the availability of underlying data is contrary to good scientific practice and would be a "catastrophe" for public health.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
7.1.2.3 Study Quality, Validity and Accepted Practices
7.1.2.3.1 Quality and Validity of Studies
Public Availability and Quality/Validity of Studies
Comment: Commenters (11392, 11399, 11408, 11473, 11480, 11487, 11489, 11576, 12455, 12457, 12546, 12720, 13788, 14397, 12715) do not agree with the rule's premise that studies are "better" or more "valid" than others, depending on whether or the degree to which their underlying data and models are available to the public. 
Commenter (11392) states that all studies go through a rigorous peer review process prior to publication, and as a result, the scientific community does not use the so-called potential for "independent validation" as a metric for quality of a study. 
Commenter (11399) states that both rule Options rest on an incorrect premise that scientific studies should be "graded" by EPA based on the transparency of their underlying data and models, and that more "transparent" science is more worthy of EPA's consideration than studies based on confidential data that cannot be made publicly available. The commenter states that this assumption is fundamentally untrue and conflates two separate issues: transparency and scientific rigor. The commenter states that this issue is distinct and separate from EPA's obligation to consider the complete range of published, peer-reviewed scientific research that is relevant to its regulatory decisions. 
Commenter (11408) states that the EPA's response to concerns regarding data availability is to weigh more heavily studies where the data is publicly available. The commenter states that this "compromise" runs counter to principles of transparency, as the Agency would rely more heavily on a study with publicly available data over one that meets the highest scientific standards and yields compelling results. 
Commenters (11473, 12546, 13788) assert that the quality of the research is not dependent on data availability and refer to the EPA's 2016 Plan to increase access to results of EPA-funded scientific research which states: "The validity of scientific conclusions drawn from research publications or their associated research data, or EPA's ability to consider those conclusions and data in its actions, does not depend on compliance with this Plan." Commenters (12546, 13788) point out that the EPA's 2016 Plan emphasizes that data availability does not affect the validity or usability of science: "Whether research data are fully available to the public or available to researchers through other means does not affect the validity of the scientific conclusions from peer-reviewed research publications."
Commenters (11487, 11479) assert that using non-scientific criteria to evaluate scientific studies is thoroughly inappropriate and at odds with established practices for evaluating study quality. The commenters state that, for instance, the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) checklist does not include public availability of data. Commenter (11479) states that the STROBE initiative deems transparency crucial but does not include any criteria that would require the underlying data to be made publicly available. The commenter states that, as external researchers conduct systematic reviews or meta-analyses on particular health outcomes, public availability of data is not among the criteria for study appraisal in the widely accepted Cochrane Systematic Review method. The commenter states that the five criteria used to appraise and down-weight studies are related to study quality, not the availability of data, and include "limitations in the design and implementation of available studies suggesting high risk of bias; indirectness of evidence; unexplained heterogeneity or inconsistency of results; imprecision of results; and high probability of publication bias." The commenter states that imposing a requirement that is disconnected from the realities of the research world would put the quality of EPA's assessments and the Agency's reputation in the scientific community at risk.
Commenters (11480, 12464) assert that both approaches outlined in the proposal remove from consideration or significantly reduce the weight of studies when raw data is not publicly available for independent validation and reanalysis. Commenter (11480) states that public availability of data and models is not a criterion of best available science and should not be considered in determining whether to consider a study in an Agency decision. Commenter (12464) states that the rule relies on the flawed assumption that the scientific reliability of a study can be best determined by a reanalysis of the original data.
Commenter (11489) contends that the value or merit of peer-reviewed science is not determined by the public availability of the underlying research data. Commenter (12455) states that the lack of public availability does not automatically mean the research is flawed. Commenter (12457) states that, whether a study has data that is publicly available is not a factor that reflects the quality of the study, nor the relevance of the study's results to EPA policy decisions.
Commenter (11576) states that, throughout the proposed rulemaking, the EPA erroneously conflates the public availability of underlying data with the accuracy of a study. The commenter states that this conflation is contrary to well-established standards of scientific practice, in which the process of peer-review ensures that studies are vetted for accuracy. The commenter states that, while many scientific journals are starting to make more data publicly available, it is important to note that the availability of data underlying published studies has no bearing on evaluating the accuracy of the studies. The commenter states that access to unprocessed data at the stage described in the SNPRM is typically not a requirement even for peer-review, as scientists can evaluate the robustness and accuracy of findings based on the presentation of processed data and the articulation of the methods and analysis used. 
Commenter (12546) states that the EPA is proposing to downgrade or exclude science that is not made publicly available or "cannot be sufficiently de-identified to protect the data subjects." The commenter states that this would codify that study quality is based on availability of underlying data, which is not the measure of study quality. The commenter states that this language is both vague and dangerous in implying that studies that do not make their data available are less informative or of less quality  -  both of which have no empirical basis. The commenter states that availability of underlying data is not a measure of study quality or validity; rather, study quality is dependent on the methods used in the study. 
Commenter (12720) notes that the EPA in section 30.5 proposes to consider studies differently based on the availability of the study's data. The commenter states that this proposal has no scientific basis as availability of data has not been empirically shown to affect the quality or validity of a study. The commenter states that, as public availability of data or access to data have not been identified in the scientific literature as factors that have any bearing on the quality or validity of science, these criteria should not be used by EPA to give greater consideration to particular studies. 
Commenter (14397) states that this SNPRM approach improperly makes openness the primary factor, or one of the primary factors, in weighing which studies to use, when robust scientific analysis calls for priority being given to studies that fit the accepted hierarchy of evidence and that follow basic scientific recommendations as to control, randomization, sample size, peer review, etc. The commenter states that studies should be considered or rejected based on their adherence to the appropriate scientific standards and the importance of their findings, not on their manner of dealing with CBI or PII, when presumably they have already been properly peer reviewed. 
Commenter (12715) asserts that the use of Good Laboratory Practice protocols or other protocols where data are publicly available does not necessarily mean that the study is of higher quality or that it has evaluated the effects that are most relevant for decision-making, and there is no scientific reason that the data generated under the highly circumscribed regulatory requirements for product registration should receive greater weight than any other valid scientific data.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Comment: Commenters (11408, 11479) state that the demand for obtaining the raw data related to EPA rulemaking does not appear to be significant and would not justify the excessive burden it would place on the Agency. The commenter states that one study, which looked at the 79 requests made between 2002 and 2012 through the Information Quality Act to EPA that asked the Agency to correct or reconsider the data supporting its regulatory decisions, found only two requests asking for the raw data. The commenter states that one possible explanation for this low number is that the public already has access to the science the EPA relies on and can fully access methods, summary data, results, and interpretations, as can all peer reviewers. The commenter states that in developing the NAAQS, for example, the EPA collects all relevant data and studies used to develop the Integrated Science Assessment and the merits of those scientific studies are publicly debated by the Clean Air Scientific Advisory Committee and pollutant review panel. The commenter states that the need for access to raw data is entirely unnecessary and arbitrarily precludes the use of critical studies.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Comment: Commenters (11403, 11498, 11576, 12718) object to the proposal package and assert that access to raw data is not determinative of the quality of the research.
Commenter (11403) asserts that public availability of raw data is not a rational metric for deciding whether a study is of high enough quality to be used for setting thresholds or enacting regulatory protections. The commenter states that the EPA must ensure that it uses high quality data to analyze environmental justice impacts vulnerable populations, regardless of the public availability of the raw data.
Commenter (11498) contends that there are credible procedures for testing results and verifying outcomes with methodologies that do not require access to raw data. The commenter states that the original proposed rule and now this SNPRM are de facto rejecting credible practices used by the scientific community and replacing them with a non-scientific metric in the evaluation of a study beyond its immediate quality. The commenter states that, as the EPA's own Scientific Advisory Board has stated, this decision risks politicizing science by using an unscientific standard to assess the validity of science.
Commenter (11576) states that there is no need to limit consideration of reliable science simply because the underlying data is not publicly available. The commenter states that less than 30 percent of scholarly literature is open-access, and only a small fraction of these include access to underlying raw data. The commenter states that journal-specific transition plans to increase access involve long timeframes to accommodate these changes, which are often voluntary opt-ins for participating researchers. The commenter states that these open-access programs typically refer only to public access to the published studies themselves rather than to underlying raw data as required in the proposed rule, which is contrary to standard scientific practice. 
Commenters (11576, 11920) refer to a joint statement by a group of scientists and editors-in-chief of leading scientific journals which states that "the merits of studies relying on data that cannot be made publicly available can still be judged." Commenters state that the group further noted that scientific studies typically contain all the information required to judge the accuracy and robustness of their findings without access to raw data as scientists are trained in assessing research publications by judging the articulation and logic of the research design, the clarity of the description of the methods used for data collection and analysis, and appropriate citation of previous results.
Commenter (12718) states that the Multi-Society comments on the SNPRM espouse a principle that must be given careful consideration: "access to raw data is not determinative of the quality of the research." The commenter states that the EPA's proposed approach in the SNPRM presumes that access to data or information must be  -  as an absolute  - accessible to EPA officials, even when that level of access might be inconsistent with Institutional Review Board (IRB) requirements, consent statements, or other ethical procedures that should be paramount in government's commitment to its citizens for protecting data and confidentiality.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Justification of How Availability Impacts Validity
Comment: Commenters (10048, 11408, 11576, 12430, 12727, 12731, 13788) express concern that the EPA has not provided any justification as to how or why the availability of confidential information or PII may impact the validity of data, studies, and models.
Commenter (11576) states that, while the EPA argues the proposed rule will strengthen the quality of the Agency's science, it points to no evidence to support that statement. The commenter states that, instead, the evidence shows that the rule's emphasis on data availability rather than accuracy will actually weaken the body of evidence available to the Agency. The commenter states that, while full consideration of the range of observations reported in peer-reviewed scientific literature provides a method for comprehensively estimating trends, means, and variances in data, arbitrarily omitting studies can result in skewed interpretations. 
Commenter (12430) states that the EPA does not articulate, let alone provide, any basis for alleging that the scientific community's established practices are faulty or inadequate. The commenter states that, nonetheless, it would replace these practices with a single threshold requirement: public availability of the underlying data. The commenter states that the fact that the proposed rule is at odds with EPA's and the scientific community's established practices -- as underscored by the many unique, substantive comments submitted in opposition to the rule by scientists, scientific organizations, and scientific journals -- suggests that the EPA intends for the SNPRM to advance political priorities rather than the quality of science the Agency uses.
Commenter (12727) indicates that EPA has not demonstrated -- and cannot demonstrate -- that a study cannot be scientifically valid unless the underlying data and models are publicly disclosed. The commenter states that mechanisms are already in place to validate studies for which disclosure of underlying data and models is either illegal or impracticable. 
Commenter (12731) states that researchers do not make data public to improve the strength or quality of their findings, and the EPA has offered no evidence to suggest that studies with publicly available underlying data are more likely to represent strong science than studies without such data availability. The commenter asserts that, like the original proposal, the SNPRM lacks any demonstration that the public unavailability of a study's underlying data or models necessarily -- or even likely -- renders the study invalid or unreliable. The commenter states that the proposal failed to identify an actual problem that needed to be addressed, or to point to even a single example of an EPA action or rulemaking that was later found to be defective because it rested on a study for which data was not publicly available.
Commenter (13788) contends that the basic premise of this rulemaking is that public health data -- including human health data -- has to be made publicly available to ensure the validity of scientific studies. The commenter states that this premise is completely unsupported and totally incorrect. The commenter states that the Agency's long standing position is that "[w]hether research data are fully available to the public or available to researchers through other means does not affect the validity of the scientific conclusions from peer-reviewed research publications." The commenter states that the EPA fails to explain why it now believes the contrary -- that study validity and public data availability are (or should be) connected.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
7.1.2.3.2 Approaches to Validity Other Than Raw Data
Comment: Commenters (10866, 11377, 11400, 11489, 12403, 12731) note that the proposal package calls for underlying data and models to be "publicly available in a manner sufficient for independent validation." These commenters object to the proposal package and generally assert that such independent validation is not the only means of determining credibility of studies.
Commenter (10866) states that the EPA fails in making its proposal to acknowledge that research studies can be both evaluated and replicated without the release of such individual data. Commenter (11377) asserts that transparency can be achieved in different ways that satisfy the demands of different kinds of science, while still maintaining respect for persons and intellectual property. Commenter (11489) states that, in cases where it is not appropriate for data to be made publicly available, there are other mechanisms intrinsic to the scientific process for substantiating the relevance and validity of research results. Commenter (12403) states that the proposed rule is not necessary to allow for independent validation of scientific studies and it would undermine existing successful science-based policy making. Commenter (12731) notes that comments on the original proposal demonstrate that mechanisms are already in place to validate studies for which disclosure of underlying data and models is either illegal or impracticable. 
Response: The EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
Comment: Commenter (11378) contends that the integrity of much environmental and health research requires the tradeoff that certain business, proprietary, and PII remain confidential. The commenter states that making such information publicly available would potentially violate long-standing human subjects research protocols (i.e. the 1974 National Research Act), as well as legally binding proprietary data agreements.
Commenter (11926) asserts that the notion that the public release of raw data is a protection against fraud, or the generation of misleading results is also flawed. The commenter states that there are instances in which an unscrupulous researcher alters or even fabricates data to fit a preconceived hypothesis, however, there is no way to tell from the data itself that it is not accurate. The commenter states that this is why the weight of evidence, replication of findings from multiple sources, and peer review by scientific journals are so important -- these factors, not the public availability of raw data, are the main safeguards against flawed or fraudulent science. 
Commenter (13788) states that the EPA provides no explanation as to why those policies are not sufficient to allow validation of scientific studies and information relied on by the Agency. The commenter states that simply stating that "EPA shall conduct independent peer review on all pivotal regulatory science used to justify significant regulatory decisions and on all pivotal science underlying influential scientific information, consistent with the requirements of the Office of Management and Budget (OMB) Final Information Quality Bulletin for Peer Review," fails to explain why this does not already occur under the Agency's past and current practice. The commenter contends that, if validation is truly the aim of this rulemaking, the EPA must explain why existing practices for peer review are insufficient to validate scientific studies. 
Response: As discussed in Section III.B of this preamble, based on a consideration of the public comments on the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. The EPA is also further clarifying how the Agency will determine the consideration to afford to pivotal science in either significant regulatory actions or influential scientific information for which the conclusions are driven by the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect.
Consistent with existing Agency practice, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information. The EPA will continue to use the following, established factors to assess the quality of studies used to develop significant regulatory actions and influential scientific information: 
 Soundness  -  The extent to which the scientific and technical procedures, measures, methods or models employed to generate the information are reasonable for, and consistent with, the intended application. 
 Applicability and Utility  -  The extent to which the information is relevant for the Agency's intended use.
 Clarity and Completeness  -  The degree of clarity and completeness with which the data, assumptions, methods, quality assurance, sponsoring organizations and analyses employed to generate the information are documented.
 Uncertainty and Variability  -  The extent to which the variability and uncertainty (quantitative and qualitative) in the information or in the procedures, measures, methods or models are evaluated and characterized.
 Evaluation and Review  -  The extent of independent verification, validation and peer review of the information or of the procedures, measures, methods or models.
When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence (consistent with Agency guidelines on hazard identification and dose-response assessment) to support a relationship between exposure and effect, the EPA will identify a subset of those studies for use in characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect. This will be based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both will be identified as pivotal science (see Section III.E).
Comment: Commenters (10040, 10748, 11480) suggest the Agency rely, in part, on well-established principles for good science such as disclosure of conflicts of interest.
Commenter (10040) urges the EPA to withdraw its proposal to restrict scientific research and follow the current, effective measures in place to ensure the use of robust, uncensored scientific research to protect the health of our citizens and our communities. The commenter states that existing protections are in place to protect and assure the quality of the science used in the process of EPA's regulatory decision making. 
Response: Consistent with existing Agency practice, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information. The EPA will continue to use the following, established factors to assess the quality of studies used to develop significant regulatory actions and influential scientific information: 
 Soundness  -  The extent to which the scientific and technical procedures, measures, methods or models employed to generate the information are reasonable for, and consistent with, the intended application. 
 Applicability and Utility  -  The extent to which the information is relevant for the Agency's intended use.
 Clarity and Completeness  -  The degree of clarity and completeness with which the data, assumptions, methods, quality assurance, sponsoring organizations and analyses employed to generate the information are documented.
 Uncertainty and Variability  -  The extent to which the variability and uncertainty (quantitative and qualitative) in the information or in the procedures, measures, methods or models are evaluated and characterized.
 Evaluation and Review  -  The extent of independent verification, validation and peer review of the information or of the procedures, measures, methods or models.
When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence (consistent with Agency guidelines on hazard identification and dose-response assessment) to support a relationship between exposure and effect, the EPA will identify a subset of those studies for use in characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect. This will be based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both will be identified as pivotal science (see Section III.E).
Comment: Commenter (11390) states that the SNPRM's approach would, in a wide range of circumstances, replace the EPA's existing, successful protocols -- already in step with established scientific procedures -- with a process driven by arbitrary, non-scientific gatekeeping. 
Commenter (12702) states that the EPA has established existing protocols and statutory requirements for the rigorous scientific review for any rulemakings, and those requirements are not tied to whether the data are publicly available. The commenter states that, thus, robust peer review, can, and sometimes should, proceed without the publication of sensitive or confidential data. The commenter urges the EPA to ensure that any final rule is consistent with the scientific journal standards cited in the rule.
Response: As discussed in Section III.B of this preamble, based on a consideration of the public comments on the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. The EPA is also further clarifying how the Agency will determine the consideration to afford to pivotal science in either significant regulatory actions or influential scientific information for which the conclusions are driven by the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect. 
Consistent with existing Agency practice, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information. The EPA is, therefore, finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate peer review, consistent with the OMB Bulletin for Peer Review and EPA's Peer Review Handbook, of individual studies identified as pivotal science if the studies have already undergone journal peer review. If the Agency conducts peer review on pivotal science, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied in analyzing dose-response data and the implications of those assumptions for the results.
Comment: Commenters (11387, 11395, 11890, 11926) suggest that there is no need for this rule as mechanisms exist for researchers to share data in order to confirm and validate studies. 
Commenters (11387, 11890) suggest that researchers in one institution can transmit data in a confidential manner to researchers in one or multiple other institutions so the data can be evaluated and analyzed in a completely new and different analysis. The commenters assert that this allows the study to be confirmed and validated, while still maintaining privacy protections of the people in the study.
Commenter (11395) states that, as discussed in comments on the proposed rule submitted by a large group of scientists in August 2018, representatives of academia, medical scientists, industry and publishers have developed protocols and guidelines specifically designed to improve reporting and evaluation of studies to improve the quality and transparency of the interpretation of findings. The commenter states that these protocols and guidelines are designed to improve the scientific basis of data evaluations without requiring public access to all study data. The commenter states that the EPA should utilize these existing tools to evaluate study quality and should build on the work already done by the scientific community.
Commenter (11890) states that, in order to honor study commitments to participants to guard their privacy and laws governing CBI and intellectual property, this could best be accomplished by a data sharing agreement with a trusted third party that agrees to uphold confidentiality of the study data. An example of this model is the Health Effects Institute reanalysis of Harvard's "Six Cities" Study and the American Cancer Society's "Study of Particulate Air Pollution and Mortality."
Commenter (11926) states that one approach is the establishment of data enclaves, in which an independent expert can access the raw data for statistical analysis, generate results to replicate the original findings and perform additional analyses, without taking possession of the data itself. The commenter states that the technology to create a secure data enclave is well-established, and many examples exist. The commenter states that this approach allows rigorous independent assessment of the raw data without compromising the sensitive information provided by human subjects. 
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (11489) expresses concern that the SNPRM does not address other mechanisms that could be used to give the EPA confidence that the findings should be considered in rulemaking. The commenter states that such mechanisms include comparing outcomes of several trials from different groups that reach the same conclusions and vetting the science through expert scientific panels convened for this purpose. The commenter states that these mechanisms are especially important for past studies where the underlying datasets may be unsuitable or unavailable for public review. The commenter also asserts that the public notice and comment period provide an opportunity for the Agency to communicate the scientific research that supports its proposal.
Response: When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence (consistent with Agency guidelines on hazard identification and dose-response assessment) to support a relationship between exposure and effect, the EPA will identify a subset of those studies for use in characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect. This will be based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both will be identified as pivotal science (see Section III.E).
Comment: Commenters (10040, 10339, 10973, 11378, 11395, 11399, 11480, 11489, 11498, 11926, 12404, 12430, 12617, 13788, 14397) suggest that peer review is an appropriate mechanism to determine the validity of studies.
Commenter (10040) asserts the use of peer-reviewed research is the foundation for EPA's decision making. Commenter (11378) encourages the EPA to utilize the most appropriate, scientifically peer-reviewed research to guide the development of environmental regulations and scientific information. Commenter (11399) states that additional "independent validation" that goes above and beyond the peer review process and the EPA existing science advisory infrastructure is not necessary to assure scientific validity and accuracy. 
Commenter (10339) expresses concern that the Agency provides no evidence that the EPA decisions have been misled by the use of data published in the peer-review literature. The commenter suggests that the stated reason for this rule lacks scientific justification and credibility. 
Commenter (10973) states that the EPA often appropriately publishes research through peer-reviewed journals. The commenter states that, ensuring that the Agency makes this information publicly available, whether through open access journals (or open access options within traditional journals), pre/post-prints, publicly available data, or some other method or combination of methods would help assure that all of EPA's processes can align to this policy and be of the greatest utility to all who may benefit. 
Commenter (11395) asserts that risking re-identification by increasing access to study data is not necessary, given the availability of data to other scientists through National Institute of Health (NIH) repositories and peer-review journals, and the alternative techniques available to verify conclusions of scientific studies. The commenter states that, under the proposed rule, a peer reviewed study that has been replicated many times by different investigators using different data may be excluded from consideration, while one that uses an inferior database that is publicly available would be included.
Commenter (11489) states that the incorporation of conclusions from peer-reviewed, quality science is critical to the integrity and effectiveness of the EPA's regulatory and rulemaking processes and its mission "to protect human health and the environment." 
Commenter (11498) states that the model of down-weighting research maintains the major flaw in deeming data that cannot be made completely public  -  for legitimate and legal reasons to protect the privacy of health and CBI  -  less scientifically valid or valuable. The commenter states that it mischaracterizes the scientific process and the range of mechanisms for disclosing and protecting scientific research results for decision-making, implying that peer-reviewed scientific research data that are not available in its raw form is not rigorous enough for use in policy; this is simply not true. Commenter (11480) states that much of the proposal hinges on data availability, which the Agency appears to erroneously conflate with accuracy instead of relying on well-established principles for good science, such as peer review. 
Commenter (11926) states that a key factor used in assessing research findings is publication in a peer-reviewed journal. Commenter (12617) states that peer review includes the evaluation of the authors' study design, research questions, methods, results, data presentation, interpretation, and conclusions. Commenter (12404) states that the proposed exclusion from consideration of any study for which underlying data are not made publicly available is not consistent with sound scientific practice because the scientific community uses other tools to validate studies without access to all underlying "raw data" including peer review. Commenter (13788) states that the EPA has long relied on peer review and already has extensive policies in place regarding influential scientific information to assess the quality of the scientific research. 
Commenter (14397) asserts that the standard process of peer review already ensures that the underlying research data and models used in scientific decision making "are publicly available in a manner sufficient for independent validation" and ensures that these studies adhere to the appropriate standards of transparency, repeatability, accuracy, appropriateness of data, and explain all assumptions included in the research. The commenter states that the internationally accepted peer review system already includes mechanisms for excluding CBI and PII. The commenter states that, for these reasons, the EPA's proposal is unnecessary and purports to solve a problem that does not exist.
Commenters (11395, 12404, 12617) suggest there are several mechanisms available for validating studies, including reanalysis, replication, and reproduction of data, which do not require access to all underlying raw data. Commenters (11395, 12404) state that available tools include replication of a study using the same methodology but different data sources and/or reproduction of a study's conclusions using different methodologies and data. Commenter (11395) states that, under the proposed rule, a study that has been replicated many times by different investigators using different data may be excluded from consideration, while one that uses an inferior database that is publicly available would be included.
Response: The EPA finds that comments received from the 2018 proposed rule and the 2020 SNPRM have merit, in part. In some cases, prior peer review or publication may be adequate, in which case initiating a new peer review may unnecessarily delay the EPA's development of a significant regulatory action or influential scientific information. However, as detailed in OMB's Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states, that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary". Peer review does not typically include reanalysis of the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information) and, thus, peer review is not considered a replacement for the data availability requirements of this rule.The EPA also agrees with commenters that the scientific community and government agencies are making great strides in data transparency; however, improvements can still be made over existing policies and mechanisms. Many scientific publications, for example, require authors to make a data availability or data access statement, which discloses where and under what conditions the underlying study data are available. Yet the EPA cannot solely rely on data availability statements made in published research because initiatives toward greater data sharing and transparency amongst scientific journals and international organizations are still being implemented, are inconsistently enforced, and the true accessibility of data in a public repository is still limited.
Comment: Commenters (10748, 11926) support a weight of evidence approach. Commenter (10748) states that it is the replication of the science and weight of evidence of numerous  - sometimes hundreds  -  of studies that actually influence scientific opinion. 
Commenter (11926) states that, in addition to the weight of evidence from multiple domains, regulators have also examined consistency in findings from independently conducted studies. The commenter states that, in the case of humans, the finding that a chemical agent is harmful becomes more compelling as studies conducted by unrelated investigators, using different subject pools, all point to the same conclusion. 
Commenter (11926) contends that the proposal ignores the history of science-based government regulation, which has always been based on the weight of evidence obtained from a variety of sources: laboratory-based chemical and biological (test-tube) analysis, laboratory-based testing on non-human animals, and observational research of human subjects. The commenter states that an example is the weight of compelling evidence from multiple sources that led to the regulation of tobacco marketing and distribution; as is well-known, research in all of the above domains established the fact that use of tobacco products caused cancer, a variety of other diseases, and early death.
Response: The EPA is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. As further described in Section II.B, the EPA is focusing on the underlying dose-response data for this rulemaking because of the influence these data have on particularly impactful decisions at the Agency. Risk assessments and regulations that target emissions and risk reduction of one or more pollutants, contaminants, or substances are integral to the Agency's mission and the underlying dose-response data that inform the quantitative value used to evaluate and mitigate potential risk are critical to understanding the assessment or regulatory action. In addition, the data underlying the dose-response assessment are more distinct than the broad range of data informing an entire risk assessment. Therefore, the EPA is concentrating its current efforts to increase transparency on a well-defined step in the quantitative assessment of risk supporting specific Agency actions. This final rule provides an important step in furthering the progress already being made toward maximizing transparency and will provide important insight for developing future statute-specific requirements.
Comment: Commenter (12617) states that scientific understanding progresses by researchers performing studies related to others in their field, by questioning one another's methods, by updating or expanding upon their own work or that of others. The commenter states that it also progresses by scientists performing systematic reviews and meta-analyses to understand large bodies of research. The commenter states that poorly designed studies and unsupported conclusions are called out by others without needing someone to recreate every step of data processing, analysis, and calculation for every study. 
Response: The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the public's access to the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety.
The EPA's attention to data transparency is also responsive to the broader interest in greater data transparency observed in the numerous transparency initiatives in the scientific community and federal government, as well as the criticism the EPA has received from members of the public, scientific community, and Congress on the transparency of the scientific basis for EPA's decisions in previous influential scientific information assessments and regulatory actions (Refs. 19, 20, 21, 22, 23). The EPA's continued progress toward maximizing transparency is vital to building and maintaining trust with the public and credibility in the Agency's decisions.
Comment: Commenters (11915, 12454, 12596) see merit in both the Tiered Access Option and Consideration Option.
Commenters (12454, 12596) suggest the final rule should combine both approaches to establish a procedural transparency assessment framework robust enough to inform the Administrator of scientific transparency in the course of using policy-making authority. The commenters state that such a framework should be premised on the Consideration Option's reliance on policymaking consideration while also leveraging the Tiered Access Option's appreciation of different levels of transparency. The commenters state that general tiers of access  -  broadly speaking, full public access, restricted researcher access, and little outside access  -  can be used to establish a procedural transparency assessment framework that signals the extent to which a study's transparency supports its conclusions or, alternatively, introduces uncertainties into its methodology. The commenters state that this framework would serve to highlight those studies that deserve greater consideration based on the transparency of their underlying data and models. 
Commenter (12596) states that the procedural transparency assessment framework outlined in these comments would inherently mitigate concerns that transparency requirements could lead to the disclosure of sensitive information. The commenter states that, under the process described herein, the EPA would generally not be in the position of disclosing data and models. The commenter states that researchers would instead be incentivized to disclose this information in order to improve their studies' transparency assessments.
Commenter (12596) suggests that the final transparency rule can be effective without, as suggested by the Tiered Access Option, dictating when the Agency will consider a study in CAA rulemaking. The commenter states that the Final Transparency Rule can enhance transparency without, as suggested by the Consideration Option, directing how the Agency should consider science in CAA rulemakings. Instead, the Final Transparency Rule can combine both approaches to establish a procedural transparency assessment framework robust enough to inform the Administrator of scientific transparency in the course of using policy-making authority under the CAA. The commenter states that such a procedural transparency assessment framework, authorized only by the Federal Housekeeping Statute, would exist separate from the specific statutory and regulatory processes already established for reviewing science under the CAA. The commenter states that, as such, the EPA should implement the framework in a manner that complements, and does not interfere with, these processes, with the goal of better informing final policy outcomes. The commenter states that, at the same time, assessments resulting from the framework should be formatted in a manner that is readily accessible not only to the Administrator, but also the general public. The commenter suggests that the EPA can accomplish these objectives through a "Transparency Scoring System."
Commenter (12596) explains that, under this System, each study covered by the Final Transparency Rule would be given a score based on the ability of outside parties to assess the study's underlying data and models. The commenter states that the scoring parameters would be based largely on categorization as outlined by the Tiered Data Option. The commenter states that, borrowing from the Consideration Option, the Administrator would then have the ability to use this score when considering studies in the rulemaking process. The commenter states that the EPA could make this system simple or more complex, depending on how the Agency feels it would be best implemented. The commenter states that a simpler approach would assign basic "whole number" transparency scores to studies, for example:
 Transparency Score 3: Study's authors make all data and models publicly available.
 Transparency Score 2: Study's authors make a majority of data and models publicly available.
 Transparency Score 1: Study's authors make a majority of data and models only available for review by select outside researchers.
 Transparency Score 0: Study's authors make less than a majority of data and models available for review by select outside researchers.
Commenter (12596) states that the EPA could also choose to implement a slightly more complex Transparency Scoring System, to address more specific scenarios, by incorporating "interval scores." The commenter states that, for example, if all of a study's data and models were made public, but sufficient contextual information to fully replicate the study (such as underlying assumptions) was not made available, the score could be 2.75 (or some other appropriate interval). The commenter states that the final rule could provide for guidance to address the details of various intervals. The commenter states that, in assessing these specifics, the EPA's guiding principle should be to implement a balanced and workable procedural transparency assessment framework. The commenter states that, on the one hand, the framework should reflect the variables implicating transparency, even if that tends towards a more complex system. The commenter states that, on the other hand, the framework should not be so burdensome that it inhibits rulemakings. The commenter states that, either way, the Final Transparency Rule needs to either address or provide for guidance on key underlying issues.
Response: As discussed in Section III.B of this preamble, based on a consideration of the public comments on the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. The EPA is also further clarifying how the Agency will determine the consideration to afford to pivotal science in either significant regulatory actions or influential scientific information. 
Further, the EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
7.1.2.4 International Concerns
Comment: Commenters (11395, 11496, 11890, 11894, 12404, 12405, 12727, 12731) express concern that the EPA has not described how science from other countries would be incorporated. 
Commenter (11890) states that researchers in other countries could not hand over private information to unauthorized people. The commenter states that data-sharing agreements do not allow sharing personal health data even with trusted friends. 
Commenter (11894) states that seven recent North American studies were done using Canadian cohorts, for which access is extremely limited. The commenter states that access is also highly limited for all ten of the European studies under the EU General Data Protection Regulation. The commenter states that the EU requires that all PII data must be controlled by a data controller, who must show that any use of the data has been consented to by the individuals involved. 
Commenter (11894) states that 30 of the 40 plus cohort studies of PM and mortality listed in the Integrated Science Assessment (ISA) for Particulate Matter (December 2019) could not be considered at all in the first version of 30.5. The commenter states that, under the alternative, 25 of them could not be given full consideration. The commenter states that this illustrates how arbitrary the SNPRM is, and indicates the proposal violates CAA section 108 (a)(2), which defines air quality criteria, among other things, as "the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare which may be expected from the presence of such air pollutant in the ambient air." The commenter states that it is possible to say with a straight face that the Medicare studies, which have a statistical power orders of magnitude greater than any previous PM epidemiologic study, are not "useful" in identifying the effects of PM2.5 in the ambient air; yet the SNPRM would either bar them from consideration or a priori downgrade their worth.
Commenter (12404) state the proposed rule and SNPRM place an undue burden on private companies that have invested in REACH (Registration, Evaluation and Authorization of Chemicals) registration. The commenter states that, in order to register a chemical under the European REACH program so that it can be used in commerce in Europe, private sector companies have to conduct studies of their chemicals (most of these are animal or in vitro studies) and submit them to the European Chemicals Agency (ECHA). The commenter states that these studies are expensive to conduct and are considered by the companies who do so to be an investment for the purpose of gaining access to the European market. The commenter states that robust summaries of these studies are available to the public, but data, associated protocols, computer codes and models, recorded factual materials and detailed descriptions of how to access and use such information are usually not made available to the public. The commenter states that one reason for this, is that companies spend a lot of money to develop these materials. The commenter states that requiring information developed at some cost to be made public in this way could give competitors access to the European market without having to make similar expenditures; thereby giving competitors an unfair advantage. The commenter states that the EPA relied on 20 ECHA Robust summaries for its risk evaluation of Pigment Violet 29 under TSCA and on five ECHA Robust summaries for its risk evaluation of 1-Bromopropane. The commenter states that the EPA would not be able to do so under this rule.
Commenter (12405) states that international studies raise additional challenges because participants are protected by European Union (EU) law. The commenter states that, going forward, an EU study's compliance with the proposed rule will need to be reconciled with the new General Data Protection Regulation, which is seen as more restrictive than the United States' Health Insurance Portability and Accountability Act (HIPAA). The commenter states that it is clear that many studies involving people located in the European Union (EU) will have a difficult time both complying with the new directive and providing enough information to EPA to be considered. The commenter states that studies coming from other countries are vital to health determinations in the United States because people in other countries are exposed to chemicals at different rates than in the U.S. The commenter states that the inability to review and use international research in determinations will guarantee the EPA is missing major findings and important data. 
Commenter (12731) refers to comments submitted by Bernard Goldstein of the Environmental Protection Network for the SAB's August 2019 consultation. The commenter states that Goldstein's comments pointed to the example of a 2010 study on the effects of formaldehyde exposure that had 34 co-authors from seven different institutions and three different countries. The commenter states that Goldstein observed that sharing the underlying data from that study would potentially require the unanimous agreement of all 34 co-authors, many of which answer to different IRBs that must approve such sharing, and some of whom are subject to different privacy laws than those that apply in the United States. The commenter states that, as Goldstein also observed, "international studies with or without significant U.S. collaboration are becoming an increasing percentage of total environmental health research. The commenter suggests that one of the inevitable outcomes of EPA's proposal is a cumbersome barrier to being able to put together a team of international investigators for studies of humans that are potentially relevant to EPA's environmental regulations" The commenter states that neither the original proposal nor the SNPRM meaningfully considers this critical question and the implications that it poses for EPA's ability to consider regulatory science.
Commenter (12727) notes the possible limitation of use of international studies due to international barriers. The commenter references Bernard Goldstein's comments to the SAB, who pointed to the example of a 2010 study on the effects of formaldehyde exposure that had 34 co-authors from seven different institutions and three different countries. According to the commenter, Goldstein observed that sharing the underlying data from that study would potentially require the unanimous agreement of all 34 co-authors, many of which answer to different IRBs that must approve such sharing, and some of whom are subject to different privacy laws than those that apply in the United States. The commenter states, as Goldstein also observed, "international studies with or without significant U.S. collaboration are becoming an increasing percentage of total environmental health research. Isn't one of the inevitable outcomes of EPA's SNPRM a cumbersome barrier to being able to put together a team of international investigators for studies of humans that are potentially relevant to EPA's environmental regulations?" To the commenter, neither the Proposal nor the SNPRM meaningfully considers this critical question and the implications that it poses for EPA's ability to consider regulatory science.
Commenter (11395) reiterates the International Society for Environmental Epidemiology's comments on the Proposal that the Canadian Community Health Survey Cohort and the Evaluation Study of Congestive Heart Failure and Pulmonary Artery Catheterization Effectiveness (ESCAPE) are unlikely to meet the requirements of the rule due to stringent Canadian and European privacy laws and thus be excluded or devalued.,￼,￼￼ Failing to consider these and similar studies, according to the commenter, would limit the breadth and quality of information available to EPA for critical decisions.
Response: The scientific community has also embraced greater data transparency, as evidenced by data sharing and availability requirements for many high-impact journals and the emergence of organizations, such as the Center for Open Science, and international initiatives like Findable, Accessible, Interoperable, and Reusable (FAIR) data principles; Facilitate Open Science Training in European Research (FOSTER); and Guidelines for Transparency and Openness Promotion (TOP) in Journal Policies and Practices that incentivize greater transparency in research. The EPA supports these efforts and is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. 
Further, the EPA cannot solely rely on data availability statements made in published research because initiatives toward greater data sharing and transparency amongst scientific journals and international organizations are still being implemented, are inconsistently enforced, and the true accessibility of data in a public repository is still limited.
7.1.2.5 Implementation Issues
7.1.2.5.1 Breach of Contract 
Comment: Commenter (11576) states that relationships with hospitals and other medical facilities rely on the confidentiality agreed upon by both researchers and the medical network. The commenter states that, in instances where the EPA is requiring private medical information to be disclosed in order for scientific studies to be used for decision-making purposes, the risk of breach of contract is not addressed in the SNPRM. The commenter states that such agreements cannot be easily renegotiated, especially for existing published studies and data. The commenter states that the alternative proposal provided in the SNPRM does not address situations where researchers would be contractually precluded from releasing confidential information and does not offer a solution for data that cannot be contractually released. The commenter states that, even under a tiered-access approach, the EPA could effectively ignore such data and the EPA fails to provide a solution for considering such studies and underlying data under the tiered-access scenario.
Response: When promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data, the Agency shall follow best practices to evaluate potential links between exposure to a pollutant, contaminant, or substance and the effect and the nature of the dose-response relationship. 
The Agency shall also give greater consideration to pivotal science based on dose-response data that include CBI, proprietary information or PII if these data are available through restricted access in a manner sufficient for independent validation. For pivotal science where there is no access to dose-response data, or access is limited, the Agency may still consider these studies but will give them lesser consideration unless the Administrator grants an exemption under 40 CFR 30.9. The Agency will identify the pivotal science that was given lesser consideration and provide a short description of why lesser consideration was given.
7.1.2.5.2 Description of the Structure for Implementing the Rule
Comment: Commenters (11192, 11361, 11890, 11919, 12617, 12720) express concern that the proposal does not identify a structure within which to implement the rule.
Commenters (11890, 12720) contend the proposal should not move forward until it is updated with supporting analysis and reposted for public review. 
Commenter (11890) states that it appears that the EPA is still gathering ideas for how the goals of the rule might be achieved. The commenter states that it is unclear whether the rule intends to create a passive system where the data owner only provides data if requested or an active system that will require submission and storage data being considered in a science assessment or rule. The commenter states that the proposal does not specify how the EPA proposes to screen studies that would be subject to the new requirements, determine who owns the data for a given study, and request access to data from authors or institutions. The commenter states that there is no structure given for how this new rule would make data models publicly available that are protected by intellectual property and possibly patent laws. The commenter states that the rule is also silent on how and where data would be stored and protected, how the data would be shared, with whom, how much this would cost, and who would pay for it. 
Commenter (12720) states that because the proposal does not include consideration of the questions listed below, it should be withdrawn. The commenter asks:
 How will the EPA ensure that studies are considered in perpetuity if it does not constantly verify and re-verify that underlying data and models remain publicly available (e.g., if journal open-access policies were to change)? 
 Who at the EPA will certify that a study's underlying data and models have been made "available" so as to satisfy the requirements of the rule? 
 When will such a determination be made, since weeks and months can elapse between when a paper is accepted to when it is formally published? 
 What contingency has the EPA planned in the case that there arises a need to verify that all (rather than a portion) of a study's underlying data has been made, and who at EPA would be qualified to make that determination? 
 If requests for data access are granted, would study data be provided only to the requestor, or made available to a broader group? 
 How would the EPA ensure that anyone with access to the data make appropriate use of it, and not share it with other individuals or groups? 
Commenter (12720) refers to the 2020 SAB Letter to the Administrator which suggests a mechanism for interested researchers to access datasets for reanalysis. The commenter states that it is not clear who would staff such a committee and draft protocols. The commenter states that the protocols themselves are likely to conflict (or at the very least, intersect) with existing, signed agreements for many studies, including major cohort studies that involve large participation. The commenter states that, as a result, such a proposal for an added layer of federal scrutiny and approval over academic studies introduces unwieldy and impractical restrictions on already well-functioning systems. The commenter states that the proposal does not address these Commenter (12720) states that the EPA has not indicated whether or not the Agency would provide the underlying data or simply link to it. The commenter states that the EPA has not made it clear whether any Agency-initiated cleaning or transformation of the data would be publicly available. 
Commenter (10042) urges the EPA to:
 Define, clarify and specify the limitations of transparency when used by the EPA.
 Determine who outside the EPA in the rulemaking process will have access to human health information and data and when in the process those non-EPA personnel will contribute to human health risk assessments.
 Codify its approach for how, when and under what circumstances it will use human health data and information in and for rulemaking and setting standards.
 Adopt and develop SOPs for managing human health data and information which will likely have a strong anonymization component that can or will be used to make human health risk assessments.
 Establish the criteria for revisiting previously conducted human health risk assessments for which human clinical health information and data were collected.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
7.1.2.6 Consistent with the Law and Protects Privacy
Comment: Commenters (11387, 11480, 12457, 13788) express concern that the proposal's requirement to make data publicly available could be in violation of state and/or federal law.
Commenter (11387) contends that the proposed rule would require all studies used by EPA to have all their data publicly available on an internet website. The commenter states that putting study data on the internet could be in violation of the federal code that requires protection of confidentiality in human subjects research, as well as the HIPAA. Commenter states that, as highlighted in comments by the EPA Science Review Board, requiring public availability of data ignores legitimate confidentiality and privacy concerns.
Commenter (12457) states that the proposed revisions should include a detailed analysis of how this will impact patient privacy in health-related studies, what additional privacy safeguards will be implemented, and how the EPA will support transparency compliance for health data while also protecting privacy in accordance with HIPAA.
Commenter (13788) notes that the supporting private personal information that the EPA in the SNPRM proposes would have to be publicly released prior to EPA's reliance on public health studies is the kind of information that the EPA itself is generally prohibited from releasing, even when faced with a FOIA request. The commenter states that FOIA exempts "personnel and medical files and similar files the disclosure of which would constitute a clearly unwarranted invasion of personal privacy." The commenter states that, in the SNPRM, the EPA by contrast makes clear that the information subject to this rulemaking would include "[p]ersonnel and medical information and similar information the disclosure of which would constitute a clearly unwarranted invasion of personal privacy." The commenter states that, not only is this information exempt from FOIA, but Congress passed the Privacy Act of 1974 in order to prohibit agencies from releasing precisely this kind of information. The commenter states that the Privacy Act concerns the release of records, and the definition of "record" explicitly includes medical history. The commenter states that requiring public disclosure of information, where EPA itself is prohibited from disclosing it, as the threshold to allowing a study's use in regulatory decision-making, is not only unjustified, it is absurd. 
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenter (11894) states that, in general under EPA practice, if a study is funded via any kind of assistance agreement, the recipient of the funding owns the data. The commenter states that the EPA lacks access to part or all of the data and models in many cases and does not have the authority to provide public access to part or all of the data and models. The commenter states that, if the EPA chooses to constrain its ability to the use of data and models only to those which are publicly available, the system will essentially shut down.
Response: This is a rule of internal procedure that does not impose requirements on any party other than the EPA. This rule imposes no costs on researchers or their institutions, and the EPA will consider and evaluate all relevant and appropriate science in its significant regulatory actions and influential scientific information. The EPA recognizes that researchers and other third parties may voluntarily consider the EPA's position on data availability, as described in this rule, as they make their own decisions about how to conduct research and the extent to which they make the underlying dose-response data available. Researchers may choose to make more data available, but the EPA recognizes that these parties will weigh their own benefits and costs and make choices that they deem appropriate.
7.1.3 40 CFR 30.5: Option 1
7.1.3.1 Support/Oppose Option 1
7.1.3.1.1 Support Option 1
Comment: Commenters (10973, 11397, 11401, 11499, 12397, 12410, 12612, 12692, 12712) express general support for the proposed Option 1.
Commenter (10973) states that the EPA's final rule should recognize evolving and historic open data access policies. The commenter states that open data access principles are increasingly, but not uniformly, an expected element of modern research. The commenter states that this is true especially when the research is sponsored by public sources or by prominent private foundations. The commenter states that there is likewise a growing expectation for open data access in the peer-review process and upon publication. The commenter states that, despite this trend, at present only a modest percentage scientific research articles are published in an open access format, and the publication of full data sets and models alongside articles is growing but small. The commenter states that different fields of study face different data constraints based upon privacy, logistical concerns, and other drivers. The commenter states that the "tiered access" model proposed in this version of the rule will help to alleviate some of these concerns.
Commenter (11397) agrees with the Option 1 approach because it provides clear direction to the Agency: do not use pivotal science unless the underlying data and models are available and allow for independent review and scrutiny. The commenter states that this is in keeping with Executive Orders 13777 and 13783, and it also eliminates gray areas and confusion that might exist if the EPA were merely directed by the rule to "give greater consideration" to publicly available data. 
Commenter (11401) recommends that any study which has fully open public data can be considered the basis of a regulatory action, assuming of course, that the study supports the action that the EPA wishes to take. The commenter states that studies based on "CBI, proprietary data or PII with tiered access in a manner sufficient for independent validation" could likewise be considered foundational as a basis for regulatory action, but require more oversight and assurance that the system for allowing tiered access is robust. 
Commenter (11499) supports the SNPRM's modified regulatory text that provides greater scope to consider studies in which underlying data can be reasonably available through tiered access so that CBI, proprietary data, or PII remains protected while still allowing sufficient opportunity for independent validation. 
Commenter (12397) cites the Columbia River Inter-Tribal Fish Commission Consumption Survey (CRITFC 1994) that was conducted in 1991/1992. The commenter states that the calculated Feed Conversion Ratio (FCR) from this study has been the basis of FCR in the States of Washington and Oregon for a number of years. The commenter states that the data underlying the calculated FCR has never been made publicly available for the purposes trying to validate the "given" FCR. The commenter states that uncertainties associated with this study are significant. The commenter states that having this information publicly available would have enabled third parties (such as the regulated community) to understand how the FCR value was derived and to have done its own calculations as to an appropriate FCR, especially for specific tribes and states.
Commenter (12410) tiering access based on clearance levels seems like an appropriate measure to limit exposure of sensitive data. Commenter (12612) states that the tiered approach to data access will increase the functionality of the rule in increasing EPA transparency. 
Commenter (12712) states that, with EPA regulations imposing such profound, significant harm to human health, their benefits must be verifiable by independent outside researchers. The commenter supports the modified transparency rule but only on the understanding that: no significant regulatory decision will be made on the basis of pivotal scientific studies that do not at the least ensure what the modified rule calls tiered access to the data, where such tiering is consistent with applicable privacy statutes; however, every attempt will be made to de-identify data to permit wider access to the data. The commenter states that, if tiered access to the data underlying a study cannot for whatever reason be granted to outside researchers, the EPA will not rely on the study for any significant regulatory decision.
Response: The EPA acknowledges and agrees with commenters that there may be pivotal science where the underlying data are not publicly available or available through restricted access. The final rule is limited to dose-response data and, as no studies are categorically excluded from consideration, the EPA will continue to rely on the full body of the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Consistent with the requirements of this rule, the EPA will identify a subset of those studies based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints that would inform a dose-response assessment, and will give greater consideration to pivotal science for which the underlying dose-response data are available.
7.1.3.1.2 Oppose Option 1
Comment: Commenter (9587) states that the EPA's proposed modification gives claims of privacy and/or confidentiality priority over transparency requirements. The commenter states that this modification practically will impose substantial limits on the effect of the rule since privacy, confidentiality, and tiered access are all concepts and practices that inhibit full transparency. The commenter endorses the previous, unmodified transparency requirements, and states that transparency should take priority over privacy and confidentiality in EPA policymaking. 
Commenter (11182) asserts that Options 1 renders the SNPRM unfeasible, unethical, and possibly illegal.
Commenter (11500) urges the EPA to finalize the proposal under which unavailable data will not be used. The commenter states that public access to data and models is necessary not only for independent validation, but also for independent scrutiny of the data and methodologies. The commenter states that, if such materials are unavailable, the public is denied a meaningful opportunity for comment. 
Commenters (10853, 11354, 11404, 12463) generally oppose proposed Option 1.
Commenter (10853) states that increasing transparency in science is an important goal that the EPA could more productively support by funding and creating data-sharing infrastructure that researchers from all sectors can use. The commenter states that the approaches in the proposed rule are too far downstream in the scientific process; unduly prescriptive; likely to increase costs and reduce supply of useful science; and likely to introduce bias and inefficiency in EPA's process of evaluating and developing regulatory science. Commenter (11404) states that the SNPRM's modified approach of tiered access is insufficient to address privacy concerns. 
Response: The EPA disagrees with commenters that the requirements of this rule will result in any meaningful delay in promulgating regulations. While this final rule requires the Agency to evaluate the availability of dose-response data for pivotal science, the incremental burden to the Agency to carry out these requirements is expected to be small given (1) the extensive scientific review the EPA already conducts regularly and (2) that the requirement is limited to pivotal science (i.e., typically a small, though highly important, subset of the studies the EPA would review). Further, with this final rule, the EPA is maintaining language in 40 CFR 30.3 stating that the statutes that the EPA administers, or their implementing regulations, will control in the event of any conflicts with the requirements of this rule. The Agency will continue to comply with and abide by the requirements in those statutes and implementing regulations, including regulatory deadlines.
7.1.3.1.3 Revision of Option 1
Comment: Commenter (12471) suggests that, if the EPA finalizes Option 1, the following modification is offered:
      When promulgating significant regulatory decisions or finalizing influential scientific information, the Agency will only use pivotal regulatory science and/or pivotal science that includes studies with restricted data and models (i.e., those that include confidential business information (CBI), proprietary data, or Personally Identifiable Information (PII) that cannot be sufficiently de-identified to protect the data subjects) if there is tiered access to these data and models in a manner sufficient for independent validation, and studies that do not include restricted data and models if the data and models are publicly available in a manner sufficient for independent validation.
Response: The EPA agrees with commenters that it is important to consider the totality of relevant, quality research. Therefore, the EPA did not finalize the 2018 proposal or the 2020 SNPRM proposed option that would have categorically required that the underlying data and models be available for independent validation in order for studies to be considered pivotal science. However, given that transparency is an important aspect of EPA's regulatory actions and assessments it should be an important consideration in how the Agency considers pivotal science. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science is unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
Comment: Commenter (12692) states that it may be many years before a significant fraction of the studies on which EPA relies is independently validated. The commenter suggests that, until studies are independently validated, the EPA should follow Lutter and Zorn's recommendation to former House science chairman Lamar Smith, whose HONEST Act (H.R. 1430) is the legislative inspiration for the Transparency proposal:
      The bill should also allow agencies to regulate in instances where they do not possess data. Specifically, agencies may rely on research published in peer-reviewed journals, even if there is not public access to such data and the agency cannot acquire it, provided that the agency states it has unsuccessfully sought the data under an agreement providing for privacy of human subjects and protecting trade secrets. In this case the agency, however, would have to state that it is using research based on non-public data and explain why it believes the research is nonetheless sufficiently reliable to be used for regulation. 
Response: Since the purpose of 40 CFR 30.5 is to determine the consideration to afford to studies based on, among other factors, the availability of the underlying dose-response data that would support independent validation via reanalysis of the data underlying pivotal science, the appropriate stage of data would not be the processed data (data that have been computed and analyzed to extract relevant information) or the final clean data set that is provided with a publication. At these two stages of data, the analysis has already been conducted, and the results have already been determined. In order to determine if these results are valid, data that had not already been computed and analyzed are needed. 
In this final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. As noted by commenters, the step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
7.1.3.2 Definitions 
Comment: Commenter (11890) states that, in the SNPRM, new terms are introduced that require more explanation and transparency. The commenter states that the language "appropriate techniques for tiered access" is used, yet "appropriate techniques" are not defined. 
Commenters (9587, 11576, 11890, 11894) express concern that "tiered access" is not defined. 
Commenter (9587) asserts that the EPA should define tiered access precisely, to maximize transparency. The commenter states that the EPA should scrutinize its regulatory models carefully, and make sure that its own definition of tiered access defines all terms with absolute clarity, provides equally clear procedures designed to maximize transparency, and facilitates reproduction studies.
Response: Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
7.1.3.3 Data That Cannot Be Sufficiently De-identified
Comment: Commenters (9455, 10866, 11361, 11395, 11473, 11576, 11890, 12403, 12546, 12720) contend that the tiered structure presented in the SNPRM does not adequately address concerns about re-identification of study participants, disclosure of private health data, and/or negative effects on future study participation, as described below.
Commenter (9455) asserts the proposed rule's system of tiered access would violate medical confidentiality laws and standards because it only reduces the risk of re-identification and it allows, illegally and unethically, sharing of confidential data. The commenter states that posting into the system imposes risk in ways that have not been properly evaluated in the proposed rule. The commenter states that, as the tiered program could only "reduce the risk of reidentification and mitigate disclosure privacy risks," it would break laws on privacy and standards of medical confidentiality. The commenter states that it would make all the data sitting ducks for hackers. 
Commenter (10866) asserts that the proposed tiered access strategy for consideration of protected health information does not adequately protect the identity and health information of participants included in research studies. Commenter (11890) states that the rule must recognize that the re-identification of study participants is possible even when stripped of the variables typically considered PII (e.g., under HIPAA). 
Commenter (11395) is concerned about the potential for re-identification of study participants in the tiered access structure. The commenter states that, because environmental studies by necessity need to use location data to establish exposures, a very significant percentage of study participants can be identified unless the data are masked to a degree that would not allow reanalysis. The commenter states that, recently, a peer reviewed study examined the identifiability of records from an environmental health study in northern California. The commenter states that, using data considered by the HIPAA to be sufficiently de-identified to be made public and which involved far fewer variables than would be required for the reanalysis proposed by EPA, the researchers were able to correctly identify over 25 percent of the study participants. The commenter also refers to the 2002 National Academy of Sciences reports which states that:
      In an experiment to discover whether confidentiality could be preserved while opening the data for public review, the study investigators attempted to disguise the identity of the study participants. They deleted as many features as possible from the questionnaires, such as the name, the state file number, the mother's maiden name, and the name of the person providing the information. However, they needed to retain a minimum set of features if other scientists were to be able to replicate the basic findings of the study... They found that even this minimum set of features could allow for identification of research participants.
Commenter (11473) asserts that the new "tiered" approach does not prevent loss of privacy. The commenter refers to an editorial published in the Lancet (Thorp et al., 2019) and signed by Editors of Science, Nature, PLOS, PNAS, Cell, and Lancet: 
      As leaders of peer-reviewed journals, we support open sharing of research data, but we also recognise the validity of scientific studies that, for confidentiality reasons, cannot indiscriminately share absolutely all data. Datasets featuring personal identifiers -- including studies evaluating genomes of thousands of people to characterise medically relevant genetic variants -- are but one example. Such data may be critical to developing new drugs or diagnostic tools but cannot be shared openly; even anonymised personal data can be subject to reidentification (e.g., Sweeney, 2013; De Montjoye et al., 2015; Rocher et al., 2019) (attached), and it has been a longstanding practice for agencies and journals to acknowledge the value of data privacy adjustments. The principles of careful data management, as they inform medicine, are just as applicable to data regarding environmental influences on public health. Discounting evidence from the decision-making process on the basis that some data are confidential runs counter to the EPA stated mission `to reduce environmental risks...based on the best available scientific information'."(USEPA, 2016).
Commenter (11576) states that the supplemental notice admits that under a tiered approach, the EPA will not fully protect confidential information. The commenter states that, while the EPA claims that its tiered-access approach will allow for a reduction in the risk of re-identification, it cannot guarantee that it can fully prevent all such data from being reidentified. The commenter states that an analysis published in 2019 assessed the risks associated with the sharing of data within environmental-health studies, including genetic and medical records, and found an increased likelihood of re-identification. The commenter states that there is little discussion as to the criteria the EPA will rely upon to develop such an approach and even then, the EPA can only guarantee a reduction of risk -- which means there would remain individuals whose private medical information is at risk of being disclosed. 
Commenter (11926) states that there are substantial risks involved in sharing of individual level research data, especially if data are available to anyone in a public release. The commenter states that enormous technological advances have been made in data linkage techniques, and substantial resources have been devoted to data linkage as businesses, government agencies and sophisticated rogue hacking groups seek to find out as much as possible about as many people as possible.
Commenter (12546) states that researchers in 2015 were able to reidentify 90 percent of their study population as unique individuals and to uncover their records, knowing just four random pieces of information, due to the uniqueness of human behavior. The commenter states that a more recent study in 2019 modeled the likelihood of individual reidentification in a "heavily incomplete" anonymized dataset, and found "that 99.98% of Americans would be correctly re-identified in any dataset using 15 demographic attributes." The commenter states that this research shows how, with only a few pieces of information, current technology can already deidentify data that scientists currently consider sufficiently anonymized.
Commenter (12720) expresses concern that the EPA does not define a level to which study data can be "sufficiently" de-identified in the proposal, which leaves this determination open to arbitrary influence. The commenter states that this description of a tiered access system is vague in not defining a "sufficient" level of de-identification. The commenter states that the EPA has not submitted any specific plan for such a determination to be made, nor has any independent group evaluated such an approach.
Commenters (9455, 11361) express concern that the tiering approach may not be consistent with IRB requirements. Commenter (9455) states that the possibility that medical data, even if ostensibly de-identified, will be posted for review would never get past any good IRB. The commenter states that the requirements of various privacy protection laws and of medical ethics preclude such posting. The commenter states that IRBs enforce these requirements in their reviews of grant proposals. The commenter states that most federal agencies that fund health research, including EPA and National Institute of Environmental Health Sciences (NIEHS), require strict medical confidentiality, and IRBs make sure that all grant proposals ensure confidentiality.
Commenter (11186) states that, while the rule discusses risk reduction techniques and creating multiple versions of a single dataset with varying levels of specificity and protection, in fact, creating more datasets can involve more risk of reidentification and breaches of confidentiality. 
Commenters (11387, 12404, 12423) assert that health studies of environmental exposures require sufficient location information to determine exposure, and that this location information makes it impossible to adequately de-identify health information. Commenter (12404) states that this is particularly true in some air pollution studies, in which participants' addresses are used to estimate exposure.
Commenter (11387) contends that the requirement that all study data be available on an internet website, means it would always be possible to use diagnosis and location information to figure out the identities of one or more people in the study. The commenter states that this violates the person's confidentiality rights, and means the study cannot be used (if it was completed in the past) or conducted (if it is a proposed future study). 
Commenter (12423) asserts that the small populations of most tribes and tribal communities and their restricted geographic locations (i.e. specific reservations and villages) make it impossible to sufficiently de-identify raw data. The commenter states that tiered access does not alleviate this issue. The commenter states that, under the proposed rule, these studies would either be excluded or heavily discounted. The commenter states that this scenario would be played out hundreds to thousands of times throughout Indian Country. The commenter states that it should be noted that rural America in general shares some features with tribes, such as substandard housing, exposure to dust from unpaved roads, poor healthcare infrastructure, greater number of businesses that are below a range of federal statute threshold reporting levels and are more likely to rely on water systems and private wells unregulated by the Clean Water Act (CWA) and Safe Drinking Water Act (SDWA). The commenter states that, for example, there are 229 Alaska tribes living in remote and isolated villages averaging just 330 people. The commenter states that they are unconnected by road to any other populations, so that any community's geographic, hydrologic, and geological features relevant to participant selection criteria could unmask the tribe. 
Commenters (9455, 11576, 12404, 12423, 12464) contend that certain studies show that it is difficult to make data public without also making participants identifiable.
Commenter (9455) contends that there is no way to completely ensure that re-identification will not happen. The commenter states that the references in Boronow demonstrate that the environmental health research community has futilely grappled with this problem for many years. 
Commenter (11576) refers to a study which discusses how even limited information made available from multiple studies involving the same dataset allowed researchers to successfully re-identify study participants. The commenter states that researchers in the study were able to re-identify study participants despite the exclusion and redaction of information that could not be shared under the safe-harbor provision of HIPAA. The commenter concludes that the EPA's proposal poses significant threats to personal privacy -- threats the EPA has yet to acknowledge and address. The commenter states that, while the EPA claims that its tiered-access approach will allow for a reduction in the risk of re-identification, it cannot guarantee that it can fully prevent all such data from being reidentified. The commenter states that there is little discussion as to the criteria the EPA will rely upon to develop such an approach and even then, the EPA can only guarantee a reduction of risk -- which means there would remain individuals whose private medical information is at risk of being disclosed. 
Commenter (12404) states that it is difficult to make data public without also making participants identifiable. The commenter states that this is particularly true in some air pollution studies, in which participants' addresses are used to estimate exposure. The commenter states that this means, that in some cases, information about participants' exposures could be used to figure out their addresses. The commenter states that a 2002 study by the National Research Council of the National Academy of Sciences found that, even after the deletion of some personal information from participants, if all the information necessary to reproduce a study's findings remained, it was possible to identify the participants.
Commenter (12423) asserts that the small populations of most tribes and tribal communities and their restricted geographic locations make it impossible to sufficiently de-identify raw data. The commenter states that, for example, a retrospective longitudinal cohort study was carried out in 2001 to examine whether there was a significant association between a range of short-term health risks and the surrogate inhalation exposures to solid waste in four Alaska Native villages. The commenter states that information was collected from 95 percent of residents, including education level, tobacco use, honey bucket use (plumbing), self-reported asthma, self-reported short-term health symptoms, and approximate total subsistence food consumption. The commenter states that, in another study conducted in 2006, and published in the American Journal of Epidemiology, birth records were used to examine the association between hazardous waste content in Alaska Village landfills and a number of birth outcomes, including congenital anomalies. The commenter states that these two studies are seminal in that they are the first and only systematic studies carried out on impacts in rural Alaska Native villages. The commenter states that they have since informed EPA policy, as well as delegated state Resource Conservation and Recovery Act (RCRA) implementation and policy. The commenter states that a substantial portion of results pertain as well to the rest of rural Alaska communities, representing in total about half of the state's population. The commenter states that, in the case of the retrospective cohort study, obtaining participant consent 19 years later is not possible. The commenter states that, in the case of the second study, the use of the Alaska Birth Record Registry was conditioned upon strict confidentiality and scrubbing procedures because of the medical records and sensitive information contained. The commenter states that, with communities as small as 50 people and just a single annual birth, it would be straightforward to identify individuals and individual medical records. The commenter states that permission from the State to modify the research agreement and release the data would not be granted.
Commenter (12464) states that recent research suggests that the risk of re-identification for environmental health data is greater than traditionally thought. The commenter states that a recent paper demonstrates that it is possible to successfully re-identify environmental health study participants using information that is publicly or commercially available. The commenter states that, if data with geographic information about the study participants becomes public, that data can be combined with information that is readily available online such as voter lists, tax and real estate information, or information that is available from data brokers. The commenter states that, in light of the risk of re-identification with environmental health data, researchers would have a strong interest in ensuring the data is restricted and not publicly available. The commenter asks if researchers are able to make this determination or will the EPA demand that certain studies be made public? The commenter asks if the EPA will decide, what process will be available if the researchers and the EPA disagree about the risk of re-identification?
Commenter (11576) states that, in the original notice, the EPA mentioned "[o]ther federal agencies [that] have developed tools and methods to de-identify private information for a variety of disciplines" as examples of approaches to de-identify and protect confidential medical information. The commenter states that, however, the Health and Human Services guidance referenced by the EPA acknowledges that de-identification does not fully protect patient information, stating that "[b]oth methods, even when properly applied, yield de identified data that retains some risk of identification. The commenter states that, although the risk is very small, it is not zero, and there is a possibility that de-identified data could be linked back to the identity of the patient to which it corresponds." The commenter states that, as commenters noted in 2018, de-identifying personal information has thus far proven to be ineffective. The commenter states that the SNPRM does not offer a resolution that would fully prevent de-identified information from being re-identified.
Response: Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The EPA follows an objective, unbiased process for identifying and evaluating scientific studies and already identifies key or pivotal studies in some of its actions (e.g., Integrated Risk Information System (IRIS) assessments). The EPA intends to issue implementation guidelines and statute-specific rulemakings that will further describe these criteria and how the EPA will identify pivotal science in its assessments and rulemakings.
The EPA did not finalize the 2018 proposal or the 2020 SNPRM proposed option that would have categorically required that the underlying data and models be available for independent validation in order for studies to be considered pivotal science. However, given that transparency is an important aspect of EPA's regulatory actions and assessments it should be an important consideration in how the Agency considers pivotal science. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science is unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
7.1.3.4 Exclusion or Discounting of Data and Studies in Tiered System
Comment: Commenters (11391, 11479, 12404, 12688, 12731) express concern that, under Option 1, the EPA would not prevent important and relevant scientific information from being disregarded.
Commenter (11391) states that, as scientists discussing the feasibility and usefulness of a tiered or controlled-access approach have pointed out, "controlled access does not permit researchers to curtail national legal and ethical policies on privacy and data protection." The commenter states that, while the EPA seems to suggest that in many instances privacy issues can be sufficiently addressed through de-identification, the EPA's own SAB has concluded that "even de-identified datasets present risks of re-identification." The commenter states that, as such, there are some circumstances where scientists simply cannot share data, even if they promise only to share it with a select group of people through a tiered access regime. The commenter states that, even under Option 1, then, the EPA would still be forced to disregard large amounts of valuable scientific information where a tiered or controlled-access approach is not legally or ethically permitted.
Commenter (11479) states that the tiered-access approach fails to address a central concern -- that it would require the release of private data for EPA to be able to use studies that rely on such data. The commenter states that, under a tiered-access approach, which would allow select researchers to gain access to sensitive data, the vast majority of studies that rely on nonpublic data  -  data such as personally identifying information, CBI, and intellectual property  -  would still be prohibited from use by EPA. The commenter states that this is because restrictions on disclosure of such data (as detailed in the next section) would still apply, even when such data are released to a select few researchers in a tiered-access system. The commenter states that, for example, the EPA relies on many studies that use personally identifying health data. The commenter states that, to use such data, researchers obtain ethics approval from IRBs and sign legal agreements with hospitals, health maintenance organizations, and other bodies that prevent them from sharing health data publicly. The commenter states that these legal and ethical obligations still apply, even if a tiered-access system exists. The commenter states that, thus, sharing health data in a tiered-access system would require researchers to breech legal and ethical agreements, as described in detail in the next section. The commenter states that researchers would be unlikely to participate in such a system if their research relies on sensitive data, consequently, the EPA will not be able to use much relevant research relying on nonpublic data, even if a tiered-access system is offered.
Commenter (11390) states that, while the EPA's revised proposal contemplates that "tiered access," such as restricted access to confidential data for independent validation by "authorized researchers," would allow research institutions and researchers to authorize restricted access. The commenter states, however, this requirement is not consistent with many contractual agreements with research subjects, and will be infeasible or impossible to implement both retroactively and prospectively in a way that allows the Agency to rely on high-quality science.
Commenter (11498) states that this restricted access approach presupposes permission and participation from the scientists doing the research. The commenter states that, since the EPA does not have the authority to grant access to confidential and private research data, this approach would depend on the consent and approval of the scientists and institutions who conducted the research. The commenter states that it is unclear if this is at all possible, as researchers and institutions, in the example of many groundbreaking epidemiological studies, enter into contracts with participants to keep their health and other sensitive information private. The commenter states that it is improbable the researchers would or could alter these legal contracts after studies are concluded to allow individuals selected by EPA to gain access to such protected information, even in a tiered scheme. The commenter states that, thus, this approach would likely exclude many credible and valuable studies, including ones containing private information that EPA has benefited from in the past. The commenter states that this reinforces our ongoing serious concern that this proposal threatens the use of the best available science in its decision-making. 
Commenter (12404) states that the tiered approach would permit data that cannot be made broadly public to be shared with a limited number of scientists. The commenter states that, if this proposal were to become a final rule, it is possible to imagine that researchers might seek and obtain informed consent from study participants to do this kind of sharing of identifiable information. The commenter states that, if they did, it might be more difficult to find willing study participants, especially in communities of color, which have good historical reasons to distrust medical researchers and which are often the most heavily exposed to pollution. The commenter states that, even if researchers were able to seek and obtain consent for this kind of limited sharing in future studies, and even if doing so did not reduce study participation in communities of color or elsewhere, this rule would still exclude a tremendous amount of scientific research conducted before there was any reason to obtain such consent.
Commenter (12688) states that Option 1, which would require consent and approval from the researchers and institutions that collect data, ignores the realities of public health research involving human subjects, which often necessarily involve contractual arrangements with subjects to protect privacy and confidentiality. The commenter states that it is certain that many past studies would not be amenable to "tiered access" to data, and this requirement would inevitably impede the use of many future studies to inform Agency actions as well.
Commenter (12731) states that, in altering its proposed approach, the EPA purports to address objections to the Agency's exclusion of valid data and models containing confidential, proprietary, or personal data that cannot be sufficiently de-identified to protect data subjects. The commenter states that the EPA's tiered access approach is an inadequate response to these objections. The commenter states that, if tiered access does not exist, for whatever reason, the Agency will ignore all studies for which data and models are not publicly available. The commenter states that, thus, the tiered access modification to the original proposal fails to cure its essential defect: the Agency would continue to disregard valid studies without any rational grounding in its statutory authorities, and in plain violation of numerous statutory requirements.
Commenter (11500) states that, when excluding materials unavailable in a manner sufficient for independent validation, the Agency should clearly identify the materials that have been excluded and explain the precise bases for exclusion.
Response: Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The EPA follows an objective, unbiased process for identifying and evaluating scientific studies and already identifies key or pivotal studies in some of its actions (e.g., IRIS assessments). The EPA intends to issue implementation guidelines and statute-specific rulemakings that will further describe these criteria and how the EPA will identify pivotal science in its assessments and rulemakings.
The EPA did not finalize the 2018 proposal or the 2020 SNPRM proposed option that would have categorically required that the underlying data and models be available for independent validation in order for studies to be considered pivotal science. However, given that transparency is an important aspect of EPA's regulatory actions and assessments it should be an important consideration in how the Agency considers pivotal science. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science is unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
7.1.3.5 Research Data Centers
Comment: Commenter (11392) notes that the SNPRM references a pilot project with the Centers for Disease Control and Prevention (CDC) involving a secure data enclave for sensitive EPA datasets. The commenter asks if the CDC was involved in writing this rule, do they have the resources to host the EPA's data and models and moderate tiered access, will the CDC be making the decision as to who can access the data and at what level, and will CDC be reviewing applications for access? The commenter states that the SNPRM does not address any of these questions. The commenter states that, if the EPA intends to make its arrangement with the CDC permanent, and establish the CDC as the host of a secure data enclave for studies considered in Agency rulemaking, the EPA must clearly articulate its rationale for partnering with a separate federal agency rather than performing the work internally. The commenter states that the EPA must also describe how such an unusual arrangement would be structured; it must be clear that the CDC is involved in this rulemaking. 
Response: Proposed 40 CFR 30.2 in the 2018 proposed rule included a definition of "research data." In the 2020 SNPRM, the EPA deleted the proposed definition of "research data." The EPA is not including a definition of "research data" in this final rule given that it is finalizing the definition of "data."
Comment: Commenters (11479, 12435, 12710, 12720, 12731) assert that the proposed model of tiered access for data involving and the precedents the SNPRM cites for it, are inappropriate to the work of the EPA. 
Commenter (11479) states that the EPA's SNPRM points to the CDC's Research Data Center (RDC) as a model, and the EPA has referenced other practices by CDC and the U.S. Food and Drug Administration (FDA) as relevant models for the Agency to follow in implementing a tiered-access approach. The commenter states that there are significant reasons that these CDC and FDA systems will not work for EPA. The commenter states that, unlike CDC and FDA, the EPA is primarily focused on environmental threats, and deidentification of personal health data is challenging and sometimes impossible, as noted above. The commenter states that, by contrast, data from clinical drug trials used by FDA and medical data aggregated at the hospital or county level used by CDC can often more readily be disclosed publicly, because doing so will not reveal identities of study subjects. 
Commenter (12435) states that the types of studies that inform the FDA are fundamentally different from those that inform the work of the EPA. The commenter states that the RDC of the CDC is not easily or affordably accessible and does not provide the raw data access that this rule demands. 
Commenter (12710) asserts that the model for tiered access cited in the preamble (from the RDC within CDC), is relevant only to data owned by the federal government. The commenter states that no model exists, nor does the EPA propose one, for tiered access to studies generated outside the federal government (e.g., states, universities, non-governmental organizations (NGOs)). 
Commenter (12720) states that the EPA does not explain why the RDC approach would work in the EPA context and the wide scope of scientific data and models that would be subject to new public data release requirements under the proposed rule. Commenter (12731) asserts that the federal tiered-access models the EPA has pointed to are relevant only to "a small fraction of published epidemiology research" and that these platforms do not address access to datasets held by universities or by private organizations like the American Cancer Society.
Commenter (12731) states that the closest the EPA comes to describing its envisioned tiered access approach is a brief reference to the RDC. The commenter states that the EPA's reference to the RDC is wholly inappropriate, as the data features are entirely different from the scope of scientific information the EPA is capturing in the SNPRM. The commenter states that the data housed within the RDC are either developed directly by the federal government or collected through federal government- designed and managed programs. The commenter states that these data are highly standardized, and the platforms housing the data are developed and managed by the government. The commenter states that the RDC is a highly controlled, integrated, and coordinated data infrastructure. The commenter states that, in contrast, the EPA's proposal targets all potential "data and models" to be considered by the Agency that are developed by a wide breadth of entities inside and outside government and that are highly variable including in terms of how they are generated, collected, curated, and stored. The commenter states that any reference to the RDC as an analogous potential model for EPA's tiered access approach is highly inappropriate and disingenuous.
Commenter (12720) states that the EPA does not grapple with the fact that the RDC restricted-use data is not "publicly available" in that there are multiple barriers to data access. 
Response: This rule is designed to build upon prior EPA actions in response to government-wide data access and sharing policies. The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator (see Section III.G). See Section III.E for a description of the factors the EPA will consider when determining the consideration to afford to pivotal science when the underlying dose-response data are not available for independent validation.
Further, proposed 40 CFR 30.2 in the 2018 proposed rule included a definition of "research data." In the 2020 SNPRM, the EPA deleted the proposed definition of "research data." The EPA is not including a definition of "research data" in this final rule given that it is finalizing the definition of "data."
Comment: Commenters (10038, 11176) support the concept of tiered access to data in the current draft of the rule and for piloting the use of the RDC of the CDC National Center for Health Statistics (NCHS). 
Commenter (11176) states that the NCHS RDC is one example for providing some level of tiered access. The commenter states that the NCHS also, along with more than a dozen other federal entities, provides data access through Federal Statistical Research Data Centers (FSRDCs). The commenter states that the EPA actually provides access to its Pollution Abatement Costs and Expenditures (PACE) survey through the FSRDCs. The commenter states that one potential limitation of the FSRDC network is that work conducted therein must be for a statistical purpose and must be for research as well as the following stipulation: "An agency with regulatory or enforcement roles that also conducts or supports research may be able to house their data in RDCs if there is a clear delineation between enforcement and research in the agency. Further, the mission of RDCs is to facilitate research." The commenter states that other existing access methods may provide additional flexibility and are actively being used by federal agencies. The commenter states that two such access examples are the NORC Data Enclave and the Inter-university Consortium for Political and Social Research (ICPSR) Virtual Data Enclave. The commenter states that the former is used by the two United States Department of Agriculture (USDA) statistical agencies, for example, and the latter by the Bureau of Justice Statistics. The commenter states that, if more restricted access is needed, programs such as the Bureau of Economic Analysis' Special Sworn Researcher Program provides access to highly confidential firm data, under tight security. The commenter states that, while the EPA is actively working to provide more access to its data, its involvement in the sharing of its sensitive data through research data centers and data enclaves is minimal, limited to a single survey to our knowledge.
Response: Access to restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. given that transparency is an important aspect of EPA's regulatory actions and assessments it should be an important consideration in how the Agency considers pivotal science. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science is unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. 
7.1.3.6 May Be Limited to Authorized Officials and Researchers
Comment: Commenters (11487, 11890) express concern that the rule does not define "authorized researchers." Commenter (11890) states that, in the SNPRM, new terms are introduced that require more explanation and transparency.
Commenter (11487) states that the SNPRM language regarding tiered access is far too vague and limited to engender confidence among researchers or those who might consider participating in studies. The commenter states that, for instance, it does not define "authorized researchers" or mention the IRBs that govern human-subjects research in academia. 
Response: Proposed 40 CFR 30.2 in the 2018 proposed rule included a definition of "research data." In the 2020 SNPRM, the EPA deleted the proposed definition of "research data." The EPA is not including a definition of "research data" in this final rule given that it is finalizing the definition of "data."
This rule does not require any researcher or other outside entity to provide data to the EPA. Nor does the rule categorically exclude studies -- even studies where the underlying dose-response data are not available for independent validation -- and therefore any incidental impact on researchers who are developing science and deciding whether to make the underlying dose-response data available is negligible. Instead, it governs internal agency procedures for determining the consideration to afford various studies according to factors that include data availability.
Further, the EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (11391) expresses concern that there is no discussion regarding who at the EPA would make decisions about when access to data should be limited or how, or how the EPA would manage the potentially significant additional capacity necessary to make and execute such decisions. The commenter states that, even an unambiguous tiered access system would not address the fundamental underlying problem: there are situations in which researchers simply cannot make data available, even if they or the EPA agree only to share that data with a select group of people. 
Commenter (11403) contends that, since the data would be PII, it would be available to the public only through tiered access, and the vast majority of the members of the public would not have more access to the data than it would under the current system. The commenter states that the proposed rule attempts to allow for the use of some CBI and PII by permitting use only if there is tiered access to the data or if the data can be sufficiently de-identified. The commenter states that, in a tiered access system, public transparency is restricted, not just access to the data. The commenter states that, while the EPA identifies the RDC as the model for its approach, why the EPA believes it is appropriate to create another database for confidential health information within the EPA is mystifying. The commenter states that this data, according to EPA, "may be limited to authorized officials and researchers and not provided to the general public." The commenter states that it would be at the EPA's discretion to determine if a member of the public was qualified to access scientific information. The commenter states that the EPA's proposed system is vastly more "secret" than the method it thinks needs fixing.
Response: The EPA is finalizing a rule of internal agency procedures, including how the Agency will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information for independent validation. The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
7.1.3.7 Implementation of Tiered System
7.1.3.7.1 Costs and Burdens
Implementation Costs Not Fully Considered
Comment: Commenters (11176, 11391, 11890, 11894, 12430, 12731, 12464) express concern that administrative burdens associated with implementation of the rule are not fully considered in the proposal and SNPRM. 
Commenter (11391) states that there is considerable resistance to tiered access in the scientific community, in no small part because of the additional administrative burdens imposed by the tiered-access agreements. The commenter states that there is no indication in the 2020 SNPRM that the EPA has considered those burdens or possible approaches for managing them. Commenter (12430) states that a requirement to provide multiple tiers of access would disproportionately burden academic, public, and even smaller corporate researchers. Commenter (12731) states that the tiered access approach would likely be difficult and costly to administer, but the EPA has not explained, even in general terms, how this approach would be implemented, the degree of burden it would impose, or who would bear the costs. 
Commenter (12464) states that the SNPRM says nothing about the costs of the tiered access alternative nor who will bear those costs. The commenter states that, without any details, or cost analysis, it is not unreasonable to assume that the cost of the tiered access approach would fall on researchers, other federal agencies such as the CDC and other third parties.
Commenters (12430, 12617, 12731, 12464) express concern that the tiered access alternative will introduce additional costs for both the government and individual researchers. Commenter (12430) states that a requirement to provide multiple tiers of access would disproportionately burden academic, public, and even smaller corporate researchers. 
Commenter (12617) states that researchers would have to dedicate time and resources to figuring out the logistics of sharing, especially with a tiered-access approach. The commenter states that researchers would have to determine how to safely protect medical and proprietary information while designing different levels of access. The commenter states that, while the SNPRM says researchers could create "multiple versions of a single dataset with varying levels of specificity and protection," the extra time and expense seems unreasonable and without any clear benefit to EPA's mission.
Commenter (12731) states that, in a recent briefing with staff on the House Committee on Science, Space, and Technology, EPA staff revealed that "key implementation responsibilities [for tiered access] will fall on the research community." The commenter states that the EPA's apparent expectation that it will impose massive new costs on the research community without acknowledging as much in the SNPRM is arbitrary and a violation of notice requirements. The commenter states that it would further privilege the EPA's reliance on studies from wealthy organizations or those with a financial incentive to expend the necessary resources for EPA to consider their work.
Commenter (12464) states that the tiered access alternative will be even more costly to implement than a requirement that all data and models be made available to the public. The commenter states that, for example, some researchers will have to submit data to a secure data repository, the operator of the repository will have to maintain and operate the facility, and researchers who want to reanalyze the data will typically need to pay fees to access the data as well as pay to travel to the repository. 
Commenters (12645, 12731) express concern the proposal and SNPRM do not address the burden of obtaining the consent of researchers. Commenter (12645) notes that there is an increasing number of co-authors in each published study and asks if each author would have to agree to share the data so that any one author would have veto power over sharing? The commenter states that the EPA could find out by sampling the scientific community represented in cited publications underlying its regulatory decisions, but like so many other aspects of the initial transparency rule and this SNPRM, it has not bothered to do any preliminary analyses or to have such analyses done by the National Academy of Sciences (NAS) or the SAB. 
Commenter (12731) states that the EPA's proposed "tiered access" approach fails to engage with or even acknowledge concerns that have been presented multiple times to the Agency regarding the cost and feasibility of establishing such arrangements on a routine basis. The commenter states that the Health Effects Institute (HEI) comments to SAB noted that tiered-access arrangements, such as proposed in the SNPRM, require the consent and collaboration of researchers and can be "challenging" to establish.
Response: The EPA agrees that neither the 2018 proposed rule nor the 2020 SNPRM included a characterization of costs to the Agency. The EPA emphasizes that this is a rule of internal procedure promulgated under the EPA's housekeeping authority. However, the EPA has identified some incremental costs that the Agency may incur as a result of this final rule. As stated in Section III.A.2, the EPA will continue its current practice of conducting extensive review of scientific studies during the development of significant regulatory actions and influential scientific information. The additional procedures required by this rule only apply to pivotal science, which is a subset of the total number of studies that the EPA would evaluate. Given the costs of the current robust process for identifying and reviewing scientific studies and documentation that are existing Agency practice, as well as that the determination of dose-response data availability is limited to pivotal science underlying significant regulatory actions and influential scientific information, the EPA anticipates that the incremental costs of this rule will be small. The Agency may also incur other administrative costs to perform analyses and evaluations to support activities such as exemption decisions made by the Administrator, and documenting these or other decisions made pursuant to the requirements of the final rule. Again, the Agency anticipates that the incremental costs for these activities will be small relative to current administrative costs for developing significant regulatory actions or influential scientific information.
Comment: Commenters (10038, 11392, 12464, 12715) express concern that the costs of data repositories such as the NCHS's RDC would exceed millions of dollars annually. 
Commenter (10038) states that the annual operation of such data repositories costs several million dollars above and beyond the initial costs of establishing such centers. Commenter (12715) states that, given the expanded scope of the SNPRM, which now covers not just pivotal regulatory science but also pivotal science supporting influential scientific information, the costs of the proposal may well exceed the $100 million annual threshold for Executive Order 12866.
Commenter (11392) notes that the Secret Science Reform Act of 2014, a legislative precursor to this SNPRM, was estimated by the Congressional Budget Office to cost "$250 million annually over the next few years" should it have passed into law and asks how much will this SNPRM cost, and what money will the EPA use to undertake a project of this magnitude that has not been authorized by Congress? Commenter (12464) states that the tiered access alternative will introduce additional costs for both the government and individual researchers. The commenter states that, as explained in our previous comment letter, the Congressional Budget Office ("CBO") estimated that a bill similar to the proposal would cost the EPA as much as $250 million per year. The commenter states that a tiered access approach will be even more costly to implement than a requirement that all data and models be made available to the public.
Commenters (11176, 11192, 11479, 11487, 11576, 12435, 11900) express concern that the SNPRM does not detail the significant costs of establishing and maintaining an RDC-type system. Commenter (11900) states that, before requiring such a system, the EPA must clearly explain how it will fund the staff and infrastructure required to collect, process, store, and mediate access to this immense trove of data.
Commenter (11192) states that the EPA fails to discuss the costs associated with building, maintaining, and providing access to that repository, nor the cost of reviewing and evaluating applications for access. Commenter (11487) states that the fact that RDC charges researchers $3,000 to access a single year's data hints at the substantial costs associated with the enterprise. The commenter states that the EPA also has not explained who would be in charge of making those determinations -- a crucial issue given that communities disproportionately harmed by pollution might not trust the EPA to make them. Commenter (12435) states that the RDC is not affordably accessible and, from a financial standpoint, the construction of such a system could be very involved and should require an assessment of cost for review along with the proposed rule.
Commenter (11479) states that, even for the subset of EPA-relevant studies for which disclosure of data in a tiered-access system is feasible, such a system would be costly for the Agency to implement and maintain and impractical, if not impossible, for individual researchers to do so. The commenter states that, in turn, a system would likely be costly for those seeking to access the data. The commenter states that the CDC's RDC, for example, costs researchers thousands of dollars to access. The commenter states that it is unlikely that stakeholders such as the public and community groups would pay such high costs to access data that has already been vetted through peer-reviewed studies. 
Commenter (11900) states that, in the case of the RDC, the data stored belongs to NCHS (or in some cases to other operating divisions of the Department of Health and Human Services [DHHS]). The commenter states that, however, the system employed by EPA will house extramural data produced by academic researchers in addition to data from their own scientists. The commenter states that the scope of data to be stored in the RDC -- essentially all epidemiologic research on environmental exposure -- is potentially massive and the EPA must explain how it will fund this system. The commenter states that the EPA must also define the burden to researchers of preparing data for entry into the repository, if such preparation is required. 
Commenter (11900) states that, to ensure equitable use of the RDC, the EPA should not charge researchers to deposit or store their data in the RDC. The commenter is concerned that the costs of either depositing extramural data into an RDC or accessing data stored in an RDC will disproportionately impact certain types of research interests. The commenter states that, an alternative model, in which researchers are given unsupervised access to the raw restricted data, depends on the integrity of researchers and a system of penalties and enforcement for misuse; the commenter does not recommend this model. 
Commenter (11900) states that a key feature of the RDC process is that researchers do not have access to the restricted data except while on site at the RDC, nor are researchers allowed to bring any external technology or datasets into the RDC. The commenter states that all computer code for running analyses at the RDC is reviewed in advance, activity inside the RDC is monitored, and the analysis output is reviewed again before being released back to the researcher. The commenter states that this process ensures that researchers never have access to the restricted data except under close supervision, which precludes any possibility of a linkage re-identification attack. The commenter states that this close oversight over the data comes at the cost of operating the RDC. 
Commenter (12715) states that making the data underlying the studies used to generate all "influential scientific information" and "significant regulatory decisions" publicly available or ensuring that such data is publicly available in the future will require significant resources. The commenter states that the additional tiered access system the EPA now proposes is complicated and unclear and merely adds to the burden of evaluation and decision-making that the EPA must conduct in order to determine what data and studies to use when developing standards and health-based criteria. The commenter states that establishing tiered access to protect PII and other confidentiality protections involves creating multiple versions of a single dataset with varying levels of specificity and protection. The commenter states that, as agencies add more tiers, they must build in sufficient controls to monitor who is accessing the data and allow only for authorized access. The commenter states that the SNPRM does not specify how confidential health information covered by the HIPAA Privacy Rule can be protected outside of HIPAA-covered entities but, again, designing and implementing such protections would be expensive and time consuming. The commenter states that, although setting up and maintaining tiered access levels involves significant additional time and cost, the SNPRM is silent on the "logistics, processes, and funding for such data sharing." 
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule. The analysis by CBO noted by the commenter is not applicable to this rule because the analyses were not for this rule and the requirements analyzed are not the same as those in this rule.
Delay
Comment: Commenters (10038, 10853, 11890, 12731) contend that a tiered access program would delay Agency actions.
Commenters (10038, 10853) state that the rule will curtail the Agency's ability to respond to crises such as the current corona virus disease (COVID)-19 pandemic.
Commenters (11890, 12731) contend that the use of a tiered access program would further delay Agency assessments in critical programs that already struggle to meet their statutory or programmatic work plans. Commenter (12731) expresses concern that the EPA's SNPRM makes no mention of and gives no consideration to practical impediments to tiered access. Commenter (12731) refers to comments of Dr. Smith who highlighted that federal research centers allowing tiered access require researchers to visit in person and require a federal employee to approve all outputs taken from the research center; Center for Medicare and Medicaid Services (CMS) allows for remote use of data, but only after the researcher signs a contract with CMS that provides for the data to be maintained on a secure server -- a process that in Dr. Smith's experience "took some months to set up and (as I understand) a considerable sum of money." 
Response: Some commenters on the 2018 proposed rule and 2020 SNPRM specifically asked why the EPA would need to peer review health and scientific studies and scientific literature that had already undergone independent peer review. The EPA disagrees with commenters that the requirements of this rule will result in any meaningful delay in promulgating regulations. While this final rule requires the Agency to evaluate the availability of dose-response data for pivotal science, the incremental burden to the Agency to carry out these requirements is expected to be small given (1) the extensive scientific review the EPA already conducts regularly and (2) that the requirement is limited to pivotal science (i.e., typically a small, though highly important, subset of the studies the EPA would review).
General
Comment: Commenter (11890) states that a tiered access program would be costly in critical programs that already struggle to meet their statutory or programmatic work plans. 
Response: The EPA has identified some incremental costs that the Agency may incur as a result of this final rule. As stated in Section III.A.2, the EPA will continue its current practice of conducting extensive review of scientific studies during the development of significant regulatory actions and influential scientific information. The additional procedures required by this rule only apply to pivotal science, which is a subset of the total number of studies that the EPA would evaluate. Given the costs of the current robust process for identifying and reviewing scientific studies and documentation that are existing Agency practice, as well as that the determination of dose-response data availability is limited to pivotal science underlying significant regulatory actions and influential scientific information, the EPA anticipates that the incremental costs of this rule will be small. The Agency may also incur other administrative costs to perform analyses and evaluations to support activities such as exemption decisions made by the Administrator, and documenting these or other decisions made pursuant to the requirements of the final rule. Again, the Agency anticipates that the incremental costs for these activities will be small relative to current administrative costs for developing significant regulatory actions or influential scientific information.
7.1.3.7.2 Timing and Planning for Implementation
Comment: Commenters (10038, 11176) assert that it is premature to enact this rule given the EPA's limited involvement with RDCs and data enclaves and no public plans for further involvement.
Commenter (10038) states that, based on the current status of available data repositories and these anticipated costs, the explicit inclusion of tiered access to data and models that have CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects in the proposed rule is premature. 
Commenter (11176) asserts that the EPA should have an operations plan and an operational infrastructure for data sharing before implementing the requirements in the proposed rule as updated by the SNPRM. The commenter states that the operations plan should address financial support and logistics for placing research data into an enclave or RDC and carrying out the reanalysis and independent validation. 
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenters (11176, 12718) state that the EPA does not have an existing infrastructure in place to implement such tiering, therefore, it is premature to enact this proposed rule. 
Commenter (11176) states that the data sharing infrastructure not in place and the mechanisms for data sharing are in the early stages of development and lacking proof of concept. The commenter states that making federally-funded research available for which the EPA does not own the data is a work in progress as the SNPRM's wording states: "Development of standard data repositories is still ongoing." The commenter states that the SNPRM also notes the White House Office of Science and Technology Policy recent call for comments on a "draft set of characteristics of data repositories used to locate, manage, share, and use data resulting from federally-funded research." The commenter states that, for a snapshot of the status of data repositories for federally funded research, consider the NIH, the federal government's likely leader in federally funded data sharing, an effort started more than a decade ago. The commenter states that, in its frequently asked questions (FAQ) page for data sharing, the NIH responds, "maybe," to question #20 on whether researchers should consider contributing their data to a data archive. The commenter states that the reply equivocates because of the many factors involved but also indicates how far the NIH is from achieving full data sharing for the research it funds. The commenter states that, for the next question of where to find guidance on preparing data for sharing and archiving, the NIH FAQ does not provide its own guidance but, except for molecular biology information, refers readers of the guidance to an outside entity, the ICPSR. The commenter states that the EPA is a relative newcomer to encouraging data sharing and, while it will benefit from the advances and experiences of other federal entities, is still likely to be years away from providing or encouraging the data access assumed in this rule. The commenter states that, unlike the NSF and NIH, the EPA does not seem to require a data management plan of its grantees, which it planned to do by 2018 in the 2016 EPA document, Plan to Increase Access to Results of EPA-Funded Scientific Research. The commenter states that NIH is seeking to expand its policy on data management and sharing (84 FR 60398), with the goal of making results more transparent to the public. The commenter states that such a requirement might be a first step towards the eventual goal of EPA funded research data being more widely accessible. The commenter states that data management plans and sharing strategies remain work in progress.
Commenter (12718) states that the EPA does not have an existing infrastructure in place to implement such tiering and has made not such efforts to satisfy capabilities that would be needed for appropriate tiered access structures in practice. The commenter states that the proposed regulation offers no meaningful articulation of how the EPA would build this capacity or whether it would instead rely on existing infrastructure. The commenter states that, if the latter, the EPA does not address how such an approach would be operationalized given legal restrictions under title 13, title 42, and law for use of Federal Statistical Research Data Centers, including for managing the risks of re-identification that could violate confidentiality protections under federal law, even if EPA employees are the source of such violations. The commenter states that application and use of tiered access models within the federal government's data architecture and infrastructure cannot be haphazardly applied. The commenter states that, if the EPA were to adopt such an approach, the Agency must clearly articulate appropriate safeguards beyond simply stating the Agency is "conducting a pilot study." The commenter states that, even then, the peculiar requirements for nongovernment funded research to be included raise concern about the breadth, applicability, and utility of such an approach as proposed.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information. 
Comment: Commenter (11176) notes that the OMB has yet to release its expected guidance for tiered access. The commenter suggests the EPA should also consider the September 2019 workshop on tiered access hosted by the Council of Professional Associations on Federal Statistics, where the workshop's recommendations for OMB include encouragement of experimentation as well as further research and development.
Response: Thank you for the recommendation.
Comment: Commenter (12431) shares the concern, expressed by the Society of Toxicology in its comments on the SNPRM, that due to the high costs of establishing and operating data repositories with the capacity for anonymizing protected data sets, the inclusion of tiered access to restricted data and models is premature. 
Commenters (10038, 12431) suggest the rule should exempt such data and models until the described data repositories have been established and are fully capable of providing tiered access to data. 
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
7.1.3.7.3 Tiering Plan
General
Comment: Commenters (10339, 10974, 11362, 11392, 11393, 11184, 11407, 11499, 12412, 12474, 12710, 13549, 13788, 14397) contend the proposal and SNPRM are unclear on several aspects of how the rule would be implemented, as described below. 
Commenters (11393, 11184, 11407, 12412, 12474, 13549) state that saying a tiered approach to accessing data is possible, without providing details, is insufficient and unacceptable. 
Commenter (10339) states that the practical details about the implementation of the rule remain unresolved, including many issues raised by the EPA's SAB draft report on the scientific and technical basis for the rule. The commenter states that it is not clear precisely if/how the EPA will pursue tiered access to data or make a final determination on whether the level of access is sufficient to be compliant with the rule. 
Commenter (10974) states that the EPA does not sufficiently describe how the proposed rule would actively and effectively preserve individuals' privacy or CBI. The commenter states that this proposed rule weakens longstanding privacy protections afforded to U.S. citizens.
Commenter (11362) questions how the EPA will implement this system and determine how, when, and with whom it will share private information. Commenter (12710) states that criteria for tiered access have yet to be developed, and the EPA makes no attempt to define tiered access in the proposed regulatory language.
Commenters (11392, 11499, 13788, 14397) assert that the rule is vague in several respects. Commenter (11392) states that the SNPRM's description of tiered access as "creating multiple versions of a single dataset with varying levels of specificity and protection" is vague and impracticable. Commenter (11499) is concerned that the overall language on tiered access is vague and that clarification on who would have access to CBI and PII would be helpful, accompanied by clarification on how the tiered access would work. Commenter (13788) states that the EPA's discussion of tiered access is vague and neither the rule nor the preamble provides sufficient detail on how such a system would be actually be implemented. Commenter (14397) states that the EPA's proposed tiered approach remains vague and unworkable.
Commenter (11391) states that even where a tiered or controlled-access approach is theoretically possible, there may be valid practical reasons why such an approach cannot be adopted. The commenter states that there is an ongoing debate among scientists about the utility of tiered access models for human health data and how they can and should work. The commenter states that some researchers and organizations have indeed begun to use this kind of approach. The commenter states that there is still considerable resistance to tiered access in the scientific community and it has been far from universally adopted.
Commenter (9455) expresses concern that this proposed rule threatens the trust of the public in environmental health research. The commenter states that the ebbing of that trust has roots in reality and is not just a perception problem. The commenter states that medical data posted on a public website are open to sophisticated data and AI programs that could identify the individuals. The commenter states that medical literature includes large numbers of research papers on patients' need for medical confidentiality. The commenter states that medical confidentiality in research also has been found necessary, especially if the research involves parents giving consent for inclusion of children. The commenter states that even if no laws and ethical standards were in place, environmental health research would be throttled if even a rumor of confidentiality breach should arise. 
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Administrative Capacity
Comment: Commenters (9455, 10974, 12731) contend that the EPA has not demonstrated that the Agency has the capacity to administer a system of tiered access. Commenter (10974) states that the EPA has not demonstrated sufficient programming resources to comply with the legal requirements afforded to individuals' privacy or CBI in the tiered approach outlined in the SNPRM.
Commenter (9455) asserts that the effort to de-identify or access-tier records of volunteers and of certificates is beyond EPA's budget and staffing. The commenter states that it cannot be left to an automated program with only spot checks for QA/QC. The commenter states that the number of individuals whose data underpin environmental standards may run into the millions. The commenter states that the number of volunteers whose medical data underpin existing standards may be as large as one half to one million. The commenter states that they are not the sole source of confidential medical data. The commenter states that many studies involve birth certificates and death certificates in the thousands for each study. The commenter states that these are privacy-protected documents. The commenter states that, even if the re-identification problem were solvable, the logistics of thorough de-identification or detailed tiering of the data would stymie the intent to post medical data.
Commenter (12731) asserts that the EPA has not demonstrated that the Agency has the capacity to administer a system of tiered access to the myriad studies that support all of its regulatory decisions and influential scientific information. The commenter states that, as for CBI, the EPA points to its existing regulations on disclosure of CBI as a potential framework that would govern tiered access to such information. The commenter states that those regulations, however, provide a detailed process for applying the rules and adjudicating disputes as to confidentiality.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Arbitrary System
Comment: Commenters (11396, 11576, 12430, 12731, 13788) contend the rule's proposed tiering system is arbitrary in certain respects.
Commenter (11396) states that the Options in the SNPRM are vague and arbitrary. Commenter (12430) states that, rather than making the EPA's proposals consistent with data privacy law, the SNPRM's proposed tiering provision underscores the arbitrariness, unworkability, and unlawfulness of the proposal. 
Commenter (11576) states that the proposed tiered-access approach is arbitrary and unexplained, as the EPA has failed to provide any analysis to justify the approach. The commenter adds that the EPA has not adequately responded to concerns previously raised in comments to substantiate how its tiered approach would remedy existing concerns. 
Commenter (12731) asserts that the EPA's preferred approach, which "would allow Agency consideration of studies where there is tiered access to data and models," is arbitrarily vague and notional. The commenter states that nowhere does the Agency specify the types of data and models it envisions would be encompassed by each tier, or even how many tiers might exist. The commenter states that the EPA fails to provide sufficient description or necessary analysis of the Agency's envisioned tiered access approach.
Commenter (12731) asserts that the EPA has arbitrarily offered no explanation of how and to what extent legal restrictions on the sharing of confidential information would impede its preferred "tiered access" approach.
Commenter (12731) asserts that the EPA failed to consider the likelihood that tiered access would not be practical or ethical if study participants did not consent to their personal information being examined by outside parties, even with restrictions. The commenter states that it would be arbitrary for EPA to ignore this important aspect of the problem, especially in disregarding the comments of its own science advisers on these issues. The commenter states that several members of the SAB expressed concerns along these lines. The commenter states that the SAB expressed a similar concern in its majority report on the proposed rule:
      It may not be feasible to identify and make available data in epidemiological studies that arise from small datasets or targeted geographic areas, especially if the Informed Consent Form indicated that only the particular researchers who conducted the study would have access to the information and data. If the participants agreed to grant only a select group of researchers access to their personal information, then that consideration should be respected, and such information should not be supplied to additional people for validation. It would probably be impractical, to go back to the participants and request their approval to provide additional people access to personal information.
Commenter (12731) states that the vagueness of the tiered access approach, if finalized, would render it arbitrary for another reason as well: researchers whose work is relevant to the regulatory process would no longer be able to pursue studies to ensure that the EPA could consider their findings. The commenter states that, by replacing its prior policy of considering peer reviewed studies with a prerequisite of indeterminate "tiered access" to data and models, the Agency has disregarded these stakeholders' reliance interests in a predictable system for accepting and considering scientific information. The commenter states that the EPA has failed to provide the more detailed justification necessary for altering its policy in a way that would inject uncertainty into policy-oriented scientific inquiry.
Commenter (13788) asserts that the EPA's failure to offer a reasoned explanation about how its proposed tiered access system would be implemented under this rule renders it unsupported and arbitrary. The commenter states that it is not enough for EPA in the SNPRM to introduce and vaguely discuss "tiered access." 
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Establishment of the Data Repository
Comment: Commenters (10974, 11487, 11496, 11576, 11894, 12710, 12720, 12731, 13788, 11900, 12464) express concern that the EPA has not yet established a data repository nor established who or what entity will create and manage the tiered access approach. 
Commenter (10974) states that, while the EPA described a model of private data treatment by the RDC, NCHS, CDC, there is no suggestion or reference in the SNPRM for how the EPA would establish this level of data stewardship, administration, and curation. Commenter (11900) asks how will researchers access data stored in a restricted-access enclave? 
Commenter (11487) states that the EPA's proposal forces an untenable choice on researchers: know that their study will be inappropriately down weighted, or accept the problems that can accompany submission of their research into a repository that has not been satisfactorily described. The commenter states that, when environmental health researchers inform potential study participants that their data will be placed into such a repository, they will likely find it harder to recruit. The commenter states that the problem will be particularly acute in historically marginalized and disproportionately polluted communities whose involvement is essential.
Commenter (11487) states that the EPA's invocation of the NCHS's RDC acknowledges the necessity of establishing a way to provide carefully controlled access to identifiable data, but it fails to demonstrate that it has fully considered how such a model might be used for the many studies the Agency ought to be considering. The commenter states that the SNPRM language merely states that the RDC is a model, and that the EPA is conducting a pilot study on how RDC might host EPA datasets. The commenter states that this falls far short of the kind of detail necessary before finalizing a rule. The commenter states that, because the RDC contains data collected by CDC and other federal agencies, not by independent academic researchers, a great deal of additional work and infrastructure would be necessary before it could host the full range of data that should be informing EPA work.
Commenter (11576) notes that the EPA suggests that the RDC is a viable model for its tiered-access protocol. The commenter states that, as the protocol has yet to be fully developed, however, members of the public have no way of providing sufficient review or comment on the possible model. The commenter states that, at the time of publication of the SNPRM, there was no indication as to how long this evaluation will take and if it will be a successful model for implementation. The commenter states the EPA still has not provided a sufficient explanation as to how the Agency will consider all information available within a published study. 
Commenter (11894) contends that the new proposal is opaque on the role the EPA intends to play in implementing a rule requiring studies and models to have public or restricted access to all data or code. The commenter states that the SAB conducted a consultation on mechanisms for applying a tiered access approach for securing PII and confidential information in August 2019. The commenter states that no consensus was attempted but several SAB members were skeptical because the EPA had not provided any specifics of how to adapt such an approach for the rule. 
Commenter (12720) states that the EPA highlights the RDC program as a model but does not explain how such an approach could be workable in the EPA context, nor does it explain the criteria that must be spelled out in a research proposal as a prerequisite for obtaining access to the data. The commenter states that the EPA apparently is planning to rely on CDC to fulfill major data management responsibilities made necessary in the SNPRM, but has not made that clear within the text of the SNPRM.
Commenters (12731, 12464) express concern that the EPA is entirely silent on who or what entity will create and manage the tiered access approach. Commenter (12464) states that, if the EPA intends to operate its own data enclave to administer the tiered access alternative, it should state that intention unambiguously, identify the source of its legal authority to do so, and address how it will pay for it. The commenter states that the SNPRM suggests that the EPA intends to address such issues in "implementation guidance," (85 FR 15403), but they are far too fundamental to be left to guidance documents.
Commenter (13788) notes that the SNPRM references models in the RDC and CDC, mentioning that the EPA is currently conducting a pilot study using the RDC's secure data enclave, but then the Agency also asserts that "[d]evelopment of standard data repositories is still ongoing." The commenter states that, thus, even the EPA acknowledges that it is far from clear how this would actually work.
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. 
Comment: Commenter (12710) asks if confidentiality agreements concerning PII in those studies can be overruled by whatever tiered access plan the EPA devises for such studies and, if so, at what risk to individuals and researchers?
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (12719) asserts that the EPA has not tested the proposed methods to ensure that private data would indeed remain secure. The commenter states that, without testing their tiered approach, the EPA cannot be sure that it has protected individuals' privacy, which raises serious ethical and legal concerns about data privacy. 
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Detailed Descriptions of Tiering System 
Comment: Several commenters (11391, 11392, 11396, 11399, 11894, 12464) contend the proposed revisions should include a more detailed description for implementation of the tiered system as described below. Commenters suggest the following implementation questions need to be addressed:
 How "tiered access" will be achieved? (12464)
 Who will determine which data and models count as "restricted"? (12464).
 To whom would the EPA release the otherwise restricted data and under what circumstances? (11396)
 Where data will be stored? (12464)
 Who will determine who has access to the stored data? (11396, 12464)
 Who will pay for the operation of such a system? (12464)
 How would the tiered access process be administered and funded? (11396)
 Where will such a system be housed within the Agency? (11392)
 The EPA is not a data management agency  --  does it have the capability to manage a large amount of sensitive data? (11392)
 Does the EPA have a robust cybersecurity strategy? (11392)
 Does the EPA have staff who are capable of running this system? (11392)
 How many people will the EPA need to hire? (11392)
 How will members of the public apply to access the sensitive data? (11392)
 Would the recipient(s) of the restricted information be a subset of the public with demonstrated scientific expertise? (11396)
 How will it evaluate the adequacy and integrity of any tiered access review that follows? (11396)
 Who at the EPA will determine external access to the data, and at what tier? (11392)
 Who at the EPA will determine what parts of the datasets belong in each tier of the system? (11392)
 How will this address restrictions put on PII protected by the Health Insurance Portability and Accountability Act, the NIH, and IRBs, which contain no applicable "tiered access" exemptions from privacy requirements? (11392)
 What framework or level of "tiered access" would be deemed acceptable to EPA? (11399)
 What is the difference between tiered and restricted access? Option 2 does not use the term "tiered access," but nonetheless discusses (85 FR 15405) giving intermediate consideration to studies based upon data with "restricted access"? (11894, 12715)
 What standards will be used to determine what counts as "restricted data and models" subject to tiered access? (12464)
 Does the EPA mean to exclude restricted access in the first version and that tiered access is another term for publicly available? (11894)
 Is the EPA is proposing to consider studies where participants and researchers have agreed from the outset to a tiered access approach, or whether the Agency is proposing to itself provide tiered access to certain confidential data, even where researchers and study participants have not agreed to such sharing at the outset? (11391)
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenters (9455, 12457, 12731) contend that the EPA fails to provide sufficient description or necessary analysis of the Agency's envisioned tiered access approach. The commenters suggest that the following information is necessary:
 Discussion of the process of allowing access to its tiered program other than noting that the highest tiers would be accessed only by certified" researchers (9455).
 Description of "certified researchers" (9455).
 Description of how a demander of access would be judged to merit access (9455).
 Identification of the access fees, a potential barrier to the public but not to regulated industry (9455).
 Protocols for this tiered sharing approach (12457).
 Descriptions of who will be given access to the different data tiers (12457).
 Operational details associated with sharing data (12457).
 What data platforms will be used to publicly share data that will enable publicly shared tiered and non-tiered data (12457).
 What funding sources will be made available to cover the additional time and cost of data sharing (12457). 
 Details relating to the determination of tiers (12731).
 Procedures for granting access to various tiers of data; operation, curation, management and oversight of data repositories; data repository ownership (12731).
 Whether, and if so, how repositories involving PII and CBI could be legal (12731).
 Safeguards to prevent hacking of web-based repositories (12731).
 Liability implications for researchers handling PII or CBI (12731).
 Considerations relating to the extent to which study authors would participate generally (12731).
 Costs (12731).
 How the Agency would ensure compliance with such statutory requirements or evaluate the extent of the restrictions they might place on its tiered access approach (12731).
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (11576) notes that the SNPRM states that "[u]nder a tiered approach to accessing data and models that include CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects, access is more restricted for more sensitive data and models. Thus, the amount of information available for analysis is dictated by the tier. The greatest amount of information is made available at the most restricted access tier." The commenter states that this fails to describe what criteria must be met or what grounds any member of the public -- including researchers or advocates -- must have in order to access potentially sensitive information.
Commenter (11890) states that the alleged transparency sought by this SNPRM cannot be obtained for many public health studies if the EPA honors existing legal and ethical requirements for protecting participants' privacy. The commenter states that these study results would need to be restricted (tiered access) to researchers with legal data-sharing agreements and safeguards regarding protection of privacy. 
Commenter (12720) contends that, even if access is restricted, the EPA needs to ensure that parties accessing the data (whether EPA staff, EPA consultants, or the public) are legally bound to protect the information, such as through a non-disclosure agreement, data use agreement or the like.
Commenter (12731) express concern that the proposed regulation should clearly address the issue of obtaining public access to datasets while maintaining the privacy of the participants and confidentiality of the data, because without such access, sensitive data and CBI could be excluded entirely from consideration as pivotal regulatory science."
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (11900) notes that the SNPRM described the restricted access tier as "limited to authorized researchers and not possible for the general public." The commenter states that, however, the EPA does not define a process for vetting applicants or criteria for granting access. The commenter states that restricted access must be limited to researchers with a legitimate scientific interest in the data, otherwise restricted access is simply `accessible to the public by application.' The commenter states that the RDC, identified as a possible model by the SNPRM, requires researchers "submit a research proposal outlining the need for restricted-use data." The commenter states that a stated primary goal of the SNPRM in making data available is to support reanalysis of study data. The commenter states that it is unclear on what basis applications seeking to perform reanalysis would be judged, because reanalysis is not typically considered research. The commenter states that the EPA must clearly define how applications to access restricted-access data will be evaluated.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (11900) states that an established practice for evaluating applications for controlled data is to have a Data Access Committee, rather than leave decision-making in the hands of a single individual. The commenter states that, for example, large intramural research studies conducted by the National Institute of Environmental Health Sciences (NIEHS), including the Sister Study and Agricultural Health Study, have committees to review data requests. The commenter states that advises the EPA to establish a Data Access Committee that includes equal representation by academic scientists, industry scientists, public-interest researchers, and community members who are nonscientists. The commenter states that, similar to requirements for IRB membership (21 CFR section 56.107), the committee should have diversity of membership (including race, gender, and cultural background), and members must not have any conflict of interest with the applications they are evaluating.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Implementation of the final rule will consider existing best practices for data governance. 
Comment: Commenter (12699) suggests the any limiting of the release of underlying information, if required, should be narrowly tailored to cover only the specific information that is unable to be disclosed. The commenter states that through a tiered approach, researchers signing non-disclosure requirements should be able to gain access to the restricted information. The commenter states that one exception would be if there was a genuine concern for access to be provided even to researchers signing non-disclosure agreements or who have been subject to other privacy and related requirements.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Opportunity to Comment on Tiered System
Comment: Commenters (11393, 11184, 11407, 11576, 12403, 12412, 12474, 12731, 13549, 12464) assert that, in the SNPRM. the EPA has failed to provide the requisite public notice. 
Commenters (11393, 11184, 11407, 11576, 12412, 12474, 13549) state that this vague aspiration -- saying a tiered approach to accessing data is possible, without details -- is insufficient and before the EPA can finalize the rule, it must lay out and analyze those approaches for public review and comment. Commenter (12731) asserts that, without any specific proposals as to how it would implement its preferred tiered access approach, the EPA has failed to provide the requisite notice. 
Commenter (11576) asserts that the tiered-access approach introduced in the SNPRM problematically and arbitrarily relies on an approach to be developed in the future without providing commenters with the opportunity to comment on the exact tiered-access methodology it will use.
Commenter (12464) suggests that the EPA has not fully thought through the implications and legal challenges to a tiered access system. The commenter states that the public does not have enough information on such a program and therefore the EPA has not given the public a meaningful opportunity to comment on the proposal. The commenter states that it is not clear how the tiered access alternative would be implemented or who would be responsible for implementing it. The commenter states that the tiered-access alternative in the SNPRM contains multiple gaps and ambiguities.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. Implementation of the final rule will consider existing best practices for data governance.
Comment: Commenters (11392, 12720, 13788, 14397) assert that it is inappropriate for EPA to use the public comment period to illuminate issues that the Agency itself should have clarified within the original proposal and SNPRM.
Commenter (11392) asserts that the Agency's solicitation for public comments on this matter is insufficient given the paucity of details on this proposed solution to the publication limitations of sensitive data. 
Commenter (12720) states that it is clear that the EPA has not adequately planned for the major complications to science activities imposed by the SNPRM, because the EPA is seeking the public to resolve fundamental implementation issues through the public comment period. The commenter states that, given these clear deficiencies, the SNPRM should be withdrawn. 
Commenter (13788) states that the EPA's assertion that "[i]nformation received during this public comment period will, among other things, help inform improved guidance and best practices related to preserving and providing access to data," does not allow commenters access to sufficient information about the course the Agency's final rule will take to permit meaningful comment. The commenter states that the whole point of an SNPRM is to clear things up and to allow enough release of information to permit the public to evaluate fully what the Agency is proposing. The commenter states that the EPA has failed in that regard concerning its proposed tiered access approach.
Commenter (14397) states that, not only is the EPA is proposing a major change to the way science is evaluated with regard to public health, but now the EPA doesn't even have a clear idea of how this policy should be implemented. The commenter states that this proposed approach would incorporate suggestions from commenters on policy implementation without giving the public a chance to review and comment on them. The commenter states that this approach ignores the nation's system of public input and amounts to no more than wishful thinking that an acceptable approach will present itself. The commenter states that this tiered approach proposal would rank studies in a way that is, ironically, lacking in transparency. 
Response: Implementation of the final rule will consider existing best practices for data governance.
Privacy Concerns 
Comment: Commenters (11192, 11361) state the proposal does not describe how tiered access would effectively mitigate privacy concerns, especially as it provides no description regarding those who could gain access to different tiers of data. 
Commenter (12731) states that the EPA fails to assess the legal and practical barriers to implementing tiered access.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. Implementation of the final rule will consider existing best practices for data governance. 
Comment: Commenters (9455, 12715) express concern that the tiered process to gain access to CBI, proprietary information, or PII may lead to cybersecurity concerns. 
Commenter (9455) states that, even if privacy protection could be accomplished satisfactorily, the EPA cannot guarantee that hackers could not enter the website and alter the data. The commenter states that protecting databases from intrusion is known to be an often-lost arms race. The commenter states that frequent news items about hacking of databases kept by large corporations and governmental agencies demonstrate this cybernetic problem. The commenter states that many parties have motivation to alter the data being proposed for EPA's public posting. The commenter states that the EPA proposes to post immense databases, analyses and models for all new standards and regulations spanning every aspect of environment and environmental health. The commenter states that the potential for massive disruption looms large. The commenter states that, while the EPA is testing tiered access by using the RDC program, most data in the RDC have nothing to do directly with industrially or commercially important standards and do not invite hacking except possibly by pranksters and ransomware blackmailers. The commenter states that they contrast with the data to be posted by EPA which would underlie standard-setting. The commenter states that the EPA data would be a prime target for funded hackers to impose re-identification of medical data and to falsify the database by making changes designed to disprove effects of pollution. 
Commenter (12715) states that this presents yet another reason that data are not made publicly available, which the SNPR does not address. The commenter states that, accordingly, cybersecurity issues need to be addressed to protect the identity of all participants in a study. The commenter notes that the National Academy of Sciences states that the identification of personal information is a cybersecurity concern: 
      In addition to cybersecurity concerns, computer scientists and cryptographers have demonstrated that statistical analyses of data sets that generate highly precise results -- such as geographic specificity or other characteristics that identify respondents -- may result in privacy breaches. This presents a new challenge that federal statistical agencies are just beginning to address. 
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. Implementation of the final rule will consider existing best practices for data governance and protection.
Comment: Commenter (11492) asserts the tiered-access approach, by which data that cannot be made public can be shared with a few for independent validation, is unworkable because it is illegal to share personal health data with anyone, so sharing it with one or a handful of people is just as illegal as sharing it with the general public. 
Commenter (11399) states that it is questionable whether a tiered approach is feasible or could be managed to the extent that confidentiality could be maintained.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. Implementation of the final rule will consider existing best practices for data governance and protection. 
Comment: Commenters (11192, 11361) contend that there are significant barriers to producing multiple datasets for different tiers of access, but perhaps the most obvious vulnerability in the proposal of a tiered access is that the EPA does not provide any criteria or suggested process regarding who would or could receive access to which tiers of data. Commenter (11361) states that, while the EPA mentions the potential creation of multiple versions of datasets that include privacy-related data transformations, the Agency fails to acknowledge that any data that needed to be transformed for any tier would need to be transformed for all tiers.
Commenter (11576) notes that the SNPRM states that "risk reduction techniques include creating multiple versions of a single dataset with varying levels of specificity and protection[,]" and that "[t]he benefit of tiered access is that data users who wish to conduct activities with a statistical purpose without first obtaining special authorization have access to the versions of the data in the least restricted tiers, allowing them to conduct research while protecting confidentiality." The commenter states that, while the notice is unreasonably short on specifics, this suggests how the tiered-access approach might operate, with tiers that are less or more restricted than others. The commenter states that the EPA does not describe which users would be granted access to the vague "higher restricted tiers" and, importantly, what type of information would be made available at each tier. The commenter states that there is not any indication of who would be responsible for reviewing access requests and otherwise managing the large databases the revised proposal would require. The commenter states that, based on the Agency's recent statements to the House Committee on Science, Space, and Technology, it appears the EPA intends for this burden to be shouldered by other agencies and private institutions.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. Implementation of the final rule will consider existing best practices for data governance and protection.
Recommendations
Comment: Commenter (9587) suggests the EPA be required to provide clear and explicit reasons why a proposed replication study should not be permitted.
Response: In this final rule, rather than considering all studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact as "pivotal science," the EPA is balancing transparency and feasibility by focusing on those studies that inform the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect. Thus, "pivotal science" includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). These are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. In the context of this rule, studies informing the dose-response modeling (e.g., benchmark response selection), studies identifying data for toxicokinetic adjustments, or studies informing the selection of uncertainty factors do not drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. Future statute-specific rulemakings may interpret "pivotal science" more broadly.
Comment: Commenter (11401) recommends that the EPA, in conjunction with this rulemaking, review its program for ensuring rigorous oversight of CBI, proprietary data or PII program. The commenter states that, for those studies that are not either fully transparent or transparent with tiered access, the EPA should determine that they cannot be used solely as the basis for a regulatory action or when finalizing influential scientific information. The commenter states that such studies may be considered, but alone cannot form the basis of a regulatory action without the inclusion of a fully transparent or transparent with tiered access study. 
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. Implementation of the final rule will consider existing best practices for data governance and protection.
Comment: Commenter (12720) suggests the EPA consider developing tiers of public access that may provide a base level of information (e.g., a robust summary or final study report which is devoid of confidential protected information) for the general user or member of the public and then restrict the availability of additional information to a smaller group when access to confidential or copyrighted information is warranted. The commenter states that, in matters of public health, it is common to require disclosure to a trusted third party. The commenter states that, in the case of environmental epidemiological studies, micro-aggregation of data can be used to protect personal identity. 
Commenter (12720) notes that the SAB letter states that it may not be practical for EPA to make all studies or other regulatory science used in a final regulatory action available to the public. The commenter suggests the EPA consider producing a list of study data considered in an evaluation and then strive to provide data for critical studies driving regulatory limits. The commenter states that the SAB notes that:
      the requirement to make data available for public inspection will be more easily implemented for some datasets than others (e.g., GLP [Good Laboratory Practices] studies include individual sample data as part of standard reporting). The expectation that data and methods will be available for all endpoints may be unrealistic. EPA could consider designating scores for information availability with the highest tier assigned to studies that define methods (e.g., protocol) as well as raw (individual sample level) data. Additional scores could be driven by sharing data on other related endpoints in the pivotal study to facilitate data interpretation. Scores could subsequently be used to evaluate data utility, uncertainty, etc. Ultimately, there are some datasets where CBI and/or PII data will require more onerous steps to protect data confidentiality. Based on EPA responses to SAB questions, it appears that the Agency is seeking approaches to manage these data issues. 
Commenter (12720) notes that any such scoring method has not been detailed by EPA in the SNPRM and could be biased given the unclear definitions and lack of evaluation criteria. The commenter states that, while the EPA may be "seeking approaches" to manage data issues, those issues should have been resolved by EPA before issuing the SNPRM. The commenter states that any such approaches determined by EPA itself in preparation of a proposed rule must be set forth for robust public inspection and comment because they are central to the ultimate impact of the SNPRM within the Agency and on stakeholders outside of the federal government. The commenter states that the SAB comment identifies "data utility, uncertainty, etc." as potential components of a scoring tool, but this is a poor representation of the complexity and depth of information available in scientific studies that would require consideration.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. Implementation of the final rule will consider existing best practices for data governance and protection.
7.1.3.8 Researchers
7.1.3.8.1 Burden
Comment: Commenter (11377) states that the expansion of scope represented by the SNPRM increases the already substantial burden that this rule would place on researchers to ensure the privacy and confidentiality of research subjects. The commenter states that the rule fails to acknowledge the enormously burdensome and costly complexities involved in redacting individuals' information in a way that both protects individuals' privacy and ensures that the data is still useful for future analysis. The commenter states that the SNPRM would make this problem worse, as burdensome redaction would be required for influential information, rather than only regulatory decision-making information. 
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Comment: Commenter (11391) states that many of the studies the EPA has traditionally relied on in regulating environmental pollutants necessarily involve confidential human health data. The commenter states that researchers, and the institutions that employ them, have a legal and ethical obligation to protect the privacy of study participants. The commenter states that, if enacted, the rule -- particularly as expanded in the 2020 SNPRM -- would make it much more difficult for researchers to fulfill that obligation and credibly promise study participants that their patient information will remain confidential. The commenter states that this could have a serious chilling effect on the conduct of important public health research in the U.S. The commenter states that privacy concerns would almost certainly hamper such research. The commenter states that critical lines of scientific inquiry that would have been pursued may not be and the quality of data that is obtained may be poorer than it otherwise would have been. 
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Comment: Commenter (11392) states that, if the EPA expects researchers to implement this system, then the Agency is creating enormous new requirements for the research community. The commenter states that describing such requirements simply as "multiple versions of a single dataset with varying levels of specificity and protection" is excessively vague for such a monumental overhaul of how the Agency considers science in its rulemaking. The commenter states that, if the EPA intends for the scientific community to adopt  --  and potentially implement  --  these dataset and access requirements, it is not offering researchers a meaningful description of the parameters on which they can comment. The commenter asks the following questions:
 If the intent is for the researcher to determine the appropriate level of access, who would determine whether a researcher's determined access levels appropriately met the requirements of the rule? 
 Would researchers be expected to review the applications for reanalysis in perpetuity? 
 Has a cost analysis been done to assess the burden this rule would place on researchers and on the institutions hosting the data? 
 What happens if a research team has since disbanded? 
 Does the EPA expect researchers in other countries to work with the Agency to manage such a data access system so their studies can be used by the United States government? 
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (11479) states that, even for the subset of research studies that could be shared within a tiered-accessed system, implementation of such a system would be onerous, costly, and infeasible for several reasons. The commenter states that de-identification of data would be an immense undertaking and is not feasible in many cases and for many study designs, particularly because the EPA is concerned with environmental threats. The commenter states that, to understand environmental threats, researchers often must know the precise locations of parameters such as environmental exposure and health effect occurrences in a subject population. The commenter states that, in many studies, these locations are personally identifying. The commenter states that it is difficult and often impossible to release this data in a way that protects study subjects' identities, and this exercise would be required under the tiered-access approach proposed in the EPA's SNPRM. The commenter states that, while the rule does not make this clear, researchers would bear the burden of deciding and implementing the tiered-access system for their data, according to EPA's conversations with Congress. The commenter states that the rule may make researchers wary of participating in policy-relevant research. The commenter states that this potential effect is antithetical to the stated goals of the proposed rule.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenter (11495) states that, even before this new expansion of scope, it was already apparent that the Agency would be unable to handle the immense logistical and technical challenges this new rule would create. The commenter states that the EPA has previously not provided web-based or clearinghouse depots for the posting of data, assuring quality control, nor have they provided a system to ensure database integrity and maintenance. The commenter states that this has been the case even in more innocuous situations that don't involve personal privacy and proprietary issues. The commenter states that, even with this dramatic expansion in the scope of data sharing requirements, there has been no indication that the EPA is ready or willing to take the responsibility to develop such a clearinghouse website to house data and the associated information or metadata, including defining a structure with all protections and securities. The commenter states that such a data system is beyond the scope and complexity of any data system the EPA has previously developed for its internal use nor does it easily fit the existing models used by the Agency, such as the one used for its Air Quality System for monitoring data. The commenter states that this system is fully contracted, maintained and assessed outside the direct control of the Agency. The commenter states that, given that the EPA will be unable to handle all of these new responsibilities, it is clear that this SNPRM will instead impose tremendous burdens on researchers and scientists outside the Agency. The commenter states that scientists and researchers will have to take time-consuming steps to make the data underlying their studies publicly available in order to have their study considered by EPA.
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
7.1.3.8.2 Funding
Comment: Commenters (10748, 12418, 12731) express concern that there is no financial support for researchers to redact data, host data, and/or provide tiered access infrastructure to the databases.
Commenter (10748) states that there is no financial or technical support for researchers, which would reduce the capacity of researchers to produce unbiased and credible research. The commenter states that the Agency would also need financial support to host data and provide tiered access infrastructure to the databases. The commenter states that this too would be expensive and burdensome to the Agency. The commenter states that, with the current effective review process and with leading scientific publications already encouraging data transparency, it does not make fiscal sense for the Agency to implement this rule. The commenter states that the proposed rule would, without doubt, impede the development and utilization of new science. The commenter states that the cost of doing research both in investigator time and required funding to develop regulated processes for broad access to data would be substantial, yet the SNPRM does not define any financial and technical support for researchers in this regard. The commenter states that, without defined financial and technical support, the result would be a crippling, if not an elimination of the strong, crucial science that has been the backbone of existent, validated regulatory reviews per the CAA mandate and found in other environmental statutes. Instead researchers may have to turn to other funding sources making their work biased and less credible.
Commenter (12418) states that the EPA has not specified the process for de-identifying data, ensuring study participant privacy, and covering associated costs, if the proposed rule is indeed adopted. The commenter states that the EPA has not indicated whether the researchers or the Agency will bear the cost of providing de-identified data and models upon request, and/or creating the additional versions of a dataset to support tiered access. The commenter states that, if the responsibility falls to researchers, this hurdle will likely further limit which studies will become available for consideration given that costs to de-identify data and additional resources to comply with data and model sharing requirements are not typically requested in grant budgets. The commenter states that, if instead the EPA will cover the cost of deidentifying data and models to sufficiently comply with the proposed rule, the anticipated costs should be assessed along with other proposed merits and drawbacks. The commenter states that the costs of de-identifying data, appropriately managing those data, and ensuring that no PII is released for the foreseeable future may far outweigh any perceived benefits of this rule.
Commenter (12731) states that recent information has also underscored the EPA's failure to consider the costs to researchers of demanding that older data be made available for reanalysis (either to the public or through a tiered access mechanism), and the implications that cost would have on EPA's access to regulatory science. The commenter states that, for example, SAB member Robert W. Merritt commented in the SAB's September 2019 consultation that data from older studies may be located in obsolete storage media or data formats that are either impossible or very costly to convert to a shareable form. The commenter states that, because the grants funding older studies have already lapsed, there would likely be little to no funding available for datasets owned by academic and non-profit institutions to be shared -- meaning that much of this research would not be eligible for EPA's consideration, and that the EPA might be drawing from a biased pool of for-profit industry studies. The commenter states that, previously, the EPA had accepted some responsibility for making information publicly available in a "cost-effective" way. The commenter states that now, it has disclaimed this role and abandoned any pretense of contributing to the transparency and validity it supposedly seeks, opting simply to discard studies out of hand (85 FR 15402). 
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenter (12464) states that an industry representative who wants access to such data faces no costs or burdens. The commenter states that, instead, academic researchers and universities would assume the responsibility and the financial burden of managing the logistics of making the data and models publicly available. The commenter states that, even then, the EPA is under no obligation to consider or use the study to inform its regulatory or informational processes. The commenter states that, even research, such as the Six Cities Study, that has been subject to a comprehensive reanalysis by the Health Effects Institute, would apparently not be eligible for consideration under the SNPRM, because its data are not available for further reanalysis by other researchers.
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
7.1.3.8.3 Other 
Comment: Commenter (11495) states that the SNPRM imposes implicit obligations on parties outside the federal government, including on researchers and potential study participants. The commenter states that researchers are often obligated to protect the confidentiality of study participants, and the SNPRM would require researchers to either breach those confidentiality agreements or risk the loss of willing study participants. 
Response: The underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (11497) asserts that the requirements for the public availability of data will not only undermine the inclusion and use of previously conducted epidemiological studies in the pool of "influential scientific information" and the science underlying "significant regulatory decisions"; they will also damage the ability of researchers to conduct future studies. The commenter states that researchers' ability to recruit potential study participants will be severely undermined if participants fear that their personal information may be made available for public scrutiny. The commenter states that this is particularly true for communities of color that are both disproportionately exposed to hazardous chemicals and have historical reasons to distrust researchers; yet, it is the exposures to these communities, and the resulting health effects, that most need to be understood and addressed.
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. The underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (12435) contends the SNPRM could increase the burdens facing science and health researchers. The commenter states that the proposal disadvantages scientists and clinical researchers whom this proposed rule would burden with reporting requirements, and study subjects whose participation and data would be prevented from informing public policy. The commenter states that the exclusion of many studies from informing policy has a direct impact on the scientific community whose work will be devalued under this proposal. The commenter states that pressure for underlying data to be public will induce participation bias in studies; people will be less likely to participate if their privacy is not assured. The commenter states that there is immeasurable harm done by politicization of the scientific basis of Agency rulemaking that Congress unquestionably intended to prevent when it mandated the role of scientific advisory boards under the CAA.
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. The underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (12457) states that the proposed revisions should include a detailed analysis that examines how these proposed revisions would impact laboratories and study investigators.
Response: The EPA is finalizing a rule of internal agency procedures, including how the Agency will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information for independent validation. The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
7.1.3.9 Potential Bias with Tiered System
Comment: Commenter (9455) states that the proposed rule fails to consider the probability of distorting studies because of minority distrust, privacy concerns, and allowing industrial access. The commenter states that the small amount of social science that analyzed the broad informed consent form indicates that members of traditionally discriminated against minority groups and people concerned about privacy shy away from giving broad consent. The commenter states that knowledge that for-profit companies will have access to the data also results in lower broad consent. The commenter states that participants in studies to be used for EPA regulation under the proposed rule would have to be informed that the data would be posted and that regulated industries would have access to them before they are asked to sign the broad informed consent forms. 
Commenter (12435) asserts that pressure for underlying data to be public will induce participation bias in studies; people will be less likely to participate if their privacy is not assured. The commenter states that there is immeasurable harm done by politicization of the scientific basis of Agency rulemaking that Congress unquestionably intended to prevent when it mandated the role of scientific advisory boards under the CAA.
Commenters (11192, 11361) contend that there are significant barriers to producing multiple datasets for different tiers of access, but perhaps the most obvious vulnerability in the proposal of a tiered access is that the EPA does not provide any criteria or suggested process regarding who would or could receive access to which tiers of data. The commenters state that, as written, the EPA could grant access to industry personnel eager to poke holes in public health studies and decline access to academic researchers interested in evaluating industry studies.
Commenter (11192) states that the proposal does not describe how tiered access would effectively mitigate privacy concerns, especially as it provides no description or metrics regarding those who could gain access to different tiers of data. The commenter states that this omission introduces an extraordinary potential vulnerability to political bias. 
Response: The EPA is finalizing a rule of internal agency procedures, including how the Agency will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information for independent validation. The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (11479) states that it is unlikely that stakeholders such as the public and community groups would pay the necessary high costs to access data that has already been vetted through peer-reviewed studies. The commenter states that the exception is regulated industry, which is likely to be the only EPA stakeholder group interested and able to access data housed in a tiered-access system. The commenter states that the SNPRM makes clear that the rule's intent is to allow stakeholders the opportunity to "reanalyze the data and models and explore the sensitivity of the conclusions to alternative assumptions..." 
Response: This rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenter (12430) states that discouraging individuals or particular groups of people from participating in future studies would introduce bias into the studies, to the detriment of the public interest in protecting health and the environment. The commenter states that, to the extent that researchers require the public availability of data for future studies, this could have a chilling effect on people's willingness to participate. The commenter states that the possibility of identification, with the specter of denial of future employment or benefits based on disclosure of medical information, may make some population groups less likely to participate in future studies. The commenter states that this chilling effect has important implications for environmental justice, as it may become more difficult to conduct studies in communities of concern.
 Commenter (12702) asserts that the SNPRM allows arbitrary decisions to be made about what science can and can't be used. The commenter states that environmental laws direct the EPA to use the "best available science" in countless situations. The commenter states that the SNPRM creates multiple confusing pathways for considering long standing public health studies, including the Option of "tiered access" or giving preference to some studies over others. The commenter states that the EPA's purported goal of "transparency" means studies will be evaluated not on their quality but on non-scientific ill-defined metrics.
Commenter (12720) expresses concern that the proposal does not sufficiently explain how a tiered access system would avoid biased handling of data and models at EPA.
Response: The EPA is finalizing a rule of internal agency procedures, including how the Agency will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information for independent validation. The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (12731) states that, because the grants funding older studies have already lapsed, there would likely be little to no funding available for datasets owned by academic and non-profit institutions to be shared -- meaning that much of this research would not be eligible for EPA's consideration, and that the EPA might be drawing from a biased pool of for-profit industry studies. 
Response: This rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenter (12731) asserts that the EPA has completely disregarded the potential for pro-industry bias by considering studies for which interested parties have arranged for tiered access. The commenter states that the EPA's recent briefing to House Science, Space, and Technology Committee staff underscores how the tiered access approach would preference industry interests: researchers would be responsible for managing the logistics of making data and models publicly available and establishing levels of tiered access; a time- consuming and labor-intensive process for investigators who have already concluded their work and published their findings, possibly having expended the full amount of funding available, unless specially interested third parties bankroll the process. The commenter states that, once established, tiered access would also allow financially interested, repeat players such as regulated entities to test the findings of whatever studies they dislike. The commenter states that the EPA has thus proposed to establish a requirement that would favor industry interests by compelling disclosure of data and methods that regulated entities can more readily furnish and that only regulated entities will likely have the resources and motivation to scrutinize with any frequency. The commenter states that such favoritism in the Agency's rule would be arbitrary unless somehow grounded in its authorizing statutes. The commenter states that the EPA would need to assess, consider, and balance the substantial unfair advantage industry would receive against any theoretical, marginal increase in the reliability of scientific information that could result from a tiered access approach. 
Response: The EPA is finalizing a rule of internal agency procedures, including how the Agency will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information for independent validation. The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. This rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenter (12720) states that the claim that publicly available dose response data and models would allow for independent validation stands in direct contradiction to the legal privacy protections that apply to key data necessary for re-analysis. The commenter states that the proposed partial redaction of sensitive information poses a cascading set of problems, because the statistical models characteristic of epidemiologic investigations rely on the inclusion of potentially confounding variables (e.g., age, sex, home address, health status, diet and alcohol consumption, smoking history) in order to properly isolate the pollution-health relationship with precision. The commenter states that, to understand the dose-response connection, these studies analyze detailed health, demographic, spatial, and behavioral information from thousands of people. The commenter states that this information is extremely sensitive and collected at the individual level. The commenter states that, as such, our nation's health privacy laws and IRB protocols require researchers to keep the data secure and confidential to prevent misuse. The commenter states that, collectively, these data points help researchers understand and isolate the cause-effect relationship between exposure to air pollution and risks for various health problems. The commenter states that it would be extremely difficult if not impossible for anyone using partially-redacted data sets derived from epidemiologic cohort studies to "validate" the results of the original studies, because such investigators would not be working with complete data sets.
Commenter (12720) states that, as further demonstration, the 2009 Integrated Science Assessment for PM2.5 notes that "[a]ppropriate statistical adjustment for confounders requires identifying and measuring all reasonably expected confounders." The commenter states that, therefore, exclusion of some potentially sensitive confounding variables from an underlying dataset likely would lead a different team of investigators to a different result. The commenter states that causing this wrongheaded and indefensible outcome results from the core approach and conceit in the proposal, revealing it to be yet again, arbitrary and capricious and an abuse of EPA discretion. The commenter states that the quantitative findings of dose-response relationships would almost certainly differ, not as a result of any true difference in the quantitative exposure-effect relationship, but because the original work relied on complete data sets and the new analyses would not -- due to the proposal. The commenter states that the resulting discrepancies in quantitative findings could serve as motivation to call the original study results into question due to faulty and incomplete re-analyses.
Response: The EPA is finalizing a rule of internal agency procedures, including how the Agency will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information for independent validation. The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
7.1.4 40 CFR 30.5: Option 2
7.1.4.1 Support/Oppose Option 2
7.1.4.1.1 Support Option 2
Comment: Commenters (9362, 10973, 11358, 11388, 11395, 12395, 12397 12406, 12453, 12471) support Option 2.
Commenter (9362) asserts their preference for Option 2 of section 30.5 but notes the phrase "other things equal" should be deleted. The commenter questions who makes the determination that "other things are equal" and states that each study should stand on its own merits and those studies in the public domain should be given greater consideration. Additionally, the commenter approves of the statement "Furthermore, the Agency will identify those studies that are given greater consideration and provide a short description of why greater consideration was given," as long as all relevant scientific information has been considered and is documented in any EPA decision-making.
Commenter (10973) states that the EPA's alternative approach giving priority for publicly available studies is commendable. The commenter states that this is more of an "opt out" scenario where all appropriate scientific information is considered unless there is a specific reason to exclude it. The commenter states that, at the same time, this approach will continue to encourage open access to data and models. The commenter states that this approach is reasonable in that it furthers the goals of transparent rulemaking and open access to scientific data while recognizing that some relevant scientific information may not be fully publicly available. The commenter states that the EPA's attention to detail in this rulemaking is essential to assure the appropriate balance of transparency with inclusion of all relevant scientific information, while addressing necessary privacy and confidentiality concerns.
Commenter (11358) states that this alternative approach will give the Agency appropriate flexibility when implementing this new policy. The commenter states that the EPA should refrain from banning all studies from being utilized within the regulatory process solely due to data availability. The commenter states that, for data that is not produced by EPA, such as data used for studies, which have been published in scientific journals, the process of peer reviewing may often be sufficient. The commenter states that, as the EPA notes in the SNPRM, raw data may not be available for a variety of reasons including privacy, age of the data, etc. The commenter states that this proposed alternative will ensure that the EPA is in a position to use all available data to inform the Agency's decision-making process. 
Commenter (11395) states that, of these two alternatives, Option 2 is preferable because it does not automatically eliminate a wide range of well-conducted studies that cannot comport with the data availability specifications. 
Commenter (12406) states that Option 2 is consistent with their 2018 comments, specifically that scientifically relevant and reliable studies should not be eliminated from consideration, but rather, accorded less weight when integrating evidence from multiple studies and across different lines of evidence and that any guidance and other relevant documents developed to assist EPA staff to comply with this rule should include specific examples and/or case studies, perhaps drawing from recent EPA rulemakings, to demonstrate what constitutes regulatory science that is material to EPA's significant regulatory decisions. The commenter recommends EPA revise the regulatory text in Option 2 to omit the word "short" to characterize the narrative that EPA will provide to describe the studies that are provided greater consideration along with its explanation why the studies received greater consideration. The commenter asserts that the Agency should provide sufficient information and explanation as may be necessary in any given circumstance.
Commenter (12471) states that, while the most robust standards for science transparency are clearly desired, the tools to achieve this worthwhile objective may not yet be in place. The commenter states that agencies and research institutions may need time to develop the tools and protocols to allow greater public access at the appropriate tiered access level considering the need to protect privacy and other confidential information. The commenter states that the EPA should note in this rulemaking that it will be revisiting this issue as the appropriate tools are developed, with the objective of increasing transparency and public access as expeditiously as possible consistent with federal law.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
The transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency. The EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public.
Comment: Commenters (11922, 12471, 12612, 12699) support Option 2 but encourage limited use of data and models that cannot be independently validated.
Commenter (11922) supports the SNPRM's proposed modifications to 40 CFR 30.5 that at the very least give extremely strong preference for data and models that can be independently validated. The commenter supports the use of information that cannot be validated only in rare circumstances. The commenter states that studies and data that cannot be validated under Option 2 should be utilized only if adequate studies that can be validated are not available to develop the necessary underlying influential or pivotal information. The commenter states that, in the rare circumstance described above, when information that cannot be validated is used in the decision-making process, the data that cannot be validated should always be afforded less consideration, if any consideration at all, than data that can be validated.
Commenter (12471) recommends the EPA give preference to studies that are fully available to the public if they are of equal (generally commensurate with) or better quality than the non-available studies. The commenter states that the EPA could note that transparency, in and of itself, is a study criterion that enhances study quality by allowing greater independent review and reanalysis. The commenter states that, if the only studies available provide limited or no access to the underlying data or models, the EPA could still consider the data and models in the rulemaking if they are otherwise of sufficient quality, but note the weakness and additional uncertainties inherent in relying on studies that are not available at any protected tiered level.
Commenter (12612) states that, if data and models are not made public, then the study should not be ignored but should be considered secondary to other comparable studies where data and models are made public.
Commenter (12699) states that, when there are genuine privacy, confidentiality, and CBI concerns, the EPA should not exclude studies, but give lesser weight to the studies when underlying information is unavailable. The commenter states that, when the EPA provides a legal justification to not disclose underlying information, there should be clear documentation provided to the public in support of the legal justification. The commenter states that, when there are subjective privacy or confidentiality concerns unrelated to law, this can be tricky to ensure the authors or agency are stating a genuine concern. The commenter states that the EPA should view and treat such assertions with skepticism, and deem the concern to be genuine only when disclosure would violate widely accepted practices or norms. The commenter states that, if the concerns are deemed genuine, then the study should not be excluded,  however, it still should be provided less weight than comparable studies. The commenter states that it should be rare for the EPA to solely rely upon studies without underlying data, and the EPA should clarify in the rule that the Agency shall take all reasonable steps to avoid such an outcome. The commenter states that the EPA should certify in the administrative record of a rulemaking that it did a full literature review and no legitimate studies without such problems exist that could have been used in their place.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information.
7.1.4.1.2 Oppose Option 2
Comment: Commenter (9587) endorses the previous, unmodified transparency requirements, and states that transparency should take priority over privacy and confidentiality in EPA policymaking. The commenter states that the EPA's proposed modification gives claims of privacy and/or confidentiality priority over transparency requirements. The commenter states that this modification practically will impose substantial limits on the effect of the rule since privacy, confidentiality, and tiered access are all concepts and practices that inhibit full transparency. 
Commenter (12712) states that the SNPRM's "consideration" to studies based on data that is neither publicly available nor available through tiered access is too vague. The commenter states that no such study, by definition unavailable for reanalysis by independent researchers, should even be a factor in an EPA decision to regulate.
Response: The EPA is finalizing a rule of internal agency procedures, including how the Agency will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information for independent validation. The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenters (11182, 11362, 11390, 11393, 11396, 11407, 11489, 11576, 11890, 11895, 12404, 12412, 12442, 12463, 12474, 12715, 13549) oppose Option 2. 
Commenters (11362, 11890) oppose the weighting of publicly available data and down-valuing of confidential data. The commenters contend that the EPA must prioritize public health in its decision making, therefore data from public health and epidemiological studies must be taken into consideration for influential scientific information and regulatory actions. 
Commenter (11390) suggests that Option 2 is designed to create the appearance of flexibility but flouts the EPA's mission and settled scientific principles. Commenter (11890) states that giving less consideration to epidemiological studies conducted in the U.S. and other countries that protect private health data solely because those data aren't publicly available is unscientific. Commenter (12442) states that this rule shifts the Agency's approach by injecting a non-scientific metric that credits a study for disclosing private individual information used.
Commenter (11489) asserts that the value or merit of peer-reviewed science is not determined by the public availability of the underlying research data, and there is no scientific justification for a rule that directs the Agency to selectively weight certain studies over others based on this factor alone. Commenter (12463) does not believe that public availability and independent validation are qualities sufficient to mean research should be weighted more heavily than research where data must remain confidential.
Commenter (11396) states that the alternative to tiered access is equally arbitrary as no standards or criteria are provided to guide the Agency's discretion under this provision and to ensure that the best available science drives EPA's decision-making. The commenter questions how much less consideration would be given to research that relies on confidential patient information and who would make these decisions? The commenter contends that placing a higher value on certain scientific studies over others without clear criteria risks politicizing EPA's decision-making process.
Commenter (11576) asserts that EPA must use the best available science and it cannot exclude or down-weight a study that has undergone rigorous peer review that sufficiently analyzed the quality of the study. The commenter notes that in their joint statement regarding the proposed rule, the editors-in-chief of six highly regarded journals voiced disagreement with the EPA's Proposal by noting that "[d]iscounting evidence from the decision-making process on the basis that some data are confidential runs counter to the EPA stated mission `to reduce environmental risks ... based on the best available scientific information.'" Additionally, the commenter states that the Agency has failed to articulate a rational connection between the availability of underlying data and the quality, significance, and scientific soundness of a study, nor could it. Therefore, the commenter asserts that by reducing the weight given to studies on the basis of data availability, the EPA would wrongly downgrade the importance of study findings, which would ultimately affect the EPA's ability to address well-documented health harms.
Commenter (12404) states that while Option 2 would not totally eliminate a study from consideration for failure to share identifying information, it would downgrade its weight in a rulemaking or as "influential science." The commenter maintains that this could easily lead to a scientifically perverse result of putting greater weight on a methodologically inferior study just because the researchers shared personally identifying information from their subjects. According to the commenter the rule would incentivize not the best science, but that which most violates privacy. Additionally, the commenter provides privacy protections are in place to prevent a repetition of Nazi experiments and of cruel, racist research conducted on African American citizens of the United States. As such, the commenter asserts it is the regulatory process itself and the American people who will suffer by being denied regulations based on the best available science if violating human subject protections is made a condition of giving a study full weight in rulemaking or "influential science."
Commenter (12463) provides that the qualities of public availability and independent validation are not sufficient to mean research should be weighted more heavily than research where data must remain confidential.
Commenter (12715) states that Option 2 is flawed not only because it weighs the value of scientific studies using a non-scientific and largely irrelevant criterion -- the public availability of data for independent validation -- but also because it identifies no criteria for when or how EPA would exercise the vast discretion the Agency creates for itself. Quite the opposite, the commenter provides, the EPA has instead requested comment on "how much consideration should be given to studies where there is limited or no access to the underlying data and models." Thus, according to the commenter, Option 2 does nothing to ensure that important and valid scientific studies for which the data may be unavailable receive any consideration at all.
Response: The EPA is finalizing a rule of internal agency procedures, including how the Agency will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information for independent validation. The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
7.1.4.2 Definition of Greater Consideration 
Comment: Commenters (9587, 10748, 10339, 11397, 11176, 11492, 12720) express concern that, while the SNPRM states that "the Agency will give greater consideration to studies where the underlying data and models are publicly available in a manner sufficient for independent validation," "greater consideration" is not defined.
Commenter (10748) states that the level of difference in "consideration" is not defined. The commenter states that this could mean that a study that provides its data but is poorly designed would be held in higher consideration than a study that is well-designed but is unable to provide all its data. The commenter states that this is disconcerting and could lead to serious health consequences by implementing rulemaking based on poor science solely because higher-quality studies were unable to provide all their data.
Commenter (11176) notes that for the term, "greater consideration," the supplement does require "a short description of why greater consideration was given" when a study is given greater consideration. However, the commenter points out there is no (i) guidance for what "greater consideration" means when weighting the results of one study with that of another; and (ii) direction that EPA must explain the weighting given to various relevant studies. The commenter contends that such transparency here and in the previous paragraph is critical to rigorous scientific consideration of the relevant evidence.
Commenter (12720) states that the EPA has not defined what "greater consideration" means in different contexts.
Commenter (9587) suggests the EPA define greater consideration by definitions for all relevant policymaking subcategories.
Commenter (9587) suggests the EPA distinguish between publicly available research and research available only via tiered access, and give greater consideration to studies whose underlying data and models are publicly available than to studies whose underlying data and models are only available via tiered access. 
Commenter (11401) suggests that "give greater consideration to" should be clearly defined to mean whether the studies can or cannot be used primarily as the basis of a regulatory action, based on their level of transparency. 
Response: If researchers want to increase the likelihood that their studies receive greater consideration by the EPA, they may take steps to ensure that the underlying dose-response data are available to the greatest extent possible. But any such response to this final rule would be purely voluntary. The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
7.1.4.3 Other Things Equal
Comment: Commenter (10972) asks how the weighting of studies will be determined. Commenters (9362, 10972, 11484, 12715) question who is responsible for determining that "other things are equal" when deciding which studies should be given greater consideration. 
Commenter (11176) states that scientific reasoning would not support the following aspect of the SNPRM: "other things being equal, the Agency will give greater consideration to studies where the underlying data and models are available in a manner sufficient for independent validation either because they are publicly available or because they are available through tiered access when the data includes CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects." The commenter states that the EPA has not scientifically justified that greater consideration should be given to such studies. 
Commenter (11390) notes that the SNPRM suggests that "other things being equal," a "high-quality" study based on publicly available data would receive more weight than a study supported by non-publicly available data. The commenter states that that explanation leaves much undefined. The commenter states that, for example, the EPA does not explain what it would consider to be a "high-quality" study, or how it would determine whether "other things" are equal in any given circumstance. The commenter states that, in the likely event that the Agency is presented with multiple studies that could be considered "high-quality" but that are based on varying amounts of data and whose conclusions are supported with varying robustness, there are no criteria for the Agency to apply in determining which studies to consider, or how heavily to weigh them.
Commenter (11392) states that "other things being equal" is a nonsensical and arbitrary standard. Commenter (11894) states that the EPA's use of "other things equal" (85 FR 15405, section 30.5(a)) is unhelpful. 
Commenter (12699) suggests that, instead of "other things equal" when comparing studies, a better measure would be to studies that are "comparable," which to this author, means something slightly less than "other things equal." 
Response: The EPA clarified its intent and the process by which it would identify and determine the consideration to afford pivotal science in the final rule and preamble. The term "other things equal" is no longer used.
7.1.4.4 No Access or Access is Limited
The SNPRM states "Where there is no access to data and models, or access is limited, the Agency may still consider these studies, depending on the other attributes of the studies."
Comment: Commenter (9362) expresses concern that the EPA may still consider studies where there is no access to data and models, or access is limited. The commenter states that using "dark data" from third party sources does nothing to promote science and good policy-making decisions. The commenter states that the EPA should be promoting the acquisition of scientific evidence and this can only happen if the EPA uses publicly available information and data in their decision-making. The commenter states that, if third parties are unwilling to share their data, their data should not be used to make public policy decisions.
Commenter (11892) notes that the SNPRM requests comment on how much consideration should be given to studies when there is limited or no access to underlying data or models. The commenter suggests that the EPA accept approved theses/dissertations from accredited colleges/universities; published peer reviewed articles; other studies conducted according to federal/state/local QA/QC, as well as citizen science if data were collected with approved QA/QC. The commenter states that all these sources of information follow established quality review processes, which impacts credibility to results.
Commenter (11186) suggests that studies with limited or no access to underlying data should be given equal consideration as studies with publicly available data. 
Commenters (11399, 12456, 12715, 12715) express concern that the SNPRM does not include criteria regarding the EPA's consideration of "other attributes" and studies where there is no access or limited access to underlying data and models.
Commenter (11399) asks what "other attributes" will the EPA consider, and how much weight will it place on such attributes? The commenter states that, with no criteria to make these determinations, it would be extremely difficult to apply this provision uniformly or objectively. 
Commenters (12456, 12715) express concern that, in the alternative proposal, the EPA would have broad discretion to use, or exclude, potentially relevant, high-quality studies where there is no access or limited access to underlying data and models. 
Commenter (12715) asserts the proposal is flawed because it identifies no criteria for when or how the EPA would exercise the vast discretion the Agency creates for itself. The commenter states that the EPA has instead requested comment on "how much consideration should be given to studies where there is limited or no access to the underlying data and models." The commenter states that, thus, Option 2 does nothing to ensure that important and valid scientific studies for which the data may be unavailable receive any consideration at all.
Commenter (12731) states that, while the Agency suggests that it "may still consider studies where there is no access or limited access to underlying data and models" the Agency's consideration of relevant, valid studies is not optional. 
Response: Though not exhaustive or exclusive, the EPA has identified several factors that balance some of the important technical considerations the EPA will consider in addition to data availability and that are particularly relevant to the stage of the analysis where dose-response data are used. The EPA will thus consider the following factors:
 the quality of the study relative to other studies for which the dose-response data are available; 
 the extent to which there are other studies for which the dose-response data are available;
 the sensitivity of the conclusions in the significant regulatory action or influential scientific information based on the use of the study;
 the extent to which the study is fit for the purpose or intended use relative to other studies for which the dose-response data are available;
 the use of exposures or doses in a range and duration that is relevant for the intended use and that minimizes the need for extrapolations;
 the extent to which the study is supported by other scientific evidence;
 the extent to which the study accounts for unique scientific considerations;
 the extent to which the study minimizes the use of defaults and assumptions, uses appropriate and strong statistical methods, and includes a robust representation of uncertainty and confidence intervals; and
 the study's consideration of a broad range of parametric dose-response or concentration-response models, a robust set of potential confounding variables, nonparametric models that incorporate fewer assumptions, various threshold models across the dose or exposure range, and models that investigate factors that might account for spatial heterogeneity.
7.1.4.5 Short Description of Why Greater Consideration
Comment: Commenter (11358) supports the EPA's suggestion that the Agency would provide a short description of why greater consideration was given to a particular study or data set. The commenter states that this will help to ensure the greater transparency that this proposal aims to promote. 
Commenters (11392, 11922, 11401, 12720) contend that a "short description of why greater consideration was given" to some studies over others is not sufficient. 
Commenter (11922) suggests that the utilization of studies that cannot be validated under Option 2 should be accompanied with a comprehensive explanation detailing why the information was given consideration as pivotal or influential and its relative weight in the decision-making process.
Commenter (11392) states that the text of the SNPRM is vague when it comes to the proposal that the Agency will provide a "short description" of weighting decisions. The commenter asks the following questions:
 Who will write this description? 
 Will it be qualitative or quantitative? 
 Will it be based on a standardized rubric or case-by-case? 
 Where will the description be published? 
 Will there be an appeals process?
Commenter (12720) states that, given the potential for such a decision on a single study to affect its application across a range of different Agency activities, a "short description" is not nearly sufficient transparency from the Agency in describing such a significant decision with potentially far reaching and long lasting consequences. Rather than a short description, the commenter suggests the EPA address the following questions:
 What information could be included in such a short description?
 How would the Agency assess and confirm public availability or verify that data and models underlying pivotal science studies remained "publicly available" into the future? 
 When would such a description be drafted or finalized by the Agency within the context of its rulemaking responsibilities?
 Would such a description be included in a central science data clearinghouse and potentially revisited and revised over time, or would it necessarily be included in individual science products published by EPA and potentially remain static. 
 How would the EPA explain and document its deliberations and decision-making around the waiver process in a way that accounts for the complexity of any endeavor to make all "data and models" underlying key scientific information publicly available?
Response: Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Though not exhaustive or exclusive, the EPA has identified several factors that balance some of the important technical considerations the EPA will consider in addition to data availability and that are particularly relevant to the stage of the analysis where dose-response data are used. The EPA will thus consider the following factors:
 the quality of the study relative to other studies for which the dose-response data are available; 
 the extent to which there are other studies for which the dose-response data are available;
 the sensitivity of the conclusions in the significant regulatory action or influential scientific information based on the use of the study;
 the extent to which the study is fit for the purpose or intended use relative to other studies for which the dose-response data are available;
 the use of exposures or doses in a range and duration that is relevant for the intended use and that minimizes the need for extrapolations;
 the extent to which the study is supported by other scientific evidence;
 the extent to which the study accounts for unique scientific considerations;
 the extent to which the study minimizes the use of defaults and assumptions, uses appropriate and strong statistical methods, and includes a robust representation of uncertainty and confidence intervals; and
 the study's consideration of a broad range of parametric dose-response or concentration-response models, a robust set of potential confounding variables, nonparametric models that incorporate fewer assumptions, various threshold models across the dose or exposure range, and models that investigate factors that might account for spatial heterogeneity.
7.1.4.6 Arbitrary Decision Making
Comment: Commenter (10752) contends the proposal's inherent ambiguity makes it unacceptable as a legal regulation. The commenter states that the EPA proposes to arbitrarily pick and choose which data to rely upon.
Response: The EPA disagrees that the selection of pivotal science is arbitrary. Title 40 CFR 30.5 provides sufficient guidance for EPA to use in the selection of pivotal science.
Comment: Commenter (11176) notes that the supplemental proposal does not reference the well-known and valued scientific practice of meta-analysis, which EPA staff regularly uses, in its discussion of weighting evidence. According to the commenter, how to weigh the consideration of studies is at the heart of meta-analysis and its exclusion from the supplemental proposal reinforces the point that EPA has much work to do in laying the foundation for accomplishing the stated goal of the proposed rule. Additionally, the commenter asserts that "a short description of why greater consideration was given" is required whenever a study is given greater consideration.
Commenter (11349) asserts that the "greater consideration" criterion is arbitrary as it does not judge a study by standard scientific evaluation, such as the strength of its design or methods which are current acceptable metrics used to evaluate the merit of a scientific study.
Commenter (11391) states that, under Option 2, the EPA has not explained why certain studies are inherently unreliable, and thus should be given less weight. The commenter states that, to unfairly downgrade some valid and relevant studies without any apparent rationale or specified procedure is arbitrary and capricious in violation of the APA.
Commenter (11399) asserts that Option 2 would impose a wholly subjective framework by which EPA would determine the weight that should be afforded to scientific studies based on their transparency. The commenter asks what "other attributes" will the EPA consider, and how much weight will it place on such attributes? The commenter states that, with no criteria to make these determinations, it would be extremely difficult to apply this provision uniformly or objectively. The commenter states that this provision is an invitation for arbitrary decision-making. The commenter states that the EPA should not serve as an "arbiter" of scientific studies in this manner. 
Commenter (13788) states that, while the details of EPA's alternative approach are unclear, assigning preferential treatment to certain studies based on criteria that are unrelated to the studies' scientific merit and quality would be arbitrary and capricious. The commenter states that the SNPRM states that "[w]hen promulgating significant regulatory decisions or finalizing influential scientific information, the Agency will, other things equal, give greater consideration to studies where the underlying data and models are publicly available." The commenter states that, however, the Agency does not explain what it means by "other things equal" such that one study would be given greater weight, nor is the fact that data can be made publicly available tied in any way to the actual scientific merit or quality of the study results. The commenter states that the SNPRM places great importance on having publicly available data for reanalysis of peer-reviewed studies, but the Agency provides minimal detail regarding how it will determine whether researchers have actually fulfilled this reanalysis requirement. The commenter states that giving the Administrator or staff leeway to "consider" one study more valuable than another based on criteria unrelated to quality clearly provides opportunity to inject politics into public health and environmental decision-making. The commenter states that, while no two studies are likely to be identical, this vague approach would actually force the Agency to rely on one more than another (whether or not the study receiving the greater weight is actually more accurate or up to date), and is arbitrary and capricious.
Commenter (11492) states that the Option 2 approach arbitrarily devalues scientific work that relies on health data containing personal identification. The commenter states that the EPA is charged with protecting public health and the environment, and as such, studies that involve health data are critical. The commenter states that, giving less consideration, or no consideration, to studies where underlying data are not publicly available is a political decision that has nothing to do with science. The commenter states that this is contrary to EPA's mission to protect human health and the environment based on best-available science.
Response: Though not exhaustive or exclusive, the EPA has identified several factors that balance some of the important technical considerations the EPA will consider in addition to data availability and that are particularly relevant to the stage of the analysis where dose-response data are used. The EPA will thus consider the following factors:
 the quality of the study relative to other studies for which the dose-response data are available; 
 the extent to which there are other studies for which the dose-response data are available;
 the sensitivity of the conclusions in the significant regulatory action or influential scientific information based on the use of the study;
 the extent to which the study is fit for the purpose or intended use relative to other studies for which the dose-response data are available;
 the use of exposures or doses in a range and duration that is relevant for the intended use and that minimizes the need for extrapolations;
 the extent to which the study is supported by other scientific evidence;
 the extent to which the study accounts for unique scientific considerations;
 the extent to which the study minimizes the use of defaults and assumptions, uses appropriate and strong statistical methods, and includes a robust representation of uncertainty and confidence intervals; and
 the study's consideration of a broad range of parametric dose-response or concentration-response models, a robust set of potential confounding variables, nonparametric models that incorporate fewer assumptions, various threshold models across the dose or exposure range, and models that investigate factors that might account for spatial heterogeneity.
Comment: Commenter (11495) states that weighing studies based on their ability to comply with EPA Federal Housekeeping internal requirements is unscientific, arbitrary, burdensome and capricious. 
Response: The EPA disagrees with the commenter. As discussed more fully in the preamble to the final rule, the EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. This rule builds on the work that is already being done to increase the availability of underlying data in the scientific community and in the federal government. The EPA also disagrees with the commenter that the requirements of this rule will result in any meaningful delay in promulgating regulations. While this final rule requires the Agency to evaluate the availability of dose-response data for pivotal science, the incremental burden to the Agency to carry out these requirements is expected to be small given (1) the extensive scientific review the EPA already conducts regularly and (2) that the requirement is limited to pivotal science (i.e., typically a small, though highly important, subset of the studies the EPA would review). 
Comment: Commenter (11576) asserts the EPA cannot arbitrarily give greater or lesser weight to studies solely because of the public availability of underlying data. The commenter states that the Agency has failed to articulate a rational connection between the availability of underlying data and the quality, significance, and scientific soundness of a study. The commenter states that, by reducing the weight given to studies on the basis of data availability, the EPA would wrongly downgrade the importance of study findings, which would ultimately affect the EPA's ability to address well-documented health harms.
Commenter (12430) states that there is simply no appropriate and non-arbitrary method for U.S. EPA to deprioritize studies or models relying on non-public data. Commenter (12688) expresses concern that Option 2 is based on arbitrary criteria.
Commenter (12731) states that, while the EPA indicates the Agency could downgrade its consideration of many valid studies and seeks input on how much to diminish their weight,  discarding or discounting otherwise valid studies, for no reason other than an a priori judgment made without regard to study quality, is arbitrary and illegal, for the same reason that disregarding such studies altogether is illegal.
Commenter (13788) states that there are critically important questions that neither the proposal nor the SNPRM answers or explains. The commenter states that the limited information provided about the alternative approach suggests that it would be arbitrary because, like the original approach, it would force the Agency to distinguish between studies and treat them differently based on arbitrary criteria. The commenter states that data availability has no connection to the validity and quality of science. 
Commenter (12715) asserts that the EPA's "alternate approach" to 40 CFR 30.5 fails to address concerns regarding the arbitrary down-weighting of relevant scientific studies. The commenter states that, under Option 2, the EPA could consider studies for which the data is not available or where access is limited. The commenter states that, however, the proposal identifies no criteria for when or how the EPA would exercise the vast discretion the Agency creates for itself. 
Response: Though not exhaustive or exclusive, the EPA has identified several factors that balance some of the important technical considerations the EPA will consider in addition to data availability and that are particularly relevant to the stage of the analysis where dose-response data are used. The EPA will thus consider the following factors:
 the quality of the study relative to other studies for which the dose-response data are available; 
 the extent to which there are other studies for which the dose-response data are available;
 the sensitivity of the conclusions in the significant regulatory action or influential scientific information based on the use of the study;
 the extent to which the study is fit for the purpose or intended use relative to other studies for which the dose-response data are available;
 the use of exposures or doses in a range and duration that is relevant for the intended use and that minimizes the need for extrapolations;
 the extent to which the study is supported by other scientific evidence;
 the extent to which the study accounts for unique scientific considerations;
 the extent to which the study minimizes the use of defaults and assumptions, uses appropriate and strong statistical methods, and includes a robust representation of uncertainty and confidence intervals; and
 the study's consideration of a broad range of parametric dose-response or concentration-response models, a robust set of potential confounding variables, nonparametric models that incorporate fewer assumptions, various threshold models across the dose or exposure range, and models that investigate factors that might account for spatial heterogeneity.
7.1.4.7 Implementation of Option 2
7.1.4.7.1 Criteria
Comment: Commenters (11192, 11361, 11390, 11391, 11399, 11911, 12720, 12731, 13788) express concern that, while the SNPRM states that "the Agency will give greater consideration to studies where the underlying data and models are publicly available in a manner sufficient for independent validation," the EPA provides no information about potential weighting criteria or calculations.
Commenter (11390) expresses concern that, in the likely event that the Agency is presented with multiple studies that could be considered "high-quality" but that are based on varying amounts of data and whose conclusions are supported with varying robustness, there are no criteria for the Agency to apply in determining which studies to consider, or how heavily to weigh them. The commenter states that, at best, the lack of clear guidance on how studies should be weighted would lead to inconsistent and unclear application of the proposed rule.
Commenter (12731) states that the EPA provides no details as to how the differential weighting alternative approach would work in practice. The commenter states that this description is entirely ambiguous, lacking sufficient detail not only as to how the EPA would intend to apply this alternative approach in practice (including how much weight it would give to the availability of data and models in assessing studies), but also under what circumstances the Agency would or would not decide to apply this alternative approach as the EPA maintains it may consider studies where underlying data is not available or only available on a limited basis. The commenter states that, in the EPA's briefing to House Science, Space, and Technology Committee staff, the Agency "was unable to expand on how this potential weighted system would operate. The Agency has not identified implementation details for the weighting approach, including any concrete ideas about how the scale of a weighted system would be structured." 
Commenter (12731) states that the fact that the EPA itself does not understand how the alternative would work confirms that the Agency has not provided adequate notice for public comment, and it cannot finalize any such alternative without first examining the details of its implementation. 
Response: Though not exhaustive or exclusive, the EPA has identified several factors that balance some of the important technical considerations the EPA will consider in addition to data availability and that are particularly relevant to the stage of the analysis where dose-response data are used. The EPA will thus consider the following factors:
 the quality of the study relative to other studies for which the dose-response data are available; 
 the extent to which there are other studies for which the dose-response data are available;
 the sensitivity of the conclusions in the significant regulatory action or influential scientific information based on the use of the study;
 the extent to which the study is fit for the purpose or intended use relative to other studies for which the dose-response data are available;
 the use of exposures or doses in a range and duration that is relevant for the intended use and that minimizes the need for extrapolations;
 the extent to which the study is supported by other scientific evidence;
 the extent to which the study accounts for unique scientific considerations;
 the extent to which the study minimizes the use of defaults and assumptions, uses appropriate and strong statistical methods, and includes a robust representation of uncertainty and confidence intervals; and
 the study's consideration of a broad range of parametric dose-response or concentration-response models, a robust set of potential confounding variables, nonparametric models that incorporate fewer assumptions, various threshold models across the dose or exposure range, and models that investigate factors that might account for spatial heterogeneity.
7.1.4.7.2 Detailed Description of Option 2
Comment: Commenters (9587, 11390, 11391, 11392, 11495, 11576, 11894, 12464, 12720, 13788) contend the proposed revisions should include a more detailed description for implementation of Option 2 as described below. The commenters suggest the following implementation questions need to be addressed:
 Are 10 studies that do not meet the data sharing requirements, but reach a similar finding, outweighed by 1 study that does meet the data sharing requirement, but has a contrary finding? (11495)
 Could the EPA explain away ignoring a pivotal public health study by simply saying it was given "less consideration," implying that it did in fact inform a problematic new pollution standard? (11392)
 How the EPA would apply such weighting? (11390, 12464)
 How would the EPA determine whether "other things" are equal in any given circumstance? (11390)
 How will the EPA create the necessary infrastructure or capacity to handle such decisions in a timely manner? (11391)
 How much greater consideration will the EPA give to studies where the underlying data and models are available for independent validation? (9587)
 How much additional weight will certain studies receive? How will that be determined? (13788)
 How would the weighting system impact existing weighting metrics used, such as the greater consideration given to human health studies over animal studies in NAAQS reviews? (11392)
 When the EPA would apply such weighting? (11390)
 What would the EPA consider to be a "high-quality" study? (11390)
 What are the parameters for making these decisions? (11391)
 While human health studies with the most medical information are currently given the most weight, animal studies will have no data-sharing complications  --  will the science-based metrics be overturned with a system based entirely on transparency of data? (11392)
 Who will have access to the information and what type of information they will have access to? (11576)
 Who will determine whether underlying data and models are available in a "sufficient" manner when, as EPA itself has pointed out, data are sometimes available in a graduated or tiered manner (i.e., data availability exists on a spectrum)? (11391)
 Will the EPA publish a regularly updated database indicating its determination about how much weight each study is given? If so, how will the public be able to access this information? (12720)
 Will the decisions made by other federal agencies using the same studies be necessarily affected by the determinations that the EPA makes? (12720)
 Will these determinations be final, or changeable? If final, will there be opportunities for the study authors or researchers to dispute the determination? If changeable, what is the process by which such a determination could be changed, and who within the EPA would have that authority? (12720)
 Would the EPA develop objective metrics to determine when and how -greater consideration" would be given to certain studies, or would decisions be made on a case-by-case basis without any established procedures? (11392)
 Would giving less "consideration" in assessments devolve into giving no weight? (11894)
Commenters (11392, 12431, 12720) contend that the EPA fails to provide sufficient description or necessary analysis of the Agency's Option 2 approach. The commenters suggest that the following information is necessary:
 The EPA must provide greater clarity on how such a weighted system would operate in practice, where decision making authority would reside, and what the implications would be for past studies that underpin the Agency's regulatory standards. (11392)
 The EPA needs to clarify whether a determination on degree of consideration would apply consistently across all the EPA staff, science advisors, and subject matter areas relevant to the EPA work. (12720)
 The EPA needs to clarify whether making data "available" for independent validation is sufficient for such increased consideration, or whether such independent validation must have actually occurred prior to the EPA making such a decision. (12720)
 Because the Agency relies on scientific studies across a range of disciplines and for a range of activities, the EPA needs to clarify how variations in weight given to specific studies would actually be implemented. (12720)
 The EPA needs to clarify what this greater consideration would entail. (12431)
Commenters (10752, 11192, 11361, 11399, 11911) contend that, as the EPA has proposed no criteria or oversight mechanisms, Option 2 is open to manipulation and, thus, is unacceptable.
Commenter (10752) states that the proposal's inherent ambiguity makes it unacceptable as a legal regulation. The commenter states that the EPA proposes to arbitrarily pick and choose which data to rely on with the presumption that the nonconfidential information is given more weight. The commenter states that it is safe to assume that, in the current regulatory climate, the EPA will always choose the industry data and models.
Commenters (11192, 11361) state that the Agency provides no example weighting criteria or maximum weighting ratios that might inform the public about what could be expected from this weighting proposal. The commenter states that, as a result ,this generalized suggestion could be used to mollify those that oppose dismissal of public health studies that couldn't make their data public, but in essence, still dismiss them by assigning them a very small weighting factor.
Commenter (11399) asserts that Option 2 would impose a wholly subjective framework by which the EPA would determine the weight that should be afforded to scientific studies based on their transparency. Commenter (11911) asserts that the SNPRM's Option 2 is open to manipulation without transparent criteria informed by independent scientific expertise. 
Response: Though not exhaustive or exclusive, the EPA has identified several factors that balance some of the important technical considerations the EPA will consider in addition to data availability and that are particularly relevant to the stage of the analysis where dose-response data are used. The EPA will thus consider the following factors:
 the quality of the study relative to other studies for which the dose-response data are available; 
 the extent to which there are other studies for which the dose-response data are available;
 the sensitivity of the conclusions in the significant regulatory action or influential scientific information based on the use of the study;
 the extent to which the study is fit for the purpose or intended use relative to other studies for which the dose-response data are available;
 the use of exposures or doses in a range and duration that is relevant for the intended use and that minimizes the need for extrapolations;
 the extent to which the study is supported by other scientific evidence;
 the extent to which the study accounts for unique scientific considerations;
 the extent to which the study minimizes the use of defaults and assumptions, uses appropriate and strong statistical methods, and includes a robust representation of uncertainty and confidence intervals; and
 the study's consideration of a broad range of parametric dose-response or concentration-response models, a robust set of potential confounding variables, nonparametric models that incorporate fewer assumptions, various threshold models across the dose or exposure range, and models that investigate factors that might account for spatial heterogeneity.
7.1.4.7.3 Other
Comment: Commenter (10339) maintains that publicly funded, peer-reviewed studies should not be disadvantaged during Agency scientific reviews. 
Response: The EPA disagrees that publicly funded, peer-reviewed studies would be disadvantaged in any way. The final rule includes sufficient flexibility in terms of the allowance for restricted access, considerations in 40 CFR 30.5, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data to minimize such potential unintended consequences.
Comment: Commenters (11176, 11390, 11492, 11495, 12688, 12727) assert there is no scientific basis for employing the Option 2 weighting system.
Commenter (11176) states that the category of "non-federally funded and non-the EPA data" likely makes up the largest category informing the EPA's many actions. The commenter states that it is also the category that poses the most challenges for making data available. The commenter states that such studies generally provide important scientific contributions and should not be discounted because their data are not readily available. The commenter states that, in our 2018 comments, we included two examples of research that might be excluded: private-sector studies and meta-analyses. The commenter states that, while such work will not necessarily be excluded through the proposed rule with the SNPRM, it is likely to be de-emphasized to the detriment of evidence-based policymaking at the EPA. The commenter states that scientific reasoning would not support the following aspect of the SNPRM: "other things being equal, the Agency will give greater consideration to studies where the underlying data and models are available in a manner sufficient for independent validation either because they are publicly available or because they are available through tiered access when the data includes CBI, proprietary data, or PII that cannot be sufficiently de-identified to protect the data subjects." The commenter states that the EPA has not scientifically justified that greater consideration should be given to such studies. 
Commenter (11492) states that the use of meta-analysis (when results from pertinent, multiple studies are combined) is becoming increasingly common, and they are crucial to understanding epidemiology and public health issues. The commenter states that the Supplemental Rule requires a brief description of why one study is given "greater consideration" over another, but does not give any direction as to how to weigh these different studies. The commenter states that failure of the Supplemental Rule to address basic questions such as these is indicative that the authors have no understanding of scientific analysis.
Commenter (11495) states that federal agencies should have the ability to individually evaluate and weight scientific studies based on the strength of the science using criteria such as participant selection, disclosure of methods and authentication of reagents, quality of exposure assessment, strength of association, consistency, specificity, temporality, evidence of dose-response, biological plausibility, and assessment of potential bias or confounding. The commenter states that weighing studies based on their ability to comply with the EPA Federal Housekeeping internal requirements is unscientific. The commenter states that it hinders the EPA by depriving the Agency of some of the best science on which it might make its decisions. 
Commenter (12688) contends that Option 2 has no basis in scientific practice. The commenter states that it will hamper the Agency's ability to use high-quality science produced by researchers at institutions like UCLA. The commenter states that regulatory agencies should do what scientists do: consider studies in accordance with the robustness of their methodology and conclusions, and not based on arbitrary criteria. 
Commenter (12727) states that the EPA proposes to "give greater consideration" when reviewing studies based on data's public availability, but this is not a science-based approach. The commenter states that the proposal to "downgrade" studies due to their public accessibility does not appear to improve any aspect of science-based decision making. The commenter states that public accessibility for the purpose of "independent validation" does not mean that the data are high quality, or even useful from a scientific perspective. The commenter states that, for example, epidemiological data on real-life human exposures are valuable for science-based decisions on chemical exposures, and yet much these data are rightfully protected as PII.
Response: Though not exhaustive or exclusive, the EPA has identified several factors that balance some of the important technical considerations the EPA will consider in addition to data availability and that are particularly relevant to the stage of the analysis where dose-response data are used. The EPA will thus consider the following factors:
 the quality of the study relative to other studies for which the dose-response data are available; 
 the extent to which there are other studies for which the dose-response data are available;
 the sensitivity of the conclusions in the significant regulatory action or influential scientific information based on the use of the study;
 the extent to which the study is fit for the purpose or intended use relative to other studies for which the dose-response data are available;
 the use of exposures or doses in a range and duration that is relevant for the intended use and that minimizes the need for extrapolations;
 the extent to which the study is supported by other scientific evidence;
 the extent to which the study accounts for unique scientific considerations;
 the extent to which the study minimizes the use of defaults and assumptions, uses appropriate and strong statistical methods, and includes a robust representation of uncertainty and confidence intervals; and
 the study's consideration of a broad range of parametric dose-response or concentration-response models, a robust set of potential confounding variables, nonparametric models that incorporate fewer assumptions, various threshold models across the dose or exposure range, and models that investigate factors that might account for spatial heterogeneity.
Comment: Commenter (11176) states that research that uses CBI, proprietary data, or PII is likely to provide greater insights into health effects pertinent to the EPA's work precisely because of these data. The commenter states that, because such data will not be "publicly available or available through tiered access," the potentially more insightful research that relies on these data is likely to be discounted by this proposed rule to the detriment of the EPA's important work. The commenter states that, for CBI, for example, the EPA uses such data to identify best available technologies (BAT) upon which many water regulations are based. The commenter states that companies only provide the data upon which BAT can be determined with the proviso that the data remain CBI. The commenter states that, without accurate data on the BAT currently in use, such regulations would be based on weaker science. The commenter states that the proposed rule poses three risks for this example: (i) businesses may have less confidence their CBI will be protected and may be reluctant to share their information; (ii) the EPA may discount research based on CBI weakening the scientific underpinning of a regulation; and (iii) water quality may suffer. 
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (11391) states that implementing Option 2 would require so much additional complex review before important Agency rulemakings and other actions could be accomplished that a significant and deeply problematic regulatory bottleneck is easily foreseeable. The commenter states that the EPA has offered no indication that it has considered these problems nor how it plans to develop the necessary internal infrastructure or capacity to deal with them. The commenter states that the SAB raised concerns in its report on the proposed rule, noting that the requirement to make data available is unclear, "mak[ing] it difficult to understand the implications of the requirement." The commenter states that, according to the SAB, depending on how the requirement is interpreted, compliance with it could "be enormously expensive and time consuming." The commenter states that the SAB goes so far as to suggest that the EPA may have to establish an office on data sharing, or peer review panel or working group to implement the requirement. The commenter shares this view.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (11395) does not agree that data accessibility should be the primary factor used to determine how a study is weighted relative to other studies. 
Response: Though not exhaustive or exclusive, the EPA has identified several factors that balance some of the important technical considerations the EPA will consider in addition to data availability and that are particularly relevant to the stage of the analysis where dose-response data are used. The EPA will thus consider the following factors:
 the quality of the study relative to other studies for which the dose-response data are available; 
 the extent to which there are other studies for which the dose-response data are available;
 the sensitivity of the conclusions in the significant regulatory action or influential scientific information based on the use of the study;
 the extent to which the study is fit for the purpose or intended use relative to other studies for which the dose-response data are available;
 the use of exposures or doses in a range and duration that is relevant for the intended use and that minimizes the need for extrapolations;
 the extent to which the study is supported by other scientific evidence;
 the extent to which the study accounts for unique scientific considerations;
 the extent to which the study minimizes the use of defaults and assumptions, uses appropriate and strong statistical methods, and includes a robust representation of uncertainty and confidence intervals; and
 the study's consideration of a broad range of parametric dose-response or concentration-response models, a robust set of potential confounding variables, nonparametric models that incorporate fewer assumptions, various threshold models across the dose or exposure range, and models that investigate factors that might account for spatial heterogeneity.
Comment: Commenter (11495) states that, without detailed, consistent guidelines delineating exactly how studies will be weighted, as well as how missing data will be accounted for, the quality of this new document in no way approaches the quality of the EPA's existing documents that define inclusion and exclusion criteria, currently based on publication quality judged by rigorous scientific standards. The commenter states the SNPRM brings forward a new and complex process without any consideration to ensure that information is "accurate, clear, unbiased and complete", as mandated by the federal government, let alone how the policy will be practically implemented. 
Commenter (12731) states that the EPA has failed to explain how differential weighting of studies based on data availability adds to, modifies, or is needed in light of existing, relevant frameworks at the Agency including, for example, frameworks developed for the EPA IRIS toxicological reviews, the development of Integrated Science Assessments, and ecological assessments.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Though not exhaustive or exclusive, the EPA has identified several factors that balance some of the important technical considerations the EPA will consider in addition to data availability and that are particularly relevant to the stage of the analysis where dose-response data are used. The EPA will thus consider the following factors:
 the quality of the study relative to other studies for which the dose-response data are available; 
 the extent to which there are other studies for which the dose-response data are available;
 the sensitivity of the conclusions in the significant regulatory action or influential scientific information based on the use of the study;
 the extent to which the study is fit for the purpose or intended use relative to other studies for which the dose-response data are available;
 the use of exposures or doses in a range and duration that is relevant for the intended use and that minimizes the need for extrapolations;
 the extent to which the study is supported by other scientific evidence;
 the extent to which the study accounts for unique scientific considerations;
 the extent to which the study minimizes the use of defaults and assumptions, uses appropriate and strong statistical methods, and includes a robust representation of uncertainty and confidence intervals; and
 the study's consideration of a broad range of parametric dose-response or concentration-response models, a robust set of potential confounding variables, nonparametric models that incorporate fewer assumptions, various threshold models across the dose or exposure range, and models that investigate factors that might account for spatial heterogeneity.
The flexibility provided in the rule in terms of the allowance for restricted access, considerations in 40 CFR 30.5, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data will minimize any potential unintended consequences.
7.1.5 Ensure Over Time More Availability of Data
Comment: Commenter (11176) suggests that, by working together with other agencies to develop common standards and provide resources to researchers and research sponsors to create data governance systems that support tiered data access, the EPA can ensure that over time more of the data and models underlying the science that informs significant regulatory decisions are available publicly or via restricted access.
Response: The final rule does not regulate or impose requirements on any party outside of the EPA but rather exclusively governs the EPA's internal process for handling pivotal science. This rule does not require any researcher or other outside entity to provide dose-response data to the EPA. It also does not direct or require any outside entity or EPA to establish data sharing mechanisms. Further, the rulemaking does not require the EPA to collect, store, or publicly disseminate dose-response data and models underlying pivotal science. Instead, it governs internal agency procedures for weighing its consideration of various studies according to factors that include data and model availability.
Comment: Commenter (11576) states that seeking to require or promote greater public accessibility to models and data underlying studies is not the EPA's role; the Agency was established to protect public health and the environment using the best available science, not to attempt to tell scientists how to do their jobs. The commenter states that there is no way to ensure that the EPA's approach would honor "legal and ethical obligations to reduce the risks of unauthorized disclosure and re-identification[,]" nor does the EPA offer any explanation as to how it would address these issues. Commenter (11576) notes that the SNPRM states that:
      the EPA is requesting comment on how to ensure that, over time, more of the data and models underlying the science that informs significant regulatory decisions and influential scientific information are available to the public for independent validation in a manner that honors legal and ethical obligations to reduce the risks of unauthorized disclosure and re-identification". ... [And] the EPA is interested in comments about how to provide sufficient incentives and support to researchers to increase access to the data that may be used as pivotal regulatory science or pivotal science.
Response: The rule does not impose requirements on any entity outside of EPA. This is a rule of internal procedures and does not direct or require any outside entity or EPA to establish data sharing mechanisms. Further, the rulemaking does not require the EPA to collect, store, or publicly disseminate dose-response data for underlying pivotal science. 
The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. 
Comment: Commenter (11892) suggests that, because access to information can be limited by for-profit publications requiring payment to view full peer-reviewed articles, all work funded in part by the federal government should allow free full access to published articles.
Response: EPA has implemented the public access requirements for EPA researchers, grantees and contractors as required by the February 22, 2013, OSTP memorandum "Increasing Access to the Results of Federally Funded Scientific Research" and the 2016 EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research." EPA researchers, grantees and contractors are required to make peer-reviewed journal articles and the underlying data and models resulting from EPA-funded research accessible to the public.
The EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research" requires the data underlying a journal article resulting from research funded by EPA be made publicly accessible. However, the rule requires the EPA to evaluate the availability of the analyzable dataset, not the more limited dataset that accompanied the publication of the study or model. For this rulemaking EPA will not identify a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. The step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
Comment: Commenter (9587) suggests the EPA should provide a time limit for its willingness to accept privacy, confidentiality, and tiered access. The commenter states that scientific research can meet full transparency standards in time, even if it cannot meet it immediately. The commenter states that a time limit will provide the EPA researchers and independent scientists an incentive to change their practices. The commenter suggests a time limit of no more than 20 years -- say, 31 December 2040. The commenter states that this time limit might be subdivided, so that fields that require substantial longitudinal studies have longer time limits than fields that do not.
Response: The EPA clarified in the final rule that the public must be able to access the information on the date of publication of the proposed rule (or, as appropriate, a supplemental notice of proposed rulemaking, or notice of availability) for the significant regulatory action or on the date of dissemination of the draft influential scientific information for public review and comment.
Comment: Commenter (10974) states that, if the EPA were to spend their time and direct their resources towards ensuring that their own data is easily accessible in updated and user-friendly tools, they would model the way for, and incentivize and support, other researchers as they work to become more creative and thorough with data transparency. The commenter states that, rather than promulgating this new rule, the EPA should be taking immediate action to demonstrate data leadership  -  doing so would be a much more progressive and useful approach, and would model good governance and public service. Furthermore, the commenter provides that EPA would be better served supporting state efforts to make existing data easier to understand, to help better communicate research results and conclusions, and to help states develop third-party partnerships.
Response: EPA has implemented the public access requirements for EPA researchers, grantees and contractors as required by the February 22, 2013, OSTP memorandum "Increasing Access to the Results of Federally Funded Scientific Research" and the 2016 EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research." EPA researchers, grantees and contractors are required to make peer-reviewed journal articles and the underlying data and models resulting from EPA-funded research accessible to the public.
The EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research" requires the data underlying a journal article resulting from research funded by EPA be made publicly accessible. However, the rule requires the EPA to evaluate the availability of the analyzable dataset, not the more limited dataset that accompanied the publication of the study or model. For this rulemaking EPA will not identify a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. The step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
Comment: Commenter (11401) states that, applying the new standard to existing studies would encourage researchers to make their underlying data more transparent in accordance with the tiered approach, thereby benefiting not just the EPA rulemakings, but scientific knowledge to the broader society. The commenter states that encouraging researchers to release data for existing studies also addresses the EPA's question in the SNPRM on "how to provide sufficient incentive...to researchers to increase access to the data that may be used as pivotal regulatory science or pivotal science." The commenter states that allowing their existing, nontransparent studies to be used in the future would discourage greater openness and hinder the contribution to better science. 
Response: The EPA is maintaining that the rule provisions apply equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that are unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator, as detailed under 40 CFR 30.7. One of the criteria the Administrator may use in his/her determination to grant a waiver is that the development of the dose-response data was completed or updated before the effective date of the rule. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
The considerations in 40 CFR 30.5, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data are intended to minimize potential unintended consequences, such as the potential skew toward newer, domestic, and/or industry-funded studies suggested by the commenters. In addition, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. The EPA believes the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. 
Comment: Commenter (12596) states that a Transparency Scoring System would provide incentives and support to researchers to increase access to the data that may be used as pivotal regulatory science or pivotal science. The commenter states that it would inform the public as to the methodologies underlying important pivotal regulatory science or pivotal science. The commenter states that those signals would be bi-directional, encouraging researchers to adopt transparency protocols with the incentive of having their studies receive greater consideration in the EPA rulemakings. The commenter states that such a framework would encourage researchers to fundamentally incorporate "transparency by design" into their studies.
Commenter (11386) contends the EPA should not incentivize researchers to provide public access to their data. The commenter states that consideration for rulemaking should not be used by the EPA as an incentive for researchers, nor should grant money be allocated on the basis of the feasibility of making data publicly available. The commenter states that both of these incentives potentially conflict with the primary purpose of the EPA and the EPA-funded research, which is to protect public health and the environment. The commenter states that it is admirable to make data public when possible, but the EPA should not pressure scientists to do so when it is not, for instance, in the best interests of the subjects in the study. The commenter states that the mission of the EPA is to protect human health and the environment, and providing regulatory incentives that influence the direction of scientific research does not fall within that mission. The commenter states that, while it would be inappropriate to base public health decisions on data transparency, it is appropriate and beneficial for the EPA to provide support and resources for increasing the access to scientific data. The commenter states that examples of ways in which the EPA can give this support include providing those carrying out the research with space on servers on which relevant data can be stored and made publicly accessible, providing access to computing power for running reanalysis, or providing a data-management team. The commenter states that these resources would lead to an increase in availability of the data and models underlying the science that informs significant regulatory decisions and influential scientific information. The commenter states that such support structures from the EPA would enable relevant data to be publicly accessible independent of the resources of the research team. The inherent value and validity of the data and of the conclusions drawn from its analyses would be unchanged by these methods of increasing transparency.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
The EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Commenter (12464) states that, while academic researchers have no incentive to revisit their old research and make the raw data and models publicly available, industry scientists have a strong monetary incentive to do so if their research supports the industry's goals.
Commenter (11103) suggests that lowering barriers associated with making data available is a better mechanism for change than the supplemental proposal. The commenter references the 2019 NASEM report, which provides that the lack of a supportive framework for adequate recordkeeping disincentivizes researchers to create the conditions for reproducibility. The commenter states EPA can address this by fostering the development of new software tools, but by providing adequate training on how to use such tools and education in the importance of transparency in scientific studies. To accomplish this, the commenter recommends partnering with educational institutions and leveraging the working relationships EPA's Office of Public Engagement and Environmental Education (OPEEE) already has, such as the U.S. Department of Education, to create programs that promote a culture of openness in the scientific community The commenter provides the "Reproducible Research" course at John Hopkins University as such a program.
Response: The EPA would like to emphasize that this final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenter (12465) expresses concern that the treatment of existing data or models may disincentivize the generation of new data under the new transparency requirements while reducing the importance of considering existing data, regardless of the scientific validity or merit of that data. Additionally, the commenter notes there are similar considerations between statutes and regulations that are outside of the scope of the supplemental proposal and recommends that these provisions be revisited in an effort to resolve the inherent conflict.
Conversely, Commenter (11401) notes that applying the new standard to existing studies would encourage researchers to make their underlying data more transparent in accordance with the tiered approach, thereby benefitting not just EPA rulemakings, but scientific knowledge to the broader society.
Response: The EPA is maintaining that the rule provisions apply equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that are unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator, as detailed under 40 CFR 30.7. One of the criteria the Administrator may use in his/her determination to grant a waiver is that the development of the dose-response data was completed or updated before the effective date of the rule. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
The considerations in 40 CFR 30.5, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data are intended to minimize potential unintended consequences, such as the potential skew toward newer, domestic, and/or industry-funded studies suggested by the commenters. In addition, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. The EPA believes the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making.
 Chapter 8: Research Data
Title 40, Code of Federal Regulations part 30.10 (40 CFR 30.10) of the notice of proposed rulemaking (NPRM) proposes to require the Environmental Protection Agency (EPA) to implement the rule consistent with the definition of research data in Uniform Administrative Requirements, Cost Principles, and Audit Requirements for Federal Awards, exemptions in Public Law 89 - 487, and other applicable federal laws. From the Uniform Administrative Requirements, Cost Principles, and Audit Requirements for Federal Awards: 
      Research data means the recorded factual material commonly accepted in the scientific community as necessary to validate research findings, but not any of the following: preliminary analyses, drafts of scientific papers, plans for future research, peer reviews, or communications with colleagues. This "recorded" material excludes physical objects (e.g., laboratory samples). Research data also do not include: (i) Trade secrets, commercial information, materials necessary to be held confidential by a researcher until they are published, or similar information which is protected under law; and (ii) Personnel and medical information and similar information the disclosure of which would constitute a clearly unwarranted invasion of personal privacy, such as information that could be used to identify a particular person in a research study. 
Several comments were received regarding the NPRM with regards to research data as defined under the Uniform Administrative Requirements. In this section these comments are organized as follows:
       Uniform Administrative Requirements
       Applicability of the Rule to Research Laboratory Samples
The specific comments provided in this Chapter reflect the comment letters that provide arguments supported by detailed rationale, examples, and references/documentation. The Docket ID numbers for these commenters are cited throughout this Chapter. In addition, some general comment letters may also include comments related to comments included and responded to in this Chapter. These comments are summarized in Appendix B. Note that points and issues raised in these general comment letters may also be raised by the more detailed comments summarized in this Chapter. The responses provided in this Chapter adequately address these more general comments.
8.1 Uniform Administrative Requirements
Comment: Commenters (6111, 6126, 6880, 6924) assert that the proposed requirements in §30.10 are inconsistent with aspects of the other proposed requirements. The commenters assert that, while proposed §30.10 requires EPA to implement the rule consistent with the definition of research data in the Uniform Administrative Requirements, the proposed rule appears to require all information be made publicly available, despite exclusions in the Uniform Administrative Requirements regarding medical information. The commenters refer to the Uniform Administrative Guidance which exempts from its definition of "research data" subject to disclosure any "medical information and similar information the disclosure of which would constitute a clearly unwarranted invasion of personal privacy, such as information that could be used to identify a particular person in a research study." 
Commenter (6126) suggests that, to be consistent with the definition of "research data" in Office of Management and Budget (OMB) Circular A-110 and with EPA's existing guidance, the rule appears to preclude collection of identifiable information. The commenter states that, however, such information could be necessary to conduct replication and validation of studies. The commenter states that this would reflect the type of data available through the Federal Statistical Research Data Centers and referred to by the U.S. Commission on Evidence-Based Policymaking, which are cited in the preamble text. The commenter states that, if EPA's intent is to only consider replicable studies those with identifiable data but to preclude such information from the definitions, the rule's organizing logic must be deemed tautological. The commenter states that clarification is needed regarding whether EPA intends for identifiable data to be available to fulfill the regulatory requirements or if the intent is to rely exclusively on de-identified data. 
Commenter (6880) states that the preamble and proposal are contradictory as to what data are subject to the proposed rule. The commenter states that, in the preamble, EPA makes the following statement that is unsupported by the text of the proposal: "Implementation of this rule will be consistent with the definition of `research data' in Uniform Administrative Requirements, Cost Principles, and Audit Requirements for Federal Awards." The commenter states that, in the cited regulation, the OMB defines "research data" within the context of identifying what information a federal agency must release to satisfy a Freedom of Information Act (FOIA) request for data generated by research funded through that agency. The commenter states that the definition specifically excludes certain confidential information from the definition of "research data" (§200.315(e)(3), (3)(i), and (3)(ii)). 
Commenter (6924) asserts that the rule's requirement to make all information publicly available that would be needed to complete a reanalysis conflicts with the exclusions in the definition of "research data."
Response: Proposed 40 CFR 30.2 in the 2018 proposed rule included a definition of research data. In the 2020 SNPRM, the EPA deleted the proposed definition of research data. While one commenter on the 2020 SNPRM noted that the exclusions in the definition of research data of trade secrets and personal and medical information were not incorporated into the definition of data, commenters did not request that the EPA maintain a definition of research data. The EPA is not including a definition of "research data" in this final rule because it includes the definition of "data" in section 30.2 as follows:
      Data means the set of recorded factual material commonly accepted in the scientific community as necessary to validate research findings in which obvious errors, such as keystroke or coding errors, have been removed and that is capable of being analyzed by either the original researcher or an independent party.
In addition, the EPA would like to emphasize that this final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science. The rule allows the EPA Administrator to grant an exemption to the transparency requirements if the Administrator determines that greater consideration is warranted when: 
 Technological or other barriers render sharing of the dose-response data infeasible;
 The development of the dose-response data was completed or updated before the effective date of this final rule;
 Making the dose-response data would conflict with laws and regulations governing privacy, confidentiality, confidential business information (CBI), or national security;
 Third-party independent validation of the study's underlying dose-response data through reanalysis has been conducted; or
 The factors in 40 CFR 30.5 used in determining the consideration to afford to the pivotal science indicate full consideration is justified.
8.2 Applicability of the Rule to Research Laboratory Samples.
Comment: Commenter (6143) suggests that the Agency apply the proposed requirements to a limited and special class of laboratory samples, the analytical results of which (from non-destructive testing) may become an essential element of the research that underpins a given regulation. The commenter states that, not only can different laboratory analysts reach different conclusions by analyzing the same sample, it is common for analytical methods to improve over time. The commenter notes that improved analytical methods provide researchers with the opportunity to re-analyze properly archived samples, potentially yielding more accurate and even different results. According to the commenter, it should become part of the researchers' protocol to retain and properly archive this special class of samples when the analytical results are likely to inform or inspire the regulatory process. The commenter asserts that it is particularly true for research conducted by or for EPA and its contractors; and by any party conducting research with the intent to inform or inspire a specific regulatory action. (While it is true that certain types of samples cannot be retained because they are inherently unstable chemically and/or physically, other types of samples can remain stable for decades when properly archived.)
Response: EPA disagrees with the commenter's suggestion to extend the scope of the rule to laboratory samples. As stated in 40 CFR 30.3, the provisions of this part do not apply to physical objects (like laboratory samples). Physical samples are not needed to independently validate pivotal science in the context of this rule. 
 Chapter 9: Science That Serves as the Basis for Informing a Final Significant Regulatory Actions
Section 30.1 of the proposed rule requires the Environmental Protection Agency (EPA) to ensure that the regulatory science underlying its actions be publicly available in a manner sufficient for independent validation. Section 30.2 of the proposed rule includes a definition of regulatory science as scientific information, including assessments, models, criteria documents, and regulatory impact analyses, that provide the basis for EPA final significant regulatory decisions. Several comments were received regarding "regulatory science". In this section these comments are organized as follows:
Section 9.1: NPRM Regulatory Science (section 30.2)
       Ensure that the Regulatory Science is Publicly Available in a Manner Sufficient for Independent Validation (§30.1)
       Regulatory Science Means Scientific Information that Provides the Basis for EPA Final Significant Regulatory Decisions (§30.2)
       EPA's Regulatory Science Should Be Consistent with the OMB's Final Information Quality Bulletin for Peer Review (83 FR 18770)
       EPA's Proposed Regulatory Science Requirements Consistency with OMB's Guidelines for Ensuring and Maximizing Quality, Objectivity, Utility and Integrity of Information Disseminated by Federal Agencies (OMB Guidelines)
       Regulatory Science that Cannot Meet the Requirements of §§30.5 and 30.6
       How Should EPA Address a Circumstance in which EPA Has a Statutory Requirement to Make a Determination for Which Scientific Information Publicly Available Sufficient for Independent Validation Does Not Exist? (83 FR 18772)
       How Does What this Rule Would Require Compare to Requirements of Environmental Statutes? (Relates to Chapter 1)
Comments that seek to clarify how the definition of "pivotal regulatory science" is distinct from the separately defined, "regulatory science" are primarily contained in the chapter of this response to comments (RTC) concerning "pivotal regulatory science" (Chapter 6). 
The specific comments provided in this Chapter reflect the comment letters that provide arguments supported by detailed rationale, examples, and references/documentation. The Docket ID numbers for these commenters are cited throughout this Chapter. In addition, some general comment letters may also include comments related to comments included and responded to in this Chapter. These comments are summarized in Appendix B. Note that points and issues raised in these general comment letters may also be raised by the more detailed comments summarized in this Chapter. The responses provided in this Chapter adequately address these more general comments.

Note on Terminology
In the final rule, the EPA has replaced the term "regulatory science" with the term "science that serves as the basis for informing a significant regulatory action." The EPA made this change in response to comments that the term "regulatory science" may be confusing because it suggests either the term refers to a scientific discipline of regulatory decision-making, or that some science the EPA considers is inherently regulatory. Neither of these interpretations reflects the Agency's intent in defining this term. The EPA considers the breadth of scientific evidence in its rulemakings; while this scientific evidence informs policy decisions, the EPA's consideration of the science does not make it "regulatory science." To reflect this fact, in the final rule the EPA is changing the proposed term "regulatory science" to "science that serves as the basis for informing a significant regulatory action." While commenters on the 2018 proposed rule used the term "regulatory science," the EPA is responding to comments with the term "science that serves as the basis for informing a significant regulatory action" to reflect the contents of the final rule.
9.1 NPRM Regulatory Science (Section 30.2)
9.1.1 Ensure that the Regulatory Science is Publicly Available in a Manner Sufficient for Independent Validation (§30.1)
9.1.1.1 General Opposition to Regulatory Science Being Required to be Publicly Available
Comment: Commenters (0044, 0671, 0732, 1943, 2687, 2909, 4591, 4595, 4848, 4894, 5181, 6111, 6125, 6147, 6158, 6897, 6904, 6907, 6924, 6940, 8281-PH5, 8281-PH29, 8281-PH46, 8281-PH84) generally oppose regulatory science underlying its actions being required to be publicly available in a manner sufficient for independent validation. Commenters generally express that, it is unnecessary and would limit the ability to use scientific data that is necessary for the EPA to meet its mission. Commenter (6105) asserts that not all scientific findings that are important are publishable, nor do scientific journals generally have sufficient space to include all data. Example comments include:
 Commenter (0732) states that there is a need to consider ALL scientifically well-planned, documented and reported studies about potential adverse health effects of chemicals. The commenter asserts that this serves to increase the likelihood that any regulatory standards developed will be based on the full and complete "weight of evidence" of data and helps reduce the uncertainty inherent in those numbers. According to the commenter, the ability of the EPA to protect health is directly connected to the quality and quantity of studies that underpin its standards.
 Commenter (4595) Commenter (4595) asserts that the EPA's Proposal risks rejecting valid scientific evidence and fundamentally mischaracterizes the way science is conducted and made available for decision-making. According to the commenter, by limiting the science EPA can use in policies and regulations, EPA will ultimately constrain itself to smaller groups of studies and risk biased outcomes. The commenter asserts that, if the pool of research is smaller - which is what this proposed rule will lead to - that smaller pool utilized creates an inherent bias and thus the rulemaking process will yield distorted results.
 Commenter (4848) contends that it is imperative that EPA allow consideration of all available scientific data pertinent to a proposed environmental rule or regulation  -  including randomly controlled human health trials and other epidemiological studies.
 Commenter (6924) In general, when assessing the body of evidence, EPA should consider both individual study quality and the overall quality of the body of evidence which requires consideration of all the relevant scientific literature. To not do so could potentially bias the assessment, as stated by the National Academies of Sciences (NAS).
 Commenter (0671) states that EPA does not need publicly available information to allow independent validation. The commenter states that any researcher who wishes to validate a study is already free to perform her or his own study asking the same scientific question. According to the commenter, this sort of replication is already performed hundreds of times a year across many areas of environmental science and is far more meaningful than the narrow and misguided notion of validation the EPA is implicitly using.
 Commenters (1943, 6940) assert that EPA needs to have access to data generated from modern, sensitive, and novel models and methodologies to protect the American public. Commenter (1943) states that any attempt to limit the applicability of scientific data will stifle innovation and progress toward a cleaner, healthier future for all Americans. 
 Commenters (2687, 6147, 6158) express concern that this rule would greatly limit the data and models upon which EPA would rely when setting public health and environmental standards. Commenter (6158) also refers to the concerns expressed by Singla et al (2018) in their detailed comments to EPA submitted May 30, 2018. 
 Commenters (2909, 5181, 8281-PH5, 8281-PH29) express concern that the proposed rule would limit the scientific research available to EPA policy makers to an extent that would undermine EPA's mission to protect public health and the environment.
 Commenter (6904) states that, if EPA excludes studies because the data cannot be made public, people may be exposed to real harm. The commenter states that the result would be decisions affecting millions based on inadequate information that fails to include well-supported studies by expert scientists.
 Commenter (4894) states that the proposed rule is based on a faulty premise, namely, that only studies whose underlying data are publicly available, sufficient to support replication, should be considered by the EPA as it develops regulations governing clean air, clean water, and exposure to toxic substances and pesticides.
 Commenter (4591) states that the goals of open science are not advanced through this proposal, which does not provide incentives, funding, or infrastructure for increasing access to data, but simply allows the Agency to disregard important, ethical, well-designed and executed studies.
 Commenter (6907) is concerned that the proposed rule on transparency would reduce the number of valid scientific studies utilized by EPA in conducting risk assessments and therefore, jeopardize the safety of pesticides and impact rulemaking under Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA).
 Commenter (6125) asserts that, while it is not flatly untrue that the proposal "does not compel" EPA to make the information available, it is worse than disingenuous not to mention also that unless the information is publicly available it cannot be used as a basis for the rulemaking decision unless EPA grants an exemption.
 Commenter (8281-PH84) expresses concern that, under this proposal, studies without a guarantee of full public access will be considered unreliable and will play no role in assessing a chemical's health effects on human health. The commenter states that this ignores the many ways in which the scientific community, regulators and the public have traditionally determined the quality and relevance of study results.
Response: In this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. The EPA has removed the term "science that serves as the basis for informing a significant regulatory action" from 40 CFR 30.1, and will not require that science that serves as the basis for informing a significant regulatory action is publicly available in a manner sufficient for independent validation. Instead, in the final rule, 40 CFR 30.4 requires that the EPA clearly identify all science that serves as the basis for informing a significant regulatory action, and make all such science that serves as the basis for informing a significant regulatory action publicly available to the extent permitted by law. The EPA clarifies that this requirement is consistent with the EPA's existing practice of making science that serves as the basis for informing a significant regulatory action available in the public docket as part of the rulemaking.
Comment: Commenter (4831) references several NAS reports and states that these reports encourage EPA to consider all available science in the rule-making process and provide guidance about how the agency could be more transparent in describing how evidence is gathered and evaluated.
Response: Consistent with existing Agency guidelines, the EPA will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Comment: Commenters (4591, 6191) concur with the Association of American Medical Colleges, Association of American Universities, Association of Public and Land-grant Universities, and Council on Governmental Relations in their July 11, 2018 statement "Re: Docket Number EPA-HQ-OA-2018-0259-0025, Strengthening Transparency in Regulatory Science," which states that "[e]ven the strongest and most sincere supporters of the open science movement have recognized that there is value in research for which underlying data are not made publicly available and acknowledge an imperative to leverage all science to develop policies and regulations."
Response: EPA agrees with the commenter that there is value in research for which the underlying data and models cannot be made available. In consideration of this concern, in this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Comment: Commenter (6135) opposes any effort by EPA, under the guise of transparency, to disregard the findings of scientific studies assessing the health impacts of air pollution when making regulatory decisions. The commenter states that, since there is no way to provide de-identified data in the public domain that would allow for secondary reanalysis of both the exposure assessment and health analysis in studies assessing the health impacts of air pollution, the EPA should instead continue to focus on assessing results across multiple studies to evaluate the veracity and general applicability of scientific findings.
Response: The EPA disagrees that this rule will cause the Agency to disregard the findings of scientific studies assessing the health impacts of air pollution when making regulatory decisions. The EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
Comment: Commenter (6907) states that, while the proposed rule suggests that fraud is prevalent in studies where data are not published, this is not the case. The commenter states that, in many instances, these unpublished data protect the humans who are part of studies. The commenter states that the scientific process ensures that studies remain reproducible even when data cannot be made publicly available. 
Response: The EPA is committed to protecting the privacy of study participants and will ensure that the implementation of this rule is consistent with law; protects privacy, confidentiality, and confidential business information (CBI); and is sensitive to national security interests. The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety.
Comment: Commenter (6915) states that, going forward, many academic scientists whose research is relevant to EPA regulations may not conduct and report their studies in a way that satisfies the requirements of the proposed rule. The commenter asserts that the proposed rule's provisions would require significant additional resources and could impose unreasonable and impractical requirements beyond those included in current protocols. The commenter states that academic researchers may not be concerned about or even consider whether their studies would qualify for use in establishing EPA regulations, and/or may not have the resources to reshape their approach to maintaining data. The commenter notes that, for those researchers who do attempt to comply with the proposed rule's requirements, the extent and nature of the data that must be maintained and made publicly available is vague and unclear, making compliance virtually impossible.
Response: EPA disagrees with the commenter that this rule would impose a burden of unreasonable or impractical requirements on researchers or other members of the public. Instead, this is an internal rule of agency procedure and thus puts no burden on any entity other than EPA. This rule is consistent with advances by the scientific community, which has also embraced greater data transparency, as evidenced by data sharing and availability requirements for many high-impact journals and the emergence of organizations, such as the Center for Open Science, and international initiatives like Findable, Accessible, Interoperable, and Reusable (FAIR) data principles; Facilitate Open Science Training in European Research (FOSTER); and Guidelines for Transparency and Openness Promotion (TOP) in Journal Policies and Practices that incentivize greater transparency in research[,].
9.1.1.2 Generally Support Requirement to Make Regulatory Science Publicly Available
Comment: Commenters (0555, 0563) generally support regulatory science underlying its actions being required to be publicly available in a manner sufficient for independent validation The commenters generally express that often analyses underpinning regulations reflect policy preferences or are grounded on false information and that EPA's proposal to strengthen the transparency of its regulatory science would lead to better regulatory outcomes.
Commenter (0555) provides that regulatory impact analyses often hinge on assessments of risk that necessarily involve assumptions and judgments but often generate precise-sounding predictions that hide not only considerable uncertainty about the actual risk, but hidden policy judgements. The commenter states that too often scientific information presented to the public and decision-makers is infused with hidden policy preferences. The commenter asserts that EPA's proposal to strengthen the transparency of its regulatory science includes reasonable steps that could improve the evidential basis for its regulatory policies, and thus improve regulatory outcomes by targeting resources to where they can achieve the largest benefits. The commenter states that, as President Obama's science advisor observed, "open communication among scientists and engineers, and between these experts and the public, accelerates scientific and technological advancement, strengthens the economy, educates the Nation, and enhances democracy." 
Commenter (0563) does not agree with the claim that peer review alone is sufficient quality control. The commenter asserts that using scientific research that is non-transparent and non-replicable increases the likelihood that policy will be grounded on false information. The commenter does not agree with the argument that EPA should be able to use all available scientific research and that excluding some may cripple its ability to protect us.
Commenter (0563) states that studies may rely on funding from sources that restrict disclosure of underlying data, and the rule would prohibit the EPA's use of those. The commenter states that is precisely why the rule is needed. The commenter adds that the funders should change their policies; and argues that their refusal suggests they have something to hide.
Commenter (0844) asserts that many of the arguments opposing the EPA's transparency rules are surreal. The commenter did not explain this position.
Response: The EPA disagrees with the commenter that its analyses are often political or grounded in false information. The EPA performs objective, high-quality analyses using the best available science. Through this rule, and consistent with existing Agency practice, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information. The EPA will continue to use the following, established factors to assess the quality of studies used to develop significant regulatory actions and influential scientific information: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review.[,] When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence (consistent with Agency guidelines on hazard identification and dose-response assessment) to support a relationship between exposure and effect, the EPA will identify a subset of those studies for use in characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect. This will be based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both will be identified as pivotal science (see Section III.E of the final rule preamble).
EPA agrees that relying on measures beyond peer review can increase transparency. However, EPA notes that peer review is still a valuable process. As noted in EPA's Peer Review Handbook (4[th] Edition): "Peer review is conducted to ensure that activities are technically defensible, competently performed, properly documented and consistent with established quality criteria. Peer review is an in-depth assessment of the assumptions, calculations, extrapolations, alternate interpretations, methodology, acceptance criteria and conclusions pertaining to the scientific or technical work product, and of the documentation that supports them." Peer review does serve to increase the credibility of a work product. This is why the final rule at 40 CFR 30.6 requires the EPA to conduct independent peer review consistent with the requirements of the Office of Management and Budget (OMB) Final Information Quality Bulletin for Peer Review and the exemptions described therein, and authorizes the EPA to evaluate whether or not to conduct additional peer review of individual studies identified as pivotal science if the studies have already undergone peer review.
Finally, EPA disagrees with the commenter that this rule is needed to convince funders of scientific studies to change policies restricting data disclosure that are consistent with laws governing privacy and intellectual property. Furthermore, this is an internal rule of agency procedure and not intended to impose requirements or burdens upon the public.
9.1.1.3 Are Studies More Valid If Data Publicly Available?
Comment: Commenters (6133, 9227) assert that EPA provides no basis for its assumption that science or studies for which data are publicly available yield more valid or reliable results than the best available, peer-reviewed, independent, credible science, for which the underlying data are not publicly available. Commenter (6133) states that neither the Proposal nor docket contains any factual, scientific, technical, logical, or legal support for the suggestion that science and data that are "publicly available in a manner sufficient for independent validation" are necessary elements for the "validity," "reliability," or "transparency" of scientific information.
Commenter (9227) states that the EPA irrationally conflates scientific "validity" and "transparency" with data availability, incorrectly assuming that eliminating the use of studies without publicly available data will improve scientific validity and transparency. The commenter states that, in the preamble to the proposed rule, EPA states that the intent of the regulation is "to strengthen the transparency of EPA regulatory science." The commenter states that, later in the preamble, EPA states: "[e]nhancing the transparency and validity of the scientific information relied upon by EPA strengthens the integrity of EPA's regulatory actions and its obligation to ensure the Agency is not arbitrary in its conclusions." The commenter states that EPA then leaps to the unexplained conclusion that barring the use of studies without publicly available data will enhance transparency and validity. The commenter states that EPA's assumption that data availability (or "transparency" in the form of data availability) ensures the use of valid science or its equivalent to using the best available science is manifestly incorrect, and hence provides an irrational basis for the proposed rule. The commenter states that neither data availability in particular, nor transparency in general, is equivalent to or a guarantee of "validity" in scientific studies.
Commenter (6133) states that the Proposal arbitrarily fails to address, much less explain, why prior EPA regulatory actions that relied upon studies, data, or other information did not reflect the "best available science" or why they were otherwise unreliable, despite failing to meet the Proposal's standards.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to national ambient air quality standards (NAAQS), risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines (IQG) or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
9.1.1.4 The Proposal is not an Exclusionary Rule
Comment: Commenter (6179) states that some have mischaracterized the Proposal as an exclusionary rule that would bar EPA from considering studies where underlying information cannot be made public. The commenter states that a plain reading of the Proposal's specific regulatory language shows no such requirement. The commenter asserts that the Proposal states that: (1) EPA should generally release regulatory science where practicable; (2) EPA shall ensure certain information underlying studies cited in significant rulemaking is made publicly available; and, (3) the Administrator has authority to exempt studies cited in significant rulemakings containing certain information, including that which is private, sensitive, or confidential, from these requirements. The commenter states that these are reasonable steps to improve the evidentiary basis for EPA's regulatory policies and improve regulatory outcomes by targeting resources to where they can achieve the largest benefits. The commenter states that it would be unsound policy for EPA to summarily exclude science from regulatory consideration  -  and the Proposal does not do so. The commenter states that the proposal presumes proactive management of private, sensitive, or confidential information and only requires disclosure of dose response data and models in appropriate circumstances. The commenter states that, rather than rigidly excluding important studies, the Proposal provides a flexible, case-by-case exemption process. 
Commenter (6179) states that the Proposal is not, and does not need to be, an exclusionary rule in order to accomplish its underlying goal of informing public discourse on significant regulations. The commenter asserts that the rule can be implemented to protect private, sensitive, or confidential information while still seeking maximum possible transparency. According to the commenter, the presence of private, sensitive, or confidential information does not necessarily make disclosure of dose response data and models impossible.
Commenter (1302) states that many have objected that this transparency ruling would undermine the ability to collect relevant data. The commenter notes that the ruling specifically allows the Agency to make data or models available in a manner that is consistent with law, protects privacy, confidentiality, CBICBI, and is sensitive to national and homeland security. The commenter adds that the data made publicly available are only those required to enable "independent validation," that is that the data provided is sufficient "for the public to understand, assess, and replicate findings." According to the commenter, since such information does not (except perhaps in extraordinary circumstances) include the identity of subjects, it is difficult to imagine how confidentiality agreements would be violated by disclosing information regarding anonymous subjects. 
Commenter (0563) suggests that §30.5 assures that the Agency, in making data or models publicly available, would do so in a fashion that is consistent with law, protects privacy, confidentiality, CBICBI, and is sensitive to national and homeland security. The commenter adds that §30.9 allows the Administrator to grant exemptions on a case-by-case basis if compliance is impracticable. The commenter concludes that the rule would not prevent the EPA from using studies that involved confidential information, such as personal health data or corporate proprietary information. 
Response: The EPA agrees that this final rule is not an exclusionary rule. The final rule is limited to dose-response data and, as no studies are categorically excluded from consideration, the EPA will continue to rely on the full body of the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Consistent with the requirements of this rule, the EPA will identify a subset of those studies based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints that would inform a dose-response assessment, and will give greater consideration to pivotal science for which the underlying dose-response data are available.
Comment: Commenter (6376) suggests that EPA should establish guidelines to ensure transparency does not come at the cost of widespread exclusion due to inadequate protections for the variety of sensitive data included in different pertinent studies. The commenter states that scientific studies involving or including sensitive health data should not be at risk of exclusion over concerns related to personal information being exposed. 
Response: The EPA agrees that it is important not to categorically exclude any study because the data underlying a study at the stage required by this rulemaking may not be available for independent validation. Therefore, the EPA is not finalizing the primary proposal in the 2020 SNPRM that would have categorically required that for studies to be considered pivotal science the underlying data would need to be available for independent validation. However, given that transparency is an important aspect of EPA's regulatory actions and assessments, it should be an important consideration in how the Agency considers pivotal science. As described in 40 CFR 30.5 of the final rule, the EPA will rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies --  based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements or quantitative analyses of an EPA significant regulatory action or influential scientific information will be identified as pivotal science.
Further, the EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
9.1.1.5 EPA Would Exclude Studies and Other Valid Data from Regulatory Decision-Making
9.1.1.5.1 EPA Should Identify Studies that Would Be Barred
Comment: Commenter (6125) expresses concern that the proposal says nothing about what studies it will exclude, aside from those identified but not mentioned by name in footnote 3 of the preamble. The commenter requests that EPA should identify the key studies that its proposal would bar that have been used to support regulations in the past and evaluate the impact of their loss on those regulations and thereby on human health and the environment.
Response: Under this final rule, the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
9.1.1.5.2 Best Available Science
Comment: Commenters (1012, 1433, 1796, 1945, 1973, 1994, 2170, 2180, 2182, 2190, 2249, 2335, 2425, 2746, 2911, 3135, 4595, 4827, 4828, 4845, 4877, 5115, 5131, 5176, 5178, 5180, 5981, 6116, 6122, 6125, 6128, 6133, 6134, 6137, 6153, 6157, 6181, 6194, 6195, 6355, 6368, 6373, 6873, 6882, 6895, 6906, 6909, 6932, 6942, 8272, 8282-PH10, 8281-PH38, 9225, 9227) oppose this proposed rule since it could undermine EPA's ability to use studies that represent high-quality studies/best-available science and thereby potentially leading to weakening protections for public health and the environment. Comments include:
 Commenter (6373) asserts that EPA must consider the "best available science" to guide policies and practices and should avoid eliminating existing data and standards until such time as new and better information is available, regardless of the source of the existing data. 
 Commenter (6932) states that risk assessment processes can be improved and recommendations have been issued in reports like the 2014 NAS report, but limiting the types of studies that can be used is not the way to improve them. The commenter notes that the best available science means that EPA should use all scientific studies  -  especially independent science from peer-reviewed journals-- available in order to inform its decisions.
 Commenter (6125) What justification is there to impose such an absolute ban on EPA studies? Would the ban extend to using research developed and relied on by other agencies?
 Commenter (6116) states that EPA does not explain how its proposed approach will ensure that it relies on the "best available science" which is an oft-repeated standard in various environmental laws. The commenter states that it is possible that EPA's proposed regulation will actually result in a failure to rely on the "best available science" as EPA will arbitrarily disqualify important scientific studies if the underlying data cannot be disclosed to the public and/or cannot be independently validated. 
 Commenter (6137) states that EPA has provided no scientific justification for ignoring an entire class of health science simply because the underlying data has not been disclosed. The commenter adds that EPA has not demonstrated that it is consistent with scientific principles to categorically exclude peer-reviewed scientific information from all consideration in a rulemaking, as EPA proposes to do. The commenter states that, if EPA has any doubts or concerns regarding the merits of a particular study, it must address those doubts or concerns for that particular study in the context of a given rulemaking, where agency staff, internal scientific experts, scientific advisory committees, or commenters contend that study is relevant. The commenter notes that, whether underlying data on which a study relies is made public or not simply has no bearing on whether a scientific study is good science, is accurate, is reliable, and is relevant to a scientific question (such as the health effects of air pollution). 
 Commenter (2911) states that the only criterion that should be used in deciding whether to use the information from a scientific study is whether the study is valid or not. Commenters (6873, 6906) expand on this idea by noting that the proposal, as written, could provide the EPA Administrator and offices with an incentive to exclude studies with useful information either inadvertently or intentionally because underlying dose response information and information about models are not available. Commenter (6906) suggests that EPA close this potential loophole by amending proposed 40 CFR §30.4 to include instructions that this rule shall not be used to exclude a study from EPA's review and consideration of conclusions drawn therein, where EPA determines the study is compatible with the "best available science." Commenter (6873) recommends that EPA close this loophole by any number of minor adjustments -- for instance, holding public hearings for each study to be questioned by the EPA might allow scientific findings to be vetted without ruling them out entirely.
 Commenter (1012) asserts that EPA must use the best available science as the foundation of its regulatory actions. Commenter (1945) states that regulatory decisions should be based on the best science, and that there is often much to be gained by allowing data to be scrutinized. 
 Commenter (1012) asserts that oftentimes the best available science (or the only available science) may include data of a sensitive nature like personal health information or CBI. Commenter (5131) states that peer-reviewed studies assessing the effects of pollution exposure on humans, based on actual personal health data, which by law must be kept confidential, are among the best science available to the Agency. 
 Commenter (3135) states that, for years, both Congress and successive administrations have required the EPA to use the best science for its decisions. The commenter states that it is a major departure for this proposal to direct EPA scientists to exclude key studies merely because they cannot meet the proposed disclosure requirement.
 Commenter (2170) states that the best available science can include research vetted through means other than the publishing of all data used. The commenter states that the protection of human health and the environment, as well as the legal, ethical, and moral prohibitions on publishing certain kinds of data, require that the best available science be inclusive of such research. The commenter states that it is unclear from a reading of the Proposed Rule whether scientific research not relying on published data sources would be (1) required to publish the data before the research could be used, even where publishing would be in clear violation of legal, ethical and/or moral standards, (2) simply ignored by EPA as if it were nonexistent, or (3) some combination of 1 and 2. The commenter states that the practical effect would be the same: a whole segment of research critical to the protection of human health, representing the best available science, would be eliminated from regulatory use.
 Commenter (4595) asserts that it has been recognized since 2002 by the D.C. Circuit Court that it would be "impractical and unnecessary" for the EPA to require that all underlying data be made public. The commenter notes that the court recognized the importance of EPA to have access to as much scientific information as possible to protect public health and the environment. The commenter adds that there remain cases when the transparent and proper use of the best available science requires that some of the underlying data cannot be made publicly available for legitimate reasons.
Response: The EPA disagrees with the commenters because this final rule does not require the EPA to categorically exclude any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Further, this rulemaking is not intended to modify the Agency's interpretations of "best available science." The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. The EPA will then identify and consider "pivotal science in accordance with the provisions of this rule," unless the implementation of the rule conflicts with statutory requirements and associated implementing regulations.
Some commenters interpreted 40 CFR 30.4 as requiring the EPA to make publicly available all science relied upon in final agency actions. The EPA clarifies that in the final rule, the requirements of 40 CFR 30.4 are consistent with the EPA's existing practice of making science that serves as the basis for informing a significant regulatory action available in the public docket as part of the rulemaking.
9.1.1.5.3 Epidemiologic Studies
Comment: Commenters (1012, 2180, 4844, 5981, 6103, 6114, 6116, 6119, 6130, 6131, 6170, 6450, 6884, 6915, 6932, 8272) express concern that this proposal would likely prevent the EPA from using any studies that could not be independently reproduced and would therefore disallow many extremely beneficial types of studies, such as those examining the results of natural disasters or industrial/occupational accidents/exposures. The commenters assert that such incidents could not be replicated from practical, safety, or ethical points of view, yet the information they provide on real world exposure and dose-response interactions is invaluable. These studies include the following:
 Exposure to high levels of smoke from wildfires or to high levels of chemicals resulting from industrial accidents or exposures. (6114, 6116, 6131) 
 Unique events such as the 2010 Deepwater Horizon oil spill. (1012, 4844, 6103, 6450) 
 The Flint water crisis and the 1979 Three Mile Island accident. (6450)
 Health impacts resulting from asbestos exposure, exposure to Dichlorodiphenyltrichloroethane (DDT), and birth defects caused by thalidomide. (6450)
 The effects of radiation from atomic bomb survivors. (6915)
 Data on wildlife toxicity from the Exxon Valdez oil spill. (6915)
 Data on the human health impacts of the September 11, 2001 World Trade Center disaster. (6915, 8913)
 Impacts of neurotoxicants from mercury contaminated Minamata Bay. (8272) 
 The BP Transworld oil spill, the Katrina pollutant disaster, the Hurricane Harvey-related toxic chemical discharges, and the Church Rock uranium mill spill. (8913)
Commenter (6915) states that, for older epidemiology data, such as data from studies on occupational exposures to workers in factories before the advent of strict Institutional Review Board (IRB) requirements, raw data are seldom if ever still available. The commenter states that such data, including high quality data generated by major corporations in conjunction with academic institutions, would not be available to EPA under the proposed rule. The commenter states that, effectively, the proposed rule would restrict the epidemiology data available for use by EPA, even where the weight of the evidence clearly supports a finding of causality and risk.
Response: The EPA disagrees with the commenters that this rule would likely prevent the EPA from using any studies that could not be independently reproduced and would therefore disallow many extremely beneficial types of studies, such as those examining the results of natural disasters or industrial/occupational accidents/exposures. In this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Further, the rule does not require EPA or any other entity to reproduce or conduct independent validation of pivotal science used in significant regulatory actions and influential scientific information.
Comment: Commenters (2171, 2336, 4591, 4595, 5019, 5115, 5165, 6133, 6916, 6919, 8275, 8281-PH16, 8281-PH26, 8281-PH38) express concern that EPA's Proposal would exclude, including published supporting studies for the NAAQS, that the Agency uses to make informed policy decisions regarding major public health and environmental laws. 
Commenters (1311, 1883, 1973, 1995, 2002, 2064, 2107, 2151, 2171, 2180, 2190, 2241, 2336, 4410, 4591, 4595, 4828, 5019, 5115, 5165, 5981, 5186, 6117, 6119, 6120, 6125, 6130, 6132, 6133, 6137, 6170, 6181, 6188, 6373, 6696, 6868, 6872, 6882, 6884, 6895, 6915, 6916, 6919, 6920, 6922, 6943, 6925, 6932, 6941, 8272, 8275, 8281-PH16, 8281-PH26, 8281-PH38) also express concern that EPA is proposing to establish a policy that could preclude it from using many of the best available peer-reviewed and scrutinized longitudinal medical and epidemiological and other studies from consideration in regulatory actions because these studies' underlying data includes confidential information. Commenter (6920) states that, by disqualifying high quality longitudinal epidemiological and clinical studies - often the most direct and relevant evidence of chemical impacts on humans - the proposed rule would diminish not strengthen the science underlying regulations. Commenter (4848) contends that longitudinal medical and epidemiological studies are often conducted over years, if not decades and is concerned that, under the Proposed Rule, they would be eliminated from consideration unless all underlying data is made public. Commenter (4595) asserts that requiring all raw data to be made publicly available before a study can be utilized in EPA decision-making will cut off EPA from foundational research that has informed EPA's work since the inception of the agency and may violate Federal laws and directives already in place. The commenters generally indicate that epidemiological studies are essential to effective environmental regulation. Comments include:
 Commenters (2171, 4410) state that laboratory work, toxicological research, and epidemiological studies are complementary, and each facet is necessary when it comes to understanding and quantifying the effects of a pollutant on human health. The commenters state that eliminating evidence from one of these three essential disciplines threatens the scientific basis for regulatory decisions and actions. 
 Commenter (1973) thinks that the proposed rule would deprive policy makers of the real-world epidemiologic evidence, based on real exposures of real people, that has been and will continue to be vital for future revisions of the NAAQS, drinking water standard, pesticide standards, and other health based standards based on a false understanding of transparency and replication in science. 
 Commenter (4828) notes that many of EPA's drinking water standards rely on epidemiological studies. The commenter states that often these studies last decades and follow hundreds, if not thousands, of patients, collecting confidential health data as well as other personal data, like the people's addresses, ages, and genders. Commenters (4828, 6872) add that, for most of these studies, the underlying data cannot be made public  -  even in redacted form  -  without sacrificing the participants' privacy or jeopardizing data confidentiality. The commenter states that studies like these established the original limits on lead, and this research continues to be essential today. Commenter (6872) states that, thus, the proposed rule could prevent EPA from considering some of the best, most up-to-date research, including some of the largest collections of epidemiological data developed over long periods of time, in assessing toxic pollutants in drinking water and the air and/or compromise data confidentiality.
Commenter (2064) states that, through peer-reviewed and validated research, paraquat is very strongly associated with an increased risk of Parkinson's disease. The commenter states that much of the research relies on large, population-based studies, which is the type of research most impacted by this rule. The commenter states that, without the ability to consider this research, the EPA will miss vital information needed to protect public health, thus placing Americans at risk. Commenter expresses concern that many of the most important studies involving environmental exposures and Parkinson's disease would be excluded from future determinations for one or more of the following reasons:
 The authors did not previously obtain the needed consents to release the data and would thus be subject to criminal penalty if they did so.
 The data cannot be deidentified appropriately while complying with federal privacy protection laws. 
 The cost to deidentify would be very high and many research institutions do not have people on staff who can perform the work needed to do so.
Commenter (6884) asserts that, in proposing this rule, EPA is essentially discounting the scientific expertise, knowledge, experience, and integrity of tens of thousands of researchers around the country that engage in epidemiological research. The commenter states that the "transparency" that is the basis for this rule would virtually undermine any studies that look at real world environmental health impacts to the people we work with every day.
Commenter (2151) states that, while environmental regulation will never be perfect and the goal should not be over-regulation, the potential costs to public health of any rule which would disallow long-term epidemiological studies is not an acceptable risk. The commenter states that there is no product or corporate bottom line so valuable that could legitimize excluding medical studies relating to a child's developing brain or the sickness of any human's body. The commenter states that no particular scientific method or study should be sacrosanct, but whatever methodology the EPA adopts it should be one in which all of the available science is considered in decision-making. 
Commenter (5118) believes epidemiological reports are valuable but more critical initial review is needed. The commenter states that there is no way to verify the procedures or results presented and the EPA reviewers are not identified.
Response: In this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
9.1.1.5.4 Reliance on Animal Toxicity Studies Over Epidemiological Studies
Comment: Commenters (1973, 2171, 4410, 6915, 6941, 8281-PH63) express concern that the proposed rule would exclude many human studies and, as a result, force EPA to increase its reliance on information from less relevant animal toxicity studies. Comments include:
 Commenter (1973) states that the major scientific issue challenging EPA's regulations is not the use of human studies that have been replicated multiple times, it is that EPA has to base many of its regulations on extrapolation from animal studies because human studies, particularly at relevant doses, have not been available. The commenter states that this proposal, by announcing that human studies, when done, will likely be ignored, will only exacerbate this problem. 
 Commenters (2171, 4410, 8281-PH63) assert the proposed rule would put regulators tasked with protecting human health in an impossible situation of relying primarily on animal models or in-vivo models that cannot be directly extrapolated to human dose-response estimates.
 Commenter (6915) notes that, in general, and specifically in EPA's 2005 Guidelines for Carcinogen Risk Assessment, human (i.e., epidemiology) data are preferred to animal data as the basis for risk assessment toxicity factors when they are of sufficient quality and are amenable to dose response modeling. The commenter states that this is because animal data always carry inherent uncertainties in regard to their relevance to humans. The commenter states that epidemiology data collected over the last 40 years, however, have been generated under the auspices of IRBs working to protect the patient or participant information obtained by academic institutions, government entities, hospitals, and other organizations, and thus disclosure of that data would be difficult, if not impossible. 
 Commenter (6941) asserts that the proposed rule could force EPA to exclude from its consideration epidemiologic studies that use confidential health data. The commenter notes that such studies are frequently the most relevant to protecting public health as they assess outcomes in humans following real world exposures. The commenter states that, without access to the information provided by such studies, EPA may instead increase its reliance on information from less relevant animal toxicity studies.
Response: The EPA disagrees with the commenters that this rule will necessarily lead the EPA to disregard human subject research in favor of animal studies. In this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. 
The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. See 40 CFR 30.5 for a description of the factors the EPA will consider when determining the consideration to afford to pivotal science when the underlying dose-response data are not available for independent validation. The EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
Comment: Commenter (6446) states that the proposed regulation would preclude EPA from considering a large number of important, peer-reviewed animal studies  --  studies that should serve as a foundation for regulatory action -- merely because of the unavailability of the raw data underlying the studies or the time and expense that would be required to obtain the raw data and provide it to the public. The commenter adds that a large amount of toxicological literature is based on animal studies that has accrued over the past 60 years that underlies the establishment of advisory levels and standards for hundreds of chemicals. The commenter notes that, in the scientific studies relied on for establishing these levels, data are generally provided in aggregated form due to journal page count limitations. According to the commenter, certain raw study data are available at an individual animal level (e.g., pathology data) online and from animal study reports performed by the National Toxicology Program (NTP), but such data are not included in study reports published by academic and industry researchers in peer-reviewed journals.
Commenter (6446) contends that a requirement to obtain an ad hoc exception to the proposed rule's requirements for large numbers of animal studies would be wholly unworkable. 
Response: The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. See 40 CFR 30.5 for a description of the factors the EPA will consider when determining the consideration to afford to pivotal science when the underlying dose-response data are not available for independent validation. The EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
The EPA disagrees that the process of the Administrator granting exemptions to the rule per 40 CFR 30.7 will be unworkable. The Agency may incur administrative costs to perform analyses and evaluations to support activities such as exemption decisions made by the Administrator, and documenting these or other decisions made pursuant to the requirements of the final rule. The Agency anticipates that the incremental costs for these activities will be small relative to current administrative costs for developing significant regulatory actions or influential scientific information.
9.1.1.5.5 Limitation of Use of Studies in Future Rulemakings
Comment: Commenters (0559, 6103, 6111, 6125, 6188, 6903, 6934, 8276) express concern that the proposed rule would shrink the universe of scientific information EPA can rely upon in support of rules in the future. Commenters provide the following examples:
 Commenter (6103) states that, in the upcoming review of soot standards, the proposed Transparency rule makes it unlikely that the EPA would be able to consider much of the scientific research that links soot pollution to adverse health effects, as it relies on medical records. The commenter adds that EPA will be forced to make decisions based on less information, making it more difficult for the agency to make science-based decisions that protect public health, as its mission requires. 
 Commenter (6111) states that late in 2017 EPA issued a draft report identifying potential approaches to deriving a maximum contaminant level goal for perchlorate. The commenter states that, to develop these approaches, EPA focused on five epidemiological studies. The commenter provides that all five studies relied on confidential patient data and all five studies were carried out in Europe, where scientists may be subject to different data confidentiality requirements than in the United States. The commenter states that the proposed rule would also prohibit EPA from continuing to rely on Needleman's critically important study in future reconsiderations of the lead NAAQS and revisions to the Lead and Copper Rule. The commenter states that, as a result, the proposed rule risks undermining the scientific basis for these EPA actions.
 Commenter (6125) states that EPA must propose an updated rule on lead in soil, dust and paint this year and EPA will propose an updated lead and copper rule for drinking water by 2020. The commenter states that, because many of the foundational lead studies analyzed children with higher exposure and blood lead levels than are commonly seen today, it would be unethical to run the same tests today, exposing children to levels of lead that we know are unhealthy. The commenter notes that, due to numerous factors related to their age, it may well not be possible to recover the original data. The commenter states that, if use of those older studies were successfully challenged because they do not meet the data requirement policy, they could not be used. The commenter asserts that their exclusion would weaken the scientific support for EPA regulations that have clearly proved their worth in the dramatic declines in blood lead levels in American children.
 Commenter (6125) suggests that the standards for arsenic and nitrate standards will most likely be affected by the proposed rule. The commenter states that both drinking water standards rely on epidemiological data including confidential patient information, as well as on older studies. The commenter adds that data from such studies may have been discarded or lost or be in an unreadable form. The commenter contends that none of these studies could realistically or ethically be reproduced since they derive from a unique cohort, and it would be unethical to expose people to ingested pollutants.
 Commenter (6903) states that one study published in 2017 indicates that particulate matter (PM2.5) and ozone exposure at levels below current NAAQS is associated with increased mortality. The commenter notes that the Proposed Rule would make it near impossible for the Agency to consider the study. 
 Commenter (6934) expresses that the rule would eliminate the use of studies like the one they hope to one day attain that rely on confidential health guidelines, discredit already peer-reviewed science, and make it impossible for citizens within their group and the countless other groups out there like theirs and individuals to gather preliminary health information for fear that this confidential information might become public once it is accepted into your review system and most of all create a double standard that allows polluting companies and industries to keep their data confidential.
Response: The EPA disagrees with the commenters that this rule will shrink the universe science the EPA can rely upon in significant regulatory actions or influential scientific information. In this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. 
The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. See 40 CFR 30.5 for a description of the factors the EPA will consider when determining the consideration to afford to pivotal science when the underlying dose-response data are not available for independent validation. The EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
9.1.1.5.6 Important Studies that Could Be Excluded from Regulatory Decision-Making
Cancer Classifications and Reference Values
Comment: Commenter (6125) states that, if the proposed rule were to apply retroactively with few or no exceptions/exemptions, most of the EPA cancer classifications and reference values would be vacated, aside from those values that are based upon accessible National Institute for Occupational Safety and Health (NIOSH) epidemiology data, National Toxicology Program (NTP) bioassay data or pesticide-related CBI data, if an exemption were made for the latter. The commenter provides a detailed analysis which focuses on two chemical categories for which a cancer classification has been assigned and/or a non-cancer Reference Dose (RfD) and/or Reference Concentration (RfC) has/have been calculated. The commenter provides the following observations/conclusions:
      1. Office of Pesticide Program (OPP) "Known" carcinogens all were assessed in collaboration with Office of Research and Development (ORD)-National Center for Environmental Assessment (NCEA) as those substances also have non-pesticidal uses and/or are an issue for other program offices. So access to their raw data would be a problem for OPP as well as everyone else, since they were not generated by registrants in response to FIFRA regulatory requirements.
      2. OPP has the raw data for chemicals on the 2013 and the RfD/RfC lists, but, of course, they cannot share them with the public because of FIFRA section 3(c)(1)(F) which provides for both "exclusive-use" and compensation rights for data submitted to EPA to support registration actions. In other words, they are CBI and one can be fined and/or jailed for releasing them without approval by their rightful owner. 
      3. OPP's regulatory decisions rarely rise to the $100 million cost threshold and most are not considered "rules" in the traditional sense. And, of those that might meet the dollar threshold, most of them are not chemical-specific. Rules such as the Worker Protection Standard or the Container rule may have met that threshold. Actions on individual requests for approval or revocation of uses do not qualify. On rare occasions, the cancellation/revocation of the entire spectrum of uses could meet the threshold if the number of uses was large enough. The 2015 proposed revocation of all of the chlorpyrifos tolerances (~80) might qualify by virtue of numbers of uses although EPA concluded that "revoking all tolerances for chlorpyrifos will not have a significant economic impact on a substantial number of small entities," so maybe it didn't. 
      4. Under the conditions of a final rule in which CBI data were exempted, OPP might be able to go forward with decision-making for chemicals based upon those data alone but would be hard-pressed to conduct scientifically-sound risk assessments for those chemicals for which the registration package is mixed. Even assessments for many of the conventional chemicals often are informed by studies from the open literature that were not funded by registrants and declared CBI. Important details in their toxicity or exposure profiles might have to be discarded leading to an underestimation of the actual risk.
      5. The Integrated Risk Information System (IRIS) assessments, for the most part, have significant age on them (late 1980's into the 1990's). This indicates that the studies used to generate those assessments are even older than that. Access to their raw data would be very much a challenge, unless the studies were conducted by NIOSH (epidemiology), the National Toxicology Program (animal studies) or the EPA, itself (human volunteer laboratory studies.
      6. If the proposed rule were to apply to both human and animal studies, then everything on IRIS could become compromised and many of the pesticides may be at risk, too, depending upon the nature of any exemption for them and the degree to which they depended upon non-CBI data. 
      7. If a chemical exhibits suggestive to known human carcinogenic potential, and the cancer risk is estimated quantitatively, then this endpoint usually drives the risk assessment and the human data and/or the long-term animal bioassays become the key studies. Data from non-government-conducted epidemiology studies are likely difficult or impossible to acquire and share. Unless the animal data are CBI data from manufacturers or from NTP bioassays, gaining access to them also is likely to be very difficult. Most all of the cancer bioassay data on pesticides are from the manufacturers. Some number of the key studies for the non-pesticide IRIS chemicals are NTP bioassays. In these two cases, EPA will have access to the raw data but will be able to share only the NTP information.
      8. For chemicals for which cancer is not the driving endpoint, the RfDs and RfCs become the important factors in quantitative assessments. Again, for conventional pesticides and biopesticides, most of the key data are from the manufacturers. In the case of the antimicrobials, some submitted studies may come from alternative non-CBI sources, presenting a challenge to access the raw data for them. For the IRIS chemicals, a small number are calculated from NTP data. Most key studies are from the peer-reviewed literature, primarily academic research. Given the age of most of the IRIS assessments, these data would be difficult, if not impossible, to acquire. 
      9. If the rule means to define reproducibility/replication as applying to the data/results of the key studies used rather than EPA's assessment of them, then the pesticide and commodity chemical manufacturers might have to submit TWO of every key study used in developing a regulatory decision, each being conducted in accordance with FIFRA Part 158 data requirements and data call-ins or Toxic Substances Control Act (TSCA) Section 4 or 5 test rules.
      10.An inability to acquire and share the raw data for the substances in both of these case examples would seriously comprise EPA's risk assessments and thwart its mission to protect public health.
Commenter (6920) notes that IRIS uses a rational weight-of-evidence method to assess available research and to provide access to a comprehensive source of toxicity data. The commenter adds that it increases the capacity to evaluate chemical hazards, and to quantify risk magnitudes and uncertainties, but it does not tell us how to manage risks. The commenter contends that the proposed exclusion of research lacking all underlying data from IRIS would be an unnecessary waste of information. Commenter (6133) notes that IRIS the Proposal would undermine decades of expert work to advance successful data evaluation methods described in the systematic review approach now underway in the EPA IRIS program.
Response: There are several important clarifications in response to the commenters' concerns. First, the final rule applies prospectively to significant regulatory actions and influential scientific information, not retrospectively. Second, this rule only applies to significant regulatory actions and influential scientific information. Many of the actions performed by the EPA Office of Pesticide Programs (OPP) are not significant regulatory actions, and therefore would not be subject to this rule. Finally, in this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. The EPA acknowledges that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. See 40 CFR 30.5 for a description of the factors the EPA will consider when determining the consideration to afford to pivotal science when the underlying dose-response data are not available for independent validation.
Six Cities Study and American Cancer Society Cancer Prevention Study Would be Excluded in Regulatory Decision-Making
Comment: Commenters (0567, 2429, 2746, 2787, 4591, 4894, 5012, 5165, 5176, 5180, 6103, 6111, 6125, 6133, 6351, 6915, 6916, 6925, 8281-PH38, 9227) express concern that the proposed rule could exclude studies that utilize confidential data, such as the Harvard Six Cities ("Six Cities") (1993) and American Cancer Society (ACS) Cancer Prevention Study II (1995) studies, each of which tracked the medical histories of thousands of people and which are the two major epidemiological studies referenced in the particulate matter (PM) NAAQS. 
Commenters (2787, 6925) state that the "Six Cities" study, which followed more than 8,000 participants for nearly twenty years was key in establishing a link between chronic air pollution exposure and increased mortality. Commenter (6925) provides that this epidemiological study relied on health data that is protected under confidentiality agreements, so these data cannot be publicly released without violating privacy. 
Commenter (9227) states that the original findings of the Harvard Six Cities Study have been validated many times over, and they have been used to inform countless EPA rule makings that address particulate matter pollution. The commenter adds, however, that the Proposal would appear to preclude EPA from using them because -- while the Study has been reanalyzed and reproduced -- the underlying data is not publicly available because of patient confidentiality protections bound by individual contractual agreements between the scientists and the research participants and by the Health Insurance Portability and Accountability Act (HIPPA). The commenter states that these reasons are unrelated to the validity, integrity or quality of the Harvard Six Cities Study. The commenter asserts that the provisions of the proposed rule that would preclude the use of the study are inconsistent with OMB's data quality guidelines, which specifically point to the Harvard Six Cities Study as an example of how data may be validated or corroborated without public release of the underlying raw data.
Commenters (5165, 6925) suggest that EPA publicly confirm that it would continue to use existing literature such as the Six Cities Study in future rulemakings, should the proposed rule be enacted.
Commenters (2429, 2787, 6125, 6915) express concern that the proposed rule could exclude studies that utilize confidential data such as the ACS Cancer Prevention Study. Commenter (2429) states that, while results from the study have been subjected to independent reanalysis and sensitivity analyses, public access of these data will be difficult given that they were collected decades ago with assurances of confidentiality for research participants. 
Commenter (4894) states that there are valid, peer reviewed studies that should be included in EPA's regulatory work, even though their underlying data sets cannot be released to the public, including the ACS's Cancer Prevention Study II. Commenter (2907) states that the ACS Cancer Prevention Study II tracked air pollution exposure and personal medical histories of nearly 670,000 people for more than two decades to understand the exact risk of air pollution on death and the results of this study underpin critical components of decision making with respect to national air quality standards.
Commenters (2787, 4827, 4894, 6925, 8272) state that the results of the Harvard Six Cities Study and the ACS Cancer Prevention Study II studies have stood up to extensive subsequent analysis, including the "Reanalysis of the Harvard Six Cities Study and the ACS Study of Particulate Air Pollution and Mortality,"  which highlights the strength of such research. Commenter (8272) states that since publication of these two landmark studies, dozens of additional peer-reviewed studies, as well as follow-up research involving the Six Cities and ACS cohorts, have replicated and expanded upon their findings and provided insights into specific effects of particulate pollution on certain groups.,
Commenter (6915) states that under the proposed rule, EPA could ignore these two foundational studies and other peer-reviewed studies built upon them in setting health-based air quality standards for particulate matter and other pollutants. The commenter states that the effect could be devastating and deadly, as EPA estimates that reductions in ambient particulate matter under the 1990 Clean Air Act (CAA) Amendments will prevent 230,000 adult deaths by 2020.
Commenter (6925) states that the fact that patient personal information is held confidential in no way undermines or casts doubt on its findings. The commenter states that this study is one of the most critical studies that has been used to establish air pollution standards. Commenter (4894) states that the studies remain valuable today., 
Response: The EPA clarifies that in this final rule the Agency is neither specifically nor categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. This includes the Harvard Six Cities ("Six Cities") (1993) and ACS Cancer Prevention Study II (1995) studies. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of personally identifiable information (PII), CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. 
The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. See 40 CFR 30.5 for a description of the factors the EPA will consider when determining the consideration to afford to pivotal science when the underlying dose-response data are not available for independent validation. The EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
Other Important Studies that Would be Excluded from Regulatory Decision-Making
Comment: Commenters (2336, 4827, 6920) state that, if adopted, the proposed rule will deprive EPA policymakers from real-world evidence and studies that are vital to EPA's review of the NAAQS into the future, such as "Air Pollution and Mortality in the Medicare Population." Commenter (6920) states that the NAAQS review process is a model of scientific transparency. The commenter notes that its reviews are renowned for their high quality. In regard to the studies on which they are based, the commenter states that the costs of publication as well as requirements for the privacy of study subjects have prohibited making all the data obtained publicly available. The commenter strongly disagrees that this diminishes the value of such epidemiology and toxicology studies to regulatory science. The commenter contends that the exclusion of such studies would significantly reduce the amount of scientific data available to establish appropriate standards to protect public health and the environment.
Similarly, commenter (6125) states that one of the most recent fine particle mortality studies created a cohort of 60 million subjects from the Medicare database. The commenter states that this study found even larger effects of fine particles at levels well below EPA's current standards. The commenter states that the Medicare database is available to any research group that pays a fee and can guarantee confidentiality of the personal data. The commenter states that, because these data are not fully available to the public, under the proposed rule, the Administrator could choose to bar consideration of even this powerful study.
Commenter (2336) provides that the history of the NAAQS standards for PM2.5, and the dose-response models which underpin them, demonstrates that a blanket public disclosure rule such as the Proposal would have been counterproductive if it had been in effect during the 1990s when the new NAAQS standard for PM2.5 was being evaluated and the underlying dose response models were being vetted. The commenter states that the rule proposed would have required that the health-outcomes dataset be made public, or the results would have been thrown out. The commenter adds that, had the rule proposed been in effect in the 1990s, it would have created a Hobson's choice for the EPA. The commenter states that it would have had to discard a pair of robust groundbreaking studies demonstrating that there is an intense dose-response function between PM2.5 and mortality rates, or it would have had to violate the privacy interests of the studies' participants and jeopardize the prospect of performing similar studies in the future. 
Commenter (8272) references two recent studies and states that they add to knowledge on susceptible populations., The commenter states that both of these studies involve data that can be obtained by investigators who apply to use it and are approved by the relevant IRB or Privacy Board, but their data sets are not publicly available. 
Response: The EPA clarifies that in this final rule the Agency is neither specifically nor categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. This includes the "Air Pollution and Mortality in the Medicare Population" study.  Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. 
The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. See 40 CFR 30.5 for a description of the factors the EPA will consider when determining the consideration to afford to pivotal science when the underlying dose-response data are not available for independent validation. The EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
Comment: Commenter (2429) states that a recent example of an important study that could be excluded under the Proposal is the Multi-Ethnic Study of Atherosclerosis and Air Pollution (MESA-Air), a 10-year prospective cohort study of more than 6,000 participants. Using these data, the commenter notes that the MESA-Air researchers have produced over 100 scientific publications examining the cardiovascular impacts of long-term exposure to air pollution, all of which could be excluded from consideration in the regulatory process.
Commenter (2977) expresses concern that the proposed rule would restrict the scientific evidence that can inform essential environmental regulations; regulations that have reduced emissions, protected our drinking water, reduced lead exposure, and saved lives. Commenter states that the studies listed below, which EPA has used to help establish acceptable levels of exposure to pollutants, would not be considered if rigid standards like the ones proposed in this transparency rule were in effect.
Commenter (2977) provided a sample list of population-based studies (included in an Appendix to their comment letter) from the National Institute of Environmental Health Sciences/Environmental Protection Agency's Children's Environmental Health and Disease Prevention Research Centers that were considered or included in EPA regulations:
 Berhane K, Zhang Y, Linn WS, et al. The Effect of Ambient Air Pollution on Exhaled Nitric Oxide in the Children's Health Study. The European respiratory journal. 2011;37(5):1029-1036. doi:10.1183/09031936.00081410. 
 Bradman A, Salvatore AL, Boeniger M, et al. Community-Based Intervention to Reduce Pesticide Exposure to Farmworkers and Potential Take-Home Exposure to their Families. Journal of exposure science & environmental epidemiology. 2009;19(1):79-89. doi:10.1038/jes.2008.18. 
 Breton CV, Salam MT, Vora H, Gauderman WJ, Gilliland FD. Genetic Variation in the Glutathione Synthesis Pathway, Air Pollution, and Children's Lung Function Growth. American Journal of Respiratory and Critical Care Medicine. 2011;183(2):243-248. doi:10.1164/rccm.201006-0849OC. 
 Coronado GD, Holte SE, Vigoren EM, et al. Do workplace and home protective practices protect farm workers? Findings from the For Healthy Kids Study. Journal of occupational and environmental medicine / American College of Occupational and Environmental Medicine. 2012;54(9):10.1097/JOM.0b013e31825902f5. doi:10.1097/JOM.0b013e31825902f5. 
 Islam T, Berhane K, McConnell R, et al. Glutathione-S-Transferase (GST) P1, GSTM1, Exercise, Ozone and Asthma Incidence in School Children. Thorax. 2009;64(3):197-202. doi:10.1136/thx.2008.099366. 
 Keet CA, McCormack MC, Pollack CE, Peng RD, McGowan E, Matsui EC. Neighborhood Poverty, Urban Residence, Race/ethnicity and Asthma: Rethinking the Inner-city Asthma Epidemic. The Journal of allergy and clinical immunology. 2015;135(3):655-662. doi:10.1016/j.jaci.2014.11.022. 
 Li Y-F, Gauderman WJ, Avol E, Dubeau L, Gilliland FD. Associations of Tumor Necrosis Factor G-308A with Childhood Asthma and Wheezing. American Journal of Respiratory and Critical Care Medicine. 2006;173(9):970-976. doi:10.1164/rccm.200508-1256OC. 
 Linn WS, Rappaport EB, Berhane KT, Bastain TM, Avol EL, Gilliland FD. Exhaled nitric oxide in a population-based study of Southern California school children. Respiratory Research. 2009;10(1):28. doi:10.1186/1465-9921-10-28. 
 McCormack MC, Breysse PN, Hansel NN, et al. Common Household Activities are Associated with Elevated Particulate Matter Concentrations in Bedrooms of Inner-City Baltimore Pre-School Children. Environmental research. 2008;106(2):148-155. doi:10.1016/j.envres.2007.08.012. 
 McConnell R, Islam T, Shankardass K, et al. Childhood Incident Asthma and Traffic-Related Air Pollution at Home and School. Environmental Health Perspectives. 2010;118(7):1021-1026. doi:10.1289/ehp.0901232. 
 Salam MT, Bastain TM, Rappaport EB, et al. Genetic Variations in Nitric Oxide Synthase and Arginase Influence Exhaled Nitric Oxide Levels in Children. Allergy. 2011;66(3):412-419. doi:10.1111/j.13989995.2010.02492.x. 
 Salam MT, Islam T, Gauderman WJ, Gilliland FD. Roles of Arginase variants, Atopy and Ozone in Childhood Asthma. The Journal of allergy and clinical immunology. 2009;123(3):596-602.e8. doi:10.1016/j.jaci.2008.12.020. 
 Salvatore AL, Bradman A, Castorina R, et al. Occupational Behaviors and Farmworkers' Pesticide Exposure: Findings From a Study in Monterey County, California. American journal of industrial medicine. 2008;51(10):782-794. doi:10.1002/ajim.20622. 
 Salvatore, A. L., Chevrier, J., Bradman, A., et al. A community-based participatory worksite intervention to reduce pesticide exposures to farmworkers and their families. American journal of public health, 2009; 99(S3), S578-S581. doi:10.2105/AJPH.2008.149146 
 Thompson B, Griffith WC, Barr DB, Coronado GD, Vigoren EM, Faustman EM. Variability in the TakeHome Pathway: Farmworkers and Non-farmworkers and their Children. Journal of exposure science & environmental epidemiology. 2014;24(5):522-531. doi:10.1038/jes.2014.12. 
 Wenten M, Gauderman WJ, Berhane K, Lin P-C, Peters J, Gilliland FD. Functional Variants in the Catalase and Myeloperoxidase Genes, Ambient Air Pollution, and Respiratory-related School Absences: An Example of Epistasis in Gene-Environment Interactions. American Journal of Epidemiology. 2009;170(12):1494-1501. doi:10.1093/aje/kwp310. 
 Wright, John Paul, et al. "Association of prenatal and childhood blood lead concentrations with criminal arrests in early adulthood." PLoS Medicine 5.5. 2008. doi:10.1371/journal.pmed.0050101 
Commenter (4844) states that, by requiring reproducibility, this rule may also exclude many landmark public health studies that were so scientifically rigorous and resource intensive that they could not be reproduced, such as the Framingham Heart Study, a 70-year long cardiovascular epidemiologic study, the Diabetes Control and Complications Trial, a 10-year study of the effects of intensive diabetes control on long-term complications, or the Nurses' Health Study, a decades-long study into the risk factors for major chronic diseases in women. 
Commenter (5012) expresses concern that the Children's Health Study conducted in southern California would also be eliminated.
Commenter (5177) submitted four attachments and a summary of each as examples of important studies that may be excluded in regulatory decision-making: 
      1. As part of a large cohort study of cancer and hematotoxicity among approximately 75,000 workers exposed to benzene sponsored by The Chinese Academy of Preventive Medicine and the U.S. National Cancer Institute, a cross-sectional study was performed of currently exposed workers in order to gain insight into early biologic effects associated with benzene exposure. The report presents the findings of the association between benzene exposure, benzene metabolite formation, and hematotoxicity.
      2. A study was conducted to determine the relation between the consumption of contaminated local fish and concentrations of total polychlorinated biphenyls (PCBs) and 68 PCB congeners in the milk of nursing Mohawk women residing near three hazardous waste sites. Estimated cumulative lifetime exposure from local fish consumption was significantly related to milk total PCB and to three congeners only among those Mohawks who gave birth from 1986 to 1989. 
      3. This report summarizes the health implications associated with exposure to PCBs, primarily through fish consumption. The weight of evidence clearly indicates that populations continue to eat fish containing PCBs and that significant health consequences are associated with consumption of large amounts of some fish. 
      4. The EPA has been performing its second Six-Year Review of the drinking water contaminants regulated under the Safe Drinking Water Act (SDWA). The primary purpose of this document is to provide a screening-level review of the health effects component of the Six-Year Review 2 effort. 
Commenter (5419) states that complete exclusion of an entire study could limit access to scientifically defensible information to regulators and other stakeholders to achieve desired regulatory aims. The commenter provides three examples of research that the Electric Power Research Institute (EPRI) believes are important for consideration in regulatory activities with an explanation of the potential issues associated with making data required to understand human health effects and dose‐response relationships publicly available. These are summarized below:
      1. Occupational Epidemiology Study of Hexavalent Chromium Cancer Risk: EPRI funded a study to update the mortality, exposure reconstruction and dose‐response modeling for a cohort of workers from the Painesville, Ohio Chromate Production Plant who were exposed to hexavalent chromium, resulting in elevated rates of lung cancer (Proctor et al., 2017). This cohort is associated with significant regulatory precedent as it updated a study used by the Occupational Safety and Health Administration for risk assessment when the hexavalent chromium Permissible Exposure Limit was lowered in 2006. The current EPA inhalation cancer slope factor, originally set in 1984, was also based on an earlier study of workers from the Painesville plant, but the EPRI study uses more robust data and included short‐term workers, thus allowing examination of the dose‐response relationship in the low exposure range. Clearly this study is of interest for risk assessment (in particular, the EPA's IRIS program); however, the original data are not publicly available as they are owned by the company that employed the workers, and EPRI's research was subject to a strict confidentiality agreement. Complete study validation, as expressed in the proposed rule (i.e., data to be publicly available in a manner sufficient for independent validation), would not be possible. 
      2. Children's Air Pollution Asthma Study (CAPAS): EPRI funded a panel study of asthmatic children involving residential exposure assessment and collection of pulmonary function and symptom data. Several papers resulted from the research related to the associations between indoor and outdoor PM2.5 and its individual components and asthma exacerbation (Habre et al., 2014a, 2014b; Rohr et al., 2014; Schachter et al., 2015; additional paper forthcoming). Additionally, CAPAS provided important information on the role of PM components in the health effects, which is a topic of interest to EPA and others, given that it is unlikely that all PM components are equally toxic. The consent form utilized in the study stated that indoor exposure data as well all questionnaire data would be kept strictly confidential.
      3. Low Dose Radiation Exposure Health Effects: EPRI has for nearly 10 years worked to increase the understanding of effects from low doses of ionizing radiation. Observational studies have been well‐utilized and form the basis for radiation protection standards around the world (Ozasa et al., 2012; Richardson et al., 2015). Since many of the data from human radiation studies are managed by other research organizations that have highly restrictive data access policies, it is unclear that special arrangements can be made with these entities to release the data for public access. EPRI is concerned that these seminal studies may be eliminated from future consideration as a result of this proposed rule.
Commenter (6111) states that studies that would be excluded from EPA consideration under the proposal form the basis for multiple regulatory actions that EPA and other agencies have taken over the course of many years. The commenter provides the following examples: 
 Lead: Early studies on the neurological effects of low-dose exposure to lead such as Herbert Needleman's 1979 paper finding a negative relationship between the level of lead in children's teeth and IQ scores. EPA relied on this study in its 1986 Air Quality Criteria document for lead. EPA's Lead and Copper Rule, which established the federal regulations for lead under the SDWA, in turn relied on that Air Quality Criteria document to identify blood lead levels of concern. EPA relied on both the 1986 Air Quality Criteria and on Needleman's research directly in establishing standards for lead-based paint hazards under the TSCA. Needleman's work, and subsequent studies building upon it, also supported EPA's decision to revise the NAAQS for lead in 2008. After 40 years, and with the principal investigator no longer alive, it is not clear that the raw data from the Needleman study is available. Even if the data were, they could not be made publicly available without invading the privacy of the study participants. Importantly, it would not be possible to conduct that same study at this time, because children no longer have blood or dental lead levels as high as they did in the 1970s as a result of EPA's implementation of the statutes.
 Arsenic: EPA's drinking water standard for arsenic under the SDWA is dependent on studies that the agency would now be compelled to ignore under the proposed rule. EPA established a drinking water standard of 10 ppb for arsenic in 2001. The Food and Drug Administration ("FDA") then relied on EPA's determination. In setting this standard, EPA relied on a National Research Council review of the scientific evidence, which "concluded that [certain epidemiological] studies from Taiwan provided the current best available data for the risk assessment of inorganic arsenic-induced cancer." The Taiwanese papers looked at rates of skin cancer and blackfoot disease in villagers from southwestern Taiwan who were exposed to high levels of arsenic in their drinking water. These studies were based on data from clinical examinations of the research subjects and therefore included confidential patient data that likely cannot be released to the public. In addition, given that the first data were collected more than 50 years ago, the studies are based on data that may no longer be available.
 PCBs: EPA's regulations establishing water quality standards for PCBs under the CWA were based in part on long-term epidemiological studies of cancer rates in workers exposed to PCBs.
 Radionuclides: EPA's S regulation for radionuclides relied on epidemiological studies of survivors from the Hiroshima and Nagasaki atomic bomb attacks.
 Methylmercury: EPA's reference dose for methylmercury in fish that will be consumed by humans relied on data from human exposures in the Faroe Islands. Precluding reliance on these and other studies for the sole reason that the underlying raw data has not been or cannot be released to the public is arbitrary, capricious, contrary to professional best practices, and antithetical to protection of public health and safety as required by the Statutes. The proposed rule will prevent EPA from relying on the "best available science."
Commenter (6373) provides four examples of research and studies the City (NYC) has conducted to evaluate the impact of environmental exposures on populations within its borders. The commenter states that all these studies have informed or currently inform EPA decision-making and are considered regulatory science to which the Rule would apply. The commenter states that, if the Rule is finalized, the valuable findings that these studies offer may be discounted by EPA, to the detriment of New Yorkers and the country. 
 The Department of Health and Mental Hygiene (DOHMH) used study designs to identify individual-level risk factors for the elevated risks of dying on extreme heat event days (e.g., ethnicity, age, and location at time of death). DOHMH used the analysis to develop the Heat Vulnerability Index. If DOHMH had to make the raw data available to public, the agency wouldn't be able to use these study designs while protecting individual privacy.
 In collaboration with Brown University, DOHMH conducted a series of analyses to examine the health impacts of air pollution on several birth outcomes, including birth weights, pre-term births, and hypertensive disorders of pregnancy.
 The World Trade Center Health Registry (WTCHR) is the largest registry in the United States that tracks the health effects of a disaster. The WTCHR collects personal identifiable information as well as protected health information, with permission from each enrollee.
 New York City Health & Nutrition Examination Survey (NYC HANES) is a community-based health survey first conducted in 2004 by DOHMH and then most recently in 2013-14 by City University of New York School of Public Health and DOHMH. The data collected enables DOHMH to learn how many New Yorkers suffer from basic conditions such as diabetes, high blood pressure, high cholesterol, and depression.
Response: The EPA clarifies that in this final rule the Agency is neither specifically nor categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. 
The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. See 40 CFR 30.5 for a description of the factors the EPA will consider when determining the consideration to afford to pivotal science when the underlying dose-response data are not available in a manner sufficient for independent validation. The EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
9.1.1.5.7 Multi-Study Issues
Comment: Commenter (6125) states that the proposal has failed to consider the effect of the rule on consolidated models. The commenter states that the Integrated Exposure Uptake Biokinetic (IEUBK) model, a crucial tool to assess lead body burden in regulatory and other settings, is based on many individual studies to support various individual relationships. The commenter asks: given the number of studies needed to support what is clearly a highly influential tool in regulation, how would EPA determine which supporting studies needed to provide data and which would not? The commenter states that EPA has not fully considered these kinds of multi-study issues, and the potential effect on the model as a whole. The commenter notes that the contributing researchers may have little motivation to prepare and make available data to satisfy this new and unique policy.
Response: The EPA is finalizing the scope of the rule to apply to dose-response data, and not models. Specifically, where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies --  based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements or quantitative analyses of an EPA significant regulatory action or influential scientific information will be identified as pivotal science. The EPA acknowledges that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
Comment: Commenter (6125) states that EPA has not considered the potentially significant implications of this policy for researchers doing meta-analyses, which have been crucially important in understanding lead, and which consider many studies by many researchers, who may have varying views on releasing their individual data for public review. The commenter provides that one example of a possible regulatory issue would be the 2006-2008 lead air standards, which were based in part on a pooled analysis, for which two of the seven primary investigators declined to provide the raw data to the public.
Commenter (6895) asserts that the rule would exclude high quality, peer-reviewed science. The commenter states that loss of information would be particularly egregious in an area called out in the notice, the "point-of-departure from which a reference value is calculated." The commenter states that reference value determinations all too often suffer from a paucity of data and will be further hindered by excluding sound science that does not meet the proposed availability requirement.
Response: The EPA clarifies that in this final rule the Agency is neither specifically nor categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
The EPA is finalizing the scope of the rule to apply to dose-response data. Specifically, where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies --  based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements or quantitative analyses of an EPA significant regulatory action or influential scientific information will be identified as pivotal science. The EPA acknowledges that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
Furthermore, "pivotal science" includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). These are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. In the context of this rule, studies informing the dose-response modeling (e.g., benchmark response selection), studies identifying data for toxicokinetic adjustments, or studies informing the selection of uncertainty factors do not drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. Future statute-specific rulemakings may interpret "pivotal science" more broadly.
9.1.1.5.8 Non-US Studies and International Studies
Comment: Commenters (1973, 6188, 6906, 8913) express concern that the proposed rule could limit recognition of analysis conducted and information available in foreign jurisdictions. Comments include:
 Commenter (1973) states that essentially all non-U.S. studies would be legally unable to meet EPA's requirements, including the Canadian Community Health Study data, the ESCAPE study, the UK national cohort study, the Netherlands National Cohort study, the Danish National Birth Cohort and the Norwegian Birth Cohort (both funded by National Institutes of Health (NIH)). The commenter states that EPA will achieve no greater "transparency", they will only deny themselves the ability to set standards based on the studies that the rest of the world relies on. 
 Commenter (6188) states that the ESCAPE study relied upon data protected by European privacy laws to combine multiple cohorts across Europe and examine the association of air pollution with mortality in study participants. The commenter notes that the findings of that study are consistent with multiple other studies that examine the effects of particulate matter exposure using different methodologies. 
 Commenter (6906) states that consideration of hazard and risk assessments from other jurisdictions for Risk Evaluations under TSCA promotes a uniform international market by coming to a common understanding of risk. The commenter notes that recognition of internationally available data also reduces EPA's burden in performing evaluations, assuming EPA can verify the quality of information meets the standard of "best available science." The commenter states that it also reduces industry's burden by allowing industry to rely on and/or provide data across countries.
 Commenter (8913) states that many studies span decades, involve multiple collaborations, and use pooled international data. The commenter adds that a prime example is a recent Lancet study by an international team which included scientists from the Radiation Epidemiology Branch of the National Cancer Institute (NCI).
Response: The EPA clarifies that in this final rule the Agency is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. This includes studies from foreign jurisdictions. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. 
The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. See 40 CFR 30.5 for a description of the factors the EPA will consider when determining the consideration to afford to pivotal science when the underlying dose-response data are not available for independent validation.
9.1.1.5.9 Older Studies
Comment: Commenters express concern that many older studies could not be used under this rule for the following reasons:
 Older studies where the authors or data sources may not be accessible. (1006, 9225)
 Open data requirements would be a new qualification for the field not in line with prior publication standards; as a result, many quality studies would be disregarded due to the old societal standards under which they were published. (1481)
 Most existing environmental health studies were completed before the movement toward public access in science, and such existing studies cannot reasonably be expected to retroactively release their data to be eligible for consideration in future regulatory decisions. (1994)
 Public access to data from older studies might not be feasible. (2472)
 Older studies that have data stored on older technology that can't be recovered. (8281-PH38)
Commenter (6125) states that, even for controlled human and animal studies, where subject data might ethically be released, the underlying data may no longer be accessible years after publication.
Commenter (6907) is concerned that, under the proposed rule, older studies on pesticides may not be considered. The commenter states that this is of particular concern for older pesticides which have now been unregistered but for which we have older studies that indicate that these pesticides pose harm to humans or the environment. The commenter asserts, for example, that there is no sound reason to redo the studies related to DDT. 
Commenter (6915) expresses concern that many of the standards that are developed or updated by EPA are for chemicals that have an extensive, older body of scientific literature on their effects, but that are not currently being actively researched. The commenter states that the vast majority of studies considered for standard-setting are not new and were not conducted, designed, or published with the goal of ensuring data availability. The commenter states that, accordingly, their data are likely unavailable and, even if data were kept, the formats in which older data are stored may not be accessible from currently available computers, potentially invalidating the use of those studies as the basis for future regulatory standards. 
Response: The EPA clarifies that in this final rule the Agency is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. This includes older studies. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. 
The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. See 40 CFR 30.5 for a description of the factors the EPA will consider when determining the consideration to afford to pivotal science when the underlying dose-response data are not available for independent validation.
Comment: Commenter (6125) expresses concern that EPA has not considered or examined the number of potentially significant studies for which EPA cannot obtain raw data or methodology due to age. The commenter includes a review of EPA's IRIS and pesticide data bases to show the age of many of the studies that underlie risk assessments for carcinogens and reference doses and concentrations (Rfd, Rfc) for a number of substances that are regulated under legislation that addresses hazardous wastes, toxic chemicals, clean water, drinking water, clean air and pesticides. The commenter notes that perhaps the most revealing and stark finding of this review comes from the examination of the IRIS data base used to determine cancer classifications, RfDs and RfCs for IRIS chemicals. The commenter provides that most of the entries are over 20 years old, and some over 30 years old. The commenter states that this guarantees that the studies used to develop the findings are at least that old, and, perhaps, decades older still. The commenter asserts that, without examining the support document for each entry, one cannot rule out that the age of the key studies may far exceed that of the entry. The commenter notes that, if a regulatory action that relied on one or more of these assessments was challenged due to the lack of the original data, under the requirements of the proposed rule, most of the cancer classifications and reference values could be vacated, aside from those values that are based upon certain NIOSH epidemiology data, NTP bioassay data or pesticide-related CBI data, and then, only if an exemption were made for the latter. 
Response: The EPA is finalizing the scope of the rule to apply to dose-response data. Specifically, where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies --  based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements or quantitative analyses of an EPA significant regulatory action or influential scientific information will be identified as pivotal science. The EPA acknowledges that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis for various reasons, including the age of the pivotal science or its underlying dose-response data. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. See 40 CFR 30.5 for a description of the factors the EPA will consider when determining the consideration to afford to pivotal science when the underlying dose-response data are not available for independent validation.
Furthermore, "pivotal science" includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). These are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. In the context of this rule, studies informing the dose-response modeling (e.g., benchmark response selection), studies identifying data for toxicokinetic adjustments, or studies informing the selection of uncertainty factors do not drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. Future statute-specific rulemakings may interpret "pivotal science" more broadly.
9.1.1.5.10 Studies Used to Support Other Agency Standards
Comment: Commenter (0671) states that no other federal agency has such a broad standard as the one envisioned here. The commenter states that EPA knows that IRBs will not allow such studies' data to be made public, so to require public data from these studies will make them invisible to the rulemaking process. 
Commenter (1973) thinks that the proposed rule would put EPA in direct opposition to other agencies' standards for considering evidence:
 Health effects of smoking, high cholesterol, etc. by the Centers for Disease Control and Prevention (CDC).
 The studies considered for issuing guidelines generated by the NIH.
 The FDA reliance on non-publicly available studies to approve drugs for use by people.
Commenter (5022) provides that some of EPA's greatest public health accomplishments, such as eliminating lead in gasoline and classifying secondhand smoke as a cause of cancer (which led to banning smoking from indoor public places), were based on the kinds of data that would be discarded under the proposal. The commenter states that such data are widely used in rulemaking proceedings by other U.S. government agencies and around the world. 
Commenter (6117) references the following policies of other agencies and asserts that none of these policies require that all individual-level underlying data and models be made available to the public: 
 NIH requires that researchers conducting agency-funded research pre-register clinical trials, pre-specify study endpoints and statistical plans, and report research results.
 All NIH applications for $500,000 or more in direct funding in any given year must contain a data sharing plan, and some institutes of the NIH share data with researchers via a secure website.
 Many federal agencies, including the EPA, require that any published articles based on federally funded research be made available via the PubMed Central website. 
 The International Committee of Medical Journal Editors has adopted a policy requiring pre-registration of clinical trials, and publication of a data sharing statement. 
 The Institute of Medicine published a report recommending that trials be registered prospectively, that summary-level results be shared with the public after trial completion, and that the metadata and patient-level data be responsibly shared with researchers 6 months after scientific publication, or 30 days after regulatory approval.
Commenter (6125) states that the CDC and other Federal Agencies involved in reducing lead risk, would not be subject to the restrictions on use of science imposed by the proposal, and thus would continue to use the older studies as evidence, creating an unwarranted science practice and policy schism in the federal government. The commenter provides the following examples:
 The CDC has acknowledged that no safe level of lead in blood has been identified and has identified a "reference level" to define especially high-risk populations and geographic areas most in need of primary prevention. Even some of the studies that the Federal government's most respected lead advisory group (CDC's Advisory Committee on Childhood Lead Poisoning Prevention, or ACCLPP) used to develop background justification for the reference level could be precluded from EPA use under this policy. If some of those 92 plus references and studies fail to meet this policy, EPA could not rely on them in developing regulations, and thus could issue regulations that fail adequately to address CDC's high-risk exposures. Moreover, EPA cost-benefit analyses could not use the CDC health effects work in calculating regulatory benefits. Further, the additional requirements in the proposal could require that EPA re-evaluate the appropriateness of the CDC dose-response relationship and give "explicit consideration" to other dose-response models even if, as in this case, they have been vetted and rejected by CDC and its advisory committee. 
 Both EPA and the U.S. Department of Housing and Urban Development (HUD) regulate similar residential sources of lead (paint, soil, and dust) - EPA in private housing, and HUD in Federally supported housing, including Federally supported units in private buildings. This policy shift increases the potential for disparate treatment and public confusion stemming from those situations. The identical risks addressed by the two programs should be analyzed consistently. HUD has already acted to incorporate CDC's statement into its programs by making its policy on dust-lead clearance levels more stringent; is still considering its response. But under this proposal, EPA might not be able to rely on as wide a range of studies as would be available to the rest of the federal family, which could result in different risk assessments and regulatory outcomes for similar, and possibly even adjacent, dwellings. 
Response: The EPA clarifies that in this final rule the Agency is neither specifically nor categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
The transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency. The EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. For example, in 2002 OMB released its Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies, which includes discussion of the importance of the reproducibility of analyses underlying influential information. The EPA's 2016 Plan to Increase Access to Results of EPA-Funded Scientific Research noted that "transparency is a core EPA value" and that increased availability of research data would accelerate scientific breakthroughs that support the Agency's mission and policymaking efforts. The EPA's Open Government Plan 5.0 also details the EPA's progress in implementing the tenets of the numerous data transparency initiatives in the federal government prior to 2018, including OMB M-10-06, the Office of Science and Technology Policy Memorandum of February 22, 2013, and OMB M-13-13. In 2019, Congress passed the Foundations for Evidence-Based Policymaking Act of 2018 (or OPEN Government Data Act, Pub. L. 115-435) into law, which included requirements for federal agencies to prioritize making their data available to the public, and OMB has released additional guidance for implementing the act[.].
9.1.1.5.11 Privacy Data 
Confidential Business Information (CBI)
Comment: Commenters (1006, 2787, 4877, 6114, 6116, 6128, 6131, 6707, 8281-PH38, 9225) express concern that this proposal would likely prevent the EPA from using many studies performed using data that contains CBI. Comments include:
 Commenter (2787) states that confidential industry data are vital to understanding the nature of chemical pollutants, the impacts of pollution, and the most effective ways to protect the environment and public health. 
 Commenter (4877) states that many studies contain CBI and this data should be exempt from blanket data sharing requirements. The commenter states, however, that studies based on this data should not be automatically excluded from the EPA's policy-making progress, which is what proposed rule is designed to do.
 Commenters (6114, 6116, 6131) express concern that this proposal would likely prevent the EPA from using many studies performed using data from industrial sources because they contain CBI, such as product formulations. The commenters state that federal law imposes criminal penalties for disclosing CBI, which could result in liability for the EPA, if they follow down the path of making research data publicly available.
 Commenter (6707) states that some data that are not ready for public release (such as proprietary or temporarily embargoed data) could not be used since they simply cannot be made public. The commenter notes, for example, that there is a famous case involving the study of the health effects of living near a major hog farming operation. The commenter provides that, because this work included the study of the effects on children, this data could not be fully released under this law, and thus this major, relevant study would not be considered when the EPA makes a decision on the environmental impacts of that hog farm. 
Response: In this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. This includes studies that use data containing CBI. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information, and will continue to handle studies and underlying data in accordance with existing CBI requirements and procedures. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
Confidential Health Data 
Comment: Commenters (2425, 4838, 4843, 4844, 6130, 6868) express concern that this rule would create procedural barriers to the use of studies that rely on confidential health information, thereby restricting the EPA's ability to consider significant amounts of relevant scientific evidence bearing directly on its core mission. Commenter (4838) states that EPA should not provide blanket limits on the use of studies that cannot be made public because they contain confidential health or business information. 
Commenters (1170, 1536, 1973, 4933, 5981, 6788) assert that since most medical research uses patient data that is subject to stringent privacy requirements, the proposed rule automatically eliminates from consideration a majority of medical research. Commenter (1536) states that the proposed rule dismisses virtually all medical research because the research protects the privacy of the patients that volunteered even though these legal obligations are a routine part of HIPAA. 
Commenter (6896) states that it is vital to have accurate information about the most pressing health and safety issues to prevent work-related injuries, illnesses, and fatalities and to protect the confidentiality of personal identifying data. The commenter asserts that quality science protects the confidentiality of construction workers personal identifying data. 
Commenter (8281-PH63) states that important studies by Minnesota's Department of Health help Minnesota derive health protected values under state law and help the state of Minnesota in litigation. These studies are only possible because we provided absolute guarantees to the participants that their data would be protected and that we would assure its confidentiality. The proposed rule will make it unlikely that public health data such as these would be available for states to use, but even more so for the EPA to use in its decision-making. 
Commenters (2429, 2907, 2908, 4884, 5013, 6188, 6696, 6882, 6915, 6943, 8281-PH26) express concern that the proposed rule would limit access to many important and valid peer-reviewed studies that have relied on confidential health data, undermining EPA's mission to protect public health. Comments include:
 Commenter (4884) states that peer-reviewed studies have ensured the sound basis of EPA rules. The commenter states that valid health effect studies are often non-public because they rely upon confidential medical records. The commenter states that, as long as the analysis of the raw data is available and correct, such studies must be used to achieve the legislative intent of the environmental laws.
 Commenter (6943) asserts the EPA should continue its current system in basing its rulemaking decisions on peer-‐reviewed scientific studies while protecting the privacy of citizens in the human database. 
 Commenter (6882) states that peer-reviewed studies assessing the effects of pollution exposure on humans, based on actual personal health data -- which by law must be kept confidential -- are among the best science available to the Agency. The commenter states that the proposal would allow EPA to ignore this data unless that confidential personal health data were made publicly available. The commenter states that we must take these studies into account when creating our policies in order to protect the public's health.
Response: In this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. This includes studies that use confidential health data. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
Comment: Commenters (1006, 1518, 1981, 4933, 5013, 5178, 6450, 6195, 6877, 6882, 6896, 8272, 8281-PH63) express concern that many important studies that rely on confidential health data could not be used. Comments include: 
 Commenter (1006) states that Google's AI study release would not be able to function with this proposal due to confidential data. The commenter states that they both have hidden the public data and the means of which the AI is coded. The commenter states that, if they wanted to do anything with the EPA in relation to a hospital it would be impossible for them. 
 Commenters (1518, 4933, 5013, 6882) express concern that, if the transparency regulation were in place, all health studies on fracking would be simply not considered because the research could not be conducted due to non-disclosure agreements. Commenter (1518) states that, if the EPA were unable to consider such studies due to a requirement to release patient information, the Agency would be ill-equipped to weigh benefits vs. costs. Commenter (5013) references three studies:
 One study found that drilling and fracking activity was associated with increased rates of hospitalization in Pennsylvania. In communities with the most wells, the rate of cardiology hospitalizations was 27 percent higher than in control communities with no fracking. 
 Another study found that living near fracking operations significantly increases asthma attacks. 
 Another study found that pregnant women who lived near active fracking operations in Pennsylvania were at a 40 percent increased risk of giving birth prematurely. 
 Commenter (1981) does not agree with the purposed rule's claims that it "will help ensure that EPA is pursuing its mission of protecting public health and the environment in a manner that the public can trust and understand." The commenter states that, where the rule would cause data affecting tribal members to be excluded as a matter of cause, we refute this claim of trust and understanding. The commenter states that fish surveys demonstrate that the proposed rule results in inequitable data representation.
 Commenter (1981) states that, under the proposed rule, fish surveys would be excluded. The commenter states that almost no activity on the reservation has more potential for significantly affecting the economic and political integrity and the health and welfare of all reservation citizens than water use, quality, and regulation. The commenter states that the proposed rule threatens to silence tribal interests where it potentially prevents the use of the fish surveys. The commenter states that, in our view, this seems contrary to "openness, clarity, [and] unobstructed access." 
 Commenter (1981) states that the fish surveys are scientifically rigorous while also protecting the fiduciary interest between the researcher and the research subject. The commenter states that one fish survey provided by the Columbia River Inter-Tribal Fish Commission included a confidentiality agreement for instance -- barring the release private information about tribal member participants -- but also included limitations, peer review, procedural information, and quality control. The commenter states that the fish surveys demonstrate the distinctive relationship between Tribes and the health of our Nation's waterways, and in so doing highlights the need for diverse representation in EPA data collection. While the general population eats 19-56 grams of fish and shellfish on average per day, tribal members in the Pacific Northwest can eat up to 797 grams (1.75 pounds) every day. Tribal populations are exposed to toxins that bioaccumulate in aquatic life due to factory effluent, urban wastewater, and runoff from agriculture and cities.
 Commenters (5178, 8272) state that a recent example of important non-public data on a neurotoxicant is research on the pesticide chlorpyrifos. The commenters report that researchers followed a cohort of children exposed to this pesticide before the current ban on residential use and found lower IQ and working memory to be associated with higher levels of prenatal chlorpyrifos exposure. The commenters state that, in a rulemaking process regarding agricultural use of chlorpyrifos, EPA requested the underlying data from the Columbia Center for Children's Environmental Health (CCEH). The commenters state that the response from CCEH explained that because of the detailed sociodemographic and health-related elements their data set contains, they did not believe they could submit extensive individual-level data to EPA in a way that would ensure participants' confidentiality. The commenters state that such concerns are not uncommon with the kinds of longitudinal data sets that allow identification of long-term consequences of environmental exposures. Commenter (8272) states that a recent analysis of the annual costs to the U.S. economy of a set of endocrine-disrupting chemicals is $340 billion, with the majority of that due to IQ point loss and intellectual disability from in-utero exposure to polybrominated diphenyl ethers (PBDEs).
 Commenter (6195) states that the past EPA administration's decision to ban chlorpyrifos was based in part on studies showing neurological problems for kids exposed to the pesticide. The commenter states that the EPA abruptly decided to reverse this proposed ban because the study authors were unable to make their "raw data" publicly available. The commenter states that the proposed rule would codify this requirement for all pollutants the EPA regulates. 
 Commenter (6450) states that studies that analyze the health impacts resulting from asbestos exposure, exposure to DDT, and birth defects caused by thalidomide would also be excluded under the proposed rule because the data contain confidential patient information protected by HIPAA.
Response: In this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. This includes studies that use confidential health data. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
Intellectual Property
Comment: Commenters (1006, 4877, 6128, 9225) express concern that many studies that include intellectual property would be excluded. Commenters (4877, 6128) state that many studies contain data that is sensitive in terms of intellectual property and this data should be exempt from blanket data sharing requirements. Commenter (9225) asserts that the proposal would restrict the types of science the Agency may use in regulatory decision making, including studies that rely on intellectual property. 
Response: In this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. This includes studies contain data that are intellectual property. The EPA further clarifies that this rule does not impose blanket data sharing requirements; this rule exclusively pertains to the internal practices of the EPA and does not regulate the conduct or determine the rights or obligations of any entity outside the federal government. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
9.1.1.5.12 Studies Controlled by Third Parties
Comment: Commenter (6125) states that, while the proposal acknowledges that data may be "controlled by third parties," the explanation of what this may mean is a vague promise that "EPA shall work with those parties to endeavor to make the data available in a manner that complies with this section." The commenter adds that the ambiguous wording never specifies what actually might qualify, nor what interactions would be required before third party studies such as the Medicare study might be given an exemption from the requirement of public availability. The commenter states that this language does not begin to address the variety of complexities involved with data controlled by a wide variety of third parties each with different policies of access, making it nearly impossible for commenters to know whether third party studies can ever be used without some type of interactive process between the study authors and EPA. The commenter contends that, even if EPA does not intend to apply this provision to categorically bar the use of third-party studies, its failure to point this out, or otherwise explain this provision makes the rule too vague to implement.
Response: This language has been removed from the final rule. 40 CFR 30.5(g) now reads: "The provisions of this section apply to dose-response data underlying studies that are pivotal science, regardless of who funded or conducted the studies. The Agency shall make all reasonable efforts to explore methodologies, technologies, and institutional arrangements for making such data available before it concludes that doing so in a manner consistent with law and protection of privacy, confidentiality, national security is not possible."
Comment: Commenter (6191) expresses concern that data resulting from research funded through sources other than EPA may be "best available" science and per EPA's current directives, should form the basis of new regulations. The commenter states that this research would not be subject to federal data access and sharing requirements and it is unreasonable to expect that anyone doing research that may one day be important for regulatory science would anticipate the importance for this future use. The commenter states that these researchers would not be subject to current public access plans and may be unable to prepare their data in a way that it could be made publicly available, whether because their agreements with human subjects do not permit this or because a lack of funding would prevent them from taking the steps necessary to enable the data to be made public. 
Response: This is a rule of internal procedure that does not impose requirements on any party other than the EPA. It therefore does not establish any requirements or expectations for third party researchers to make the dose-response data underlying their research publicly available, or accessible through a secure data enclave. Furthermore, under this rule the EPA will continue to consider and evaluate all relevant and appropriate science in its significant regulatory actions and influential scientific information. EPA further notes that this internal procedural rule does not interpret or apply any statutory requirement to use "best available science" or any variation thereof. The EPA will undertake such efforts in forthcoming statute-specific transparency or programmatic regulations.
9.2 Regulatory Science Means Scientific Information that Provides the Basis for EPA Final Significant Regulatory Decisions (§30.2)
Comment: Commenters (0567, 4932, 6180, 6356) express concerns regarding the proposed definition of "regulatory science" and provided several suggested changes to the definition for the EPA to consider.
Commenter (0567) recommends a slight revision of EPA's definition by deleting the words "significant" and "final". The commenter states that a generalized definition as recommended by the Institute for Regulatory Science is: 
      Regulatory science consists of applied versions of various scientific disciplines used in different regulatory processes.
Commenter (0567) adds that a simplified, generally applicable definition is: 
      Regulatory science consists of the scientific segment of the regulatory process. 
Commenter (0567) also notes that other federal agencies have also promoted definitions of regulatory science:
 FDA: Regulatory science is the science of developing new tools, standards, and approaches to assess the safety, efficacy, quality and performance of all FDA-regulated products.
 NIH: Regulatory science fosters the development, evaluation and availability of new or improved tools, methods, standards, and applied science that support a better understanding and improved evaluation of product safety, quality, effectiveness, and manufacturing throughout the product life cycle.
Commenter (4932) suggests that EPA alter the definition of "regulatory science" by striking the word "final" from the proposed definition to make the Proposal fully consistent with Executive Order 12866. The commenter states that implication of including the word "final" in those definitions is that the transparency objective (i.e., the disclosure of pivotal regulatory science) does not apply to advanced notices of proposed rulemaking or proposed rulemaking. The commenter states that the public must have access to pivotal regulatory science at these earlier steps in the rulemaking process in order to assess effectively the merit of the Agency's regulatory science. 
Commenter (6356) suggests EPA remove the word "significant" from the proposed definition of "Regulatory Science".
Commenter (6180) suggests "regulatory science" means scientific information, including assessments, models, criteria documents (including narrative criteria translators), and regulatory impact analyses, that provide the basis for EPA final significant regulatory decisions.
Response: The EPA agrees with commenters that the term "regulatory science" may be confusing because it suggests either the term refers to a scientific discipline of regulatory decision-making (akin to FDA), or that some science the EPA considers is inherently regulatory. Neither of these interpretations reflects the Agency's intent in defining this term. The EPA considers the breadth of scientific evidence in its rulemakings; while this scientific evidence informs policy decisions, the EPA's consideration of the science does not make it "regulatory science." To reflect this fact, in the final rule the EPA is changing the proposed term "regulatory science" to "science that serves as the basis for informing a significant regulatory action." 
In the 2018 proposed rule the EPA combined both general categories of scientific information, such as assessments and models, with specific examples of EPA scientific products, such as criteria documents and regulatory impact analyses in the proposed definition for "regulatory science." The EPA acknowledges that this may increase confusion and is therefore limiting the final definition to general categories. As such, the EPA is altering the definition of "science that serves as the basis for informing a significant regulatory action" in 40 CFR 30.2 to mean "studies, analyses, models, and assessments of a body of evidence that provide the basis for EPA significant regulatory actions." Examples of models include those used in regulatory impact analyses. Examples of assessments of a body of evidence include risk assessments, hazard identifications, IRIS assessments, and criteria documents.
Comment: Commenter (6143) suggests that the definition of regulatory science should include the study protocol. The commenter states that, when a study is conducted, the most important step is the careful production of a protocol which addresses all aspects of the study. The commenter states that these should at a minimum include objectives, study design, methods for collection of data, methods for handling the data, methods of analysis, and how results will be interpreted and reported. The commenter states that all the basic parameters should be documented, and include, sensitivity, specificity, variability, reproducibility, bias etc. The commenter states that the a priori definition of how results will be interpreted avoids the temptation to look at the data and perform analyses which have the potential to be biased. The commenter states that studies that are hypothesis generating should not be used as hypothesis testing studies. The commenter states that making protocols available to the public for research funded by EPA could go a long way towards improving the underlying science. The commenter states that all too often research is carried out and variables that should have been identified at the outset are considered after the study results are examined and the data reanalyzed or interpreted differently in the light of information obtained after completion of the research.
Response: The EPA disagrees with the commenter that the definition of science that serves as the basis for informing a final significant regulatory action should specifically include the study protocol. Protocols are often provided with the studies, analyses, models, and assessments of a body of evidence that constitute the definition of the term. Further, this rule does not govern how individuals outside of EPA conduct science, but rather is a rule of internal procedure.
Comment: Commenter (6133) states that the phrase "provides the basis" does nothing to illustrate the meaning of regulatory science, or to limit or particularize its scope, because it is equally vague and unexplained. The commenter adds that all science, data, and information considered by EPA, and relied upon by EPA, "provides the basis" for final EPA regulatory decisions, insofar as EPA includes those materials in its administrative record, certifies that record for judicial review, and may cite and rely upon that information in explaining and defending its final regulatory decisions. The commenter states that, accordingly, the proposed "regulatory science" definition is capacious and unbounded. The commenter states that, alternatively, the phrase "provides the basis" in the proposed "regulatory science" (§30.2) definition could mean that science was one of many studies considered, that it was the bedrock study upon which regulation was grounded, that EPA relied on the study, or that the study was critical to EPA's determination. The commenter asserts that the term "regulatory science" lacks statutory authority, is vague, inconsistent, unexplained, and otherwise arbitrary and capricious.
Commenter (6137) states that EPA's Proposed Rule lacks the requisite standards and principles that are the hallmark of lawful agency decision-making. The commenter states that the vague definitions proposed in §30.2 invite limitless agency discretion to decide when the Rule's requirements apply. The commenter states that the Proposed Rule's definition of "regulatory science" lacks any discernible meaning. The commenter states that there is no standard or definition for determining when scientific information does or does not "provide the basis" for a regulatory decision, nor are there any standards or definitions for what subset of "regulatory decisions" should be deemed "significant."
Response: The EPA agrees with the commenters that the use of "provides the basis" in the definition indicates an expansive scope, and clarifies in the final rule that the scope of science that serves as the basis for informing a significant regulatory action is equivalent to the science included in the public docket as part of a rulemaking, but not all of that body of science would typically be considered "pivotal science." The EPA disagrees that the definition lacks any discernible meaning, as the Agency's procedure for making available science that provides the basis for a regulation in the administrative record is a well-established practice.
9.3 EPA's Regulatory Science Should Be Consistent with the OMB's Final Information Quality Bulletin for Peer Review (83 FR 18770)
Comment: Commenter (6125) states that, while the proposed rule cross-references the OMB Information Quality Bulletin for Peer Review, it is not clear what the proposed rule is intended to add to the Information Quality Bulletin with which EPA already complies or how in fact it would operate. The commenter adds that, while the Bulletin recognizes that an agency need not conduct peer review of studies which already have been adequately peer reviewed (70 FR at 2671), it is not clear if EPA is aware of this. Lastly, the commenter notes that problems may be raised by cross-referencing a document (Information Quality Bulletin for Peer Review) that can be amended. 
Response: The EPA finds merit in this comment, in part. However, in this rule, the EPA is not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review does not typically include reanalysis of the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information) and, thus, peer review is not considered a replacement for the data availability requirements of this rule.
9.4 EPA's Proposed Regulatory Science Requirements Consistency with OMB's Guidelines for Ensuring and Maximizing Quality, Objectivity, Utility and Integrity of Information Disseminated by Federal Agencies (OMB Guidelines)
Comment: Commenter (0555) contends that the proposed requirements are not a radical departure from existing guidelines, including 2004 OMB guidance and the 2011 Executive Order 13563.
Conversely, commenters (6137, 6916, 9227) state that the Proposed Rule contravenes the OMB requirements, as it will prevent EPA from considering all relevant information by precluding consideration of certain data. 
Commenter (6137) states that, according to EPA's IQG, EPA uses a "weight-of-evidence" approach that "considers all relevant information and its quality, consistent with the level of effort and complexity of detail appropriate to a particular risk assessment." The commenter notes that, pursuant to EPA's guidelines, EPA must ensure that the information it disseminates "is accurate, reliable and unbiased." The commenter states that, to do so, it uses the best available science and supporting studies conducted in accordance with sound and objective scientific practices, including, when available, peer reviewed science and supporting studies; and data collected by accepted methods or best available methods. 
Commenter (6916) asserts that the provisions in the proposed rule that would preclude the use of proprietary or confidential data in regulatory actions are inconsistent with OMB Guidelines that indicate that transparency "does not override other compelling interests such as privacy, trade secrets, intellectual property, and other confidentiality protections," that direct agencies to conduct robustness checks when data cannot be released, or that encourage agencies "to address ethical, feasibility, and confidentiality issues with care." The EPA does not explain either (a) why it considers the proposed rule to be consistent with the OMB Guidelines, or if it is, why (b) the regulations it now proposes are necessary in light of the OMB Guidelines' directives.
Commenter (9227) asserts that the discussion in the proposed rule relating data availability to data quality are inconsistent with provisions in the OMB Guidelines that indicate high quality science can be judged by many factors and that controls be implemented "flexibly, and in a manner appropriate to the nature . . . of the information to be disseminated." The commenter also asserts that provisions in the rule requiring public availability of data are inconsistent with provisions in the OMB Guidelines that provide for differential treatment of data due to privacy and confidentiality concerns, that indicate that not all data should be held to a reproducibility requirement, that "the objectivity standard does not override other compelling interests such as privacy, trade secrets, intellectual property, and other confidentiality protections, and that direct agencies to apply robustness checks where public access to data and methods cannot occur.
Commenter (9227) asserts that, by outright prohibiting EPA from relying on a study to support a significant rulemaking if that study's underlying data and models are not publicly available, EPA's proposed rule departs from OMB's unambiguous language instructing agencies that they "shall not" require that all data and models be subject to the reproducibility requirement, and that "the objectivity standard does not override other compelling interests." The commenter adds that the fact that EPA's proposed rule includes a discretionary "exemption" provision does not correct this problem, as that provision would not require the Administrator even to consider whether an exemption is warranted, let alone grant such an exemption under appropriate circumstances.
Commenter (6916) states that the OMB Guidelines also establish and reflect a presumption of objectivity and favored status for peer-reviewed science, which the EPA Proposal flips on its head. The commenter states that this presumption of objectivity and favor for externally peer-reviewed studies is also an element of the EPA Data Quality Act Guidelines, which also reflect the overall principle that "[a]gencies should adopt a common sense approach that build on existing processes and procedures[, and] ... do not impose unnecessary administrative burdens or inhibit agencies from disseminating quality information to the public." The commenter states that adding a mandatory agency-conducted `independent' peer review process (as this Proposal does) for all "pivotal regulatory science" used in making "regulatory decisions" does not comport at all with the EPA or OMB Guidelines implementing the Data Quality Act, EPA's statements in the Proposal notwithstanding. 
Response: This final rule makes incremental steps towards increasing data transparency by codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation and thereby improving the public's access to the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. This rule is designed to build upon OMB M-19-15, which highlights the need to characterize the sensitivity of an agency's conclusions to analytic assumptions, as well as other federal guidance documents that require greater data transparency. The EPA's attention to data transparency is also responsive to the broader interest in greater data and model transparency observed in the numerous transparency initiatives in the scientific community and federal government, as well as the criticism the EPA has received from members of the public, scientific community, and Congress on the transparency of the scientific basis for EPA's decisions in previous influential scientific information assessments and regulatory actions.[,][,][,][,] The EPA's continued progress toward maximizing transparency is vital to building and maintaining trust with the public and credibility in the Agency's decisions.
The EPA further clarifies that in this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available or not available in a manner sufficient for independent validation. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
Finally, the EPA disagrees with the commenter that this final rule undervalues the role of peer review. 40 CFR 30.6 states: "The EPA shall conduct independent peer review consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review and the exemptions described therein. The EPA will evaluate whether or not to conduct additional peer review of individual studies identified as pivotal science if the studies have already undergone peer review. Because transparency in pivotal science includes addressing issues associated with assumptions used in analyzing dose-response data, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied and the implications of those assumptions for the results." Adequate peer review remains an essential factor to pivotal science in the EPA's final regulatory actions and influential scientific information.
9.5 Regulatory Science that Cannot Meet the Requirements of §§30.5 and 30.6
Comment: Commenter (6924) states that the rule's requirements for specific types of defaults, test methods, dose-response models and/ or analyses are not supported by current science. The commenter refers to the proposed rule which states that "EPA will use peer-reviewed information, standardized test methods, consistent data evaluation procedures, and good laboratory practices to ensure transparent, understandable, and reproducible scientific assessments." The commenter states that it is not appropriate to require use of standardized test methods (guideline studies) and good laboratory practices (GLP) studies, as described below.
      Guideline and GLP studies  -  mostly sponsored by industry to meet regulatory requirements - don't necessarily use modern methods for evaluating chemicals and are not designed to address issues with evaluated health effects from low-dose exposures, complex and systemic endocrine effects, behavioral or learning effects, metabolic perturbations, or upstream effects like reduced sperm count or reduced anogenital distance which are predictors of infertility. Furthermore, GLP and Guideline studies are not consistently associated with higher quality research, proper study design or correct statistical analysis., Much of the research available in the scientific literature is hypothesis driven-- for example, exploring mechanisms to explain how a detrimental health impact occurs. These experimental protocols are tailored to address the question being asked and generally there are no standardized test methods that could be followed. Yet, these experimental findings are critically important contributions to the body of scientific evidence. For the highest quality scientific assessment, a systematic review approach which considers all relevant scientific evidence and evaluates study quality according to established criteria should be used, as recommended by the NAS. Inappropriately using GLP as a measure of study quality would lower the overall quality of the assessment because other studies which are higher quality and/ or more directly relevant to the study question would be downgraded or excluded.
Commenter (6872) states that it is important that EPA have the ability to consider data that do not precisely fulfill the proposed rule's criteria if the data are informative and of high quality. The commenter states that, for example, requiring GLP sounds "good" but, in practice, many academic research laboratories conduct stellar research but may not meet all GLP criteria. The commenter states that data from such laboratories should be considered and weighed on its merits and should not be excluded from EPA consideration by rule. The commenter states that data from new and improved, but non-standard test methods, should be available for EPA to evaluate on their merits and use as appropriate.
Response: The EPA clarifies that in this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. In addition, this rule does not apply to dose-response models. Further, this final rule does not establish requirements that the EPA only consider standardized test methods (guideline studies) and GLP studies. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available or not available in a manner sufficient for independent validation. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
Comment: Commenter (6935) states that a final rule should not disqualify a study from consideration as pivotal regulatory science if it addresses most key factors in §§30.5 and 30.6 and can be independently assessed and verified. The commenter states that the Proposed Regulation generally would increase transparency with respect to dose response data and models. The commenter states that, in a final rule, EPA also should balance strict adherence to new transparency requirements against the need for regulatory certainty and efficiency. The commenter states that EPA should consider that only the rare past study, from inside or outside the Agency, has addressed all these factors. The commenter states that it is a worthwhile goal that future studies, especially those within the Agency, should meet as many of the requirements as possible from proposed provisions 30.5 and 30.6.
Response: The EPA clarifies that in this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. See 40 CFR 30.5 for a description of the factors the EPA will consider when determining the consideration to afford to pivotal science when the underlying dose-response data are not available in a manner sufficient for independent validation. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available or not available in a manner sufficient for independent validation. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
9.6 How Should EPA Address a Circumstance in which EPA Has a Statutory Requirement to Make a Determination for Which Scientific Information Publicly Available Sufficient for Independent Validation Does Not Exist?
Comment: Commenters (6120, 6895) notes that in the proposed rule, the EPA requested comments on how to address circumstances in which the EPA has a statutory requirement to make a determination when scientific information publicly available in a manner sufficient for independent validation does not exist. 
Commenter (6120) states that this situation highlights the importance of using all available data, including studies of accidental exposures to chemicals that do not follow standardized test methods and cannot be reproduced or validated, academic studies on mechanisms such as endocrine disruption, and environmental sampling. The commenter states that all new compelling peer-reviewed research published in the scientific literature should be considered in regulatory decisions. The commenter states that this also underscores the importance of a panel of independent scientists to peer review the information and consider all the evidence. The commenter also states that, in the absence of data, the EPA should make conservative assumptions protective of vulnerable populations such as children and women of childbearing age.
Commenter (6895) notes that it is likely that in such circumstances such data does exist. The commenter states that, in the very odd circumstance that such data does not exist, it behooves EPA to develop the data. The commenter states that, since EPA regulations and other actions are applicable to all of the country, the data used to make these decisions should be collected at the community and larger aggregate level.
Commenter (6375) suggests the following approach where EPA has a statutory requirement to make a determination for which scientific information publicly available in a manner sufficient for independent validation does not exist. 
      EPA should solicit input from EPA Offices (such as the Office of Pesticide Programs) and other federal agencies (such as FDA) that rely heavily on proprietary studies, many of which are not published in the peer-reviewed scientific literature and for which the data have historically not been available to the public for independent validations to make a determination as required by statute. As described above, EPA's Office of Pesticide Programs relies heavily on proprietary studies/CBI and apparently has devised mechanisms that balance protection with transparency of pivotal regulatory science and the public's right to now. One mechanism is Data Evaluation Records (DERs) that summarize proprietary studies. The other is human health risk assessments that integrate findings in DERs such that stakeholders can assess the impact of a particular endpoint (e.g. health effect) in a particular study ("pivotal regulatory science") on the overall risk assessment. Both DERs and risk assessments have been made available on the docket or by other means. While this is not as transparent as making the underlying studies fully available, sufficiently detailed DERs and risk assessments are arguably as detailed and transparent as many health studies and risk assessments published in the peer-reviewed scientific literature.
Response: The EPA clarifies that in this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available or not available in a manner sufficient for independent validation. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
Further, with this final rule, the EPA is maintaining language in 40 CFR 30.3 stating that the statutes that the EPA administers, or their implementing regulations, will control in the event of any conflicts with the requirements of this rule. The Agency will continue to comply with and abide by the requirements in those statutes and implementing regulations, including regulatory deadlines.
Comment: Commenter (6362) suggests that the regulatory text in §30.4 be clarified. The commenter states that §30.4 appears to apply to EPA's use of studies (or other regulatory science) relied upon when EPA takes any final agency action. The commenter states that, in those instances, EPA should make all such studies available to the public to the "extent practicable." The commenter states that EPA should provide greater clarity regarding what it intends to do in circumstances where raw data cannot be made publicly available.
Response: The EPA has altered the final rule to increase clarity and address the commenter's concern. The EPA has removed the term "science that serves as the basis for informing a significant regulatory action" from 40 CFR 30.1, and will not require that science that serves as the basis for informing a significant regulatory action is publicly available in a manner sufficient for independent validation. Instead, in the final rule, 40 CFR 30.4 requires that the EPA clearly identify all science that serves as the basis for informing a significant regulatory action, and make all such science that serves as the basis for informing a significant regulatory action publicly available to the extent permitted by law. The EPA clarifies that this requirement is consistent with the EPA's existing practice of making science that serves as the basis for informing a significant regulatory action available in the public docket as part of the rulemaking.
9.7 How Does What this Rule Would Require Compare to Requirements of Environmental Statutes? (Relates to Chapter 1)
9.7.1 Best Available Science
Comment: Commenters (0671, 2249, 4827, 6188, 6922, 9227) assert that the Proposal would violate EPA's statutory requirement to consider the best available science. Commenter (6188) states that the rule does not explain how the EPA will be able to fulfill its statutory duties without the ability to consider the best available science, which includes robust, peer-reviewed, and cutting-edge studies that rely upon protected data that cannot legally be made publicly available. 
Commenter (6922) states that numerous statutes that EPA is charged with implementing require that EPA rely on the best available science and health information. The commenter states that any final rule must ensure that EPA is able to conduct its decision-making in a manner consistent with these and similar statutory directives. The commenter states that Congress did not mention transparency or any other reason why EPA should exclude consideration of the best available science. The commenter states that, in the Proposal, EPA did not provide any reason to exclude consideration of epidemiological studies based on the quality of science or reliability of results. The commenter states that transparency concerns or any other goal declared by the Agency cannot override EPA's statutory obligation to consider the full range of peer-reviewed, sound scientific research that is available and relevant to its regulatory decisions. The commenter states that these concerns cannot interfere with the Agency's general obligations to conduct informed, reasoned decision-making. The commenter provides the following statutory citations as examples of this obligation: 33 U.S.C. § 1314(a)(1); 42 U.S.C. § 7408(a)(2); 15 U.S.C. § 2625(h); and 42 U.S.C. § 300g-1(b)(3)(A).
Commenters (2249, 4827) suggest that, if EPA does not use the best available science, it would not be carrying out the mission given to it by Congress to protect Americans and the environment. Commenter (4827) states that the proposal directly contravenes the comprehensive federal and state regulatory programs Congress envisioned when drafting the CAA of 1970. Commenter (4827) asserts that it reduces our public health legislation to mere declarations, as EPA would be severely delayed, if not rendered entirely unable, to establish future Standards using the "best available science."
Commenter (9227) asserts that EPA's statutory authorities generally require the agency to consider all available data when undertaking significant rulemakings. The commenter states that EPA's statutory authorities mandate a variety of requirements for what scientific information EPA must consider in rulemaking. The commenter states that, to take one example that appears in numerous statutes, including TSCA, CAA, SDWA, and the Endangered Species Act, Congress has often required agencies to act on the "best available science." The commenter states that, for an agency to comply with this obligation, the agency must at least consider all available scientific information. The commenter states that "best" means "of the most excellent, effective, or desirable type or quality." The commenter states that "available" means "able to be used or obtained." The commenter states that "science" means "the intellectual and practical activity encompassing the systematic study of the structure and behavior of the physical and natural world through observation and experiment." The commenter states that assessing which science is "best" requires consideration of the overall quality of the science, and the public availability of underlying data is, at best, one of many aspects that should inform that assessment of overall quality. The commenter states that EPA cannot reasonably elevate the interest in public availability of all underlying information above all other factors in assessing the "best available science." The commenter states that, textually, EPA's approach is unlawful.
Commenter (9227) asserts that an agency "cannot ignore available. . . information." The commenter states that numerous courts have indicated that a plaintiff or petitioner can establish a violation of the "best available science" requirement by "point[ing] to any scientific evidence that the agency failed to consider." The commenter states that "the best available data requirement. . . prohibits [an agency] from disregarding available scientific evidence that is in some way better than the evidence [it] relies on.'" The commenter states that "An agency does. . . have an obligation to deal with newly acquired evidence in some reasonable fashion." The commenter states that EPA's proposal will result in EPA precluding itself from considering certain studies that are "available," thus violating the requirement that EPA rely on the best available science. 
Commenter (9227) further asserts that the requirement that agencies use "best available" science or information often means that the agency must act even if the available science or information is imperfect. The commenter states that "even if the available scientific and commercial data were quite inconclusive, [the agency] may -- indeed must -- still rely on it" when the agency has a duty to act. The commenter states that "where the information is not readily available, we cannot insist on perfection." The commenter states that, just as the Courts have recognized that they cannot expect perfection, agencies cannot choose to ignore certain studies or sources of information based solely on whether the data is publicly available -- especially where the validity of those studies has been established using techniques that do not rely on public availability of underlying data. 
Commenter (9227) states that, in direct violation of statutory requirements to consider, for example, "any other information available" or "the latest scientific knowledge [that is] useful" or "best available science," the Proposal would prohibit EPA from considering relevant and high quality science whenever the underlying data for a study is not publicly available. The commenter states that, through the Proposal, EPA unlawfully tries to engraft an additional statutory requirement onto each of these statutes, requiring that to be considered a study's underlying data must be publicly available. The commenter states that, for EPA's Proposal to succeed, EPA must demonstrate that a study cannot be "other information available to the Administrator" or the "latest scientific knowledge useful in indicating" health or welfare effects or the "best available science," or any of a number of other statutory formulations if the underlying data is not publicly available. The commenter states that EPA's Proposal fails to do so, and it could not do so.
Commenter (9227) states that there are many reasons that underlying study data may not be available that have no bearing on the quality or validity of the study. The commenter states that these include legal restrictions or concerns about privacy (especially with respect to studies involving human subjects), confidentiality, CBICBI, or national security. The commenter states that, if this requirement were applied retroactively to existing studies, it may no longer be possible to make underlying data and models publicly available. The commenter states that EPA acknowledges these impediments in proposed section 30.9, which provides the Administrator with discretion -- but not an obligation -- to allow the agency to consider a study for which underlying data or models are not publicly available if he determines that public disclosure is infeasible. The commenter states that, where the Administrator fails to exercise his discretion to grant an exemption pursuant to proposed section 30.9, or where data or models are unavailable for reasons that do not satisfy the infeasibility standard, proposed section 30.5 would prohibit EPA from considering such studies, regardless of whether they meet the statutory criteria for consideration.
Commenter (9227) states that the only way that this prohibition could comport with EPA's statutory obligations is if a study for which underlying data is not available cannot be, for example, "other information available" or "the latest scientific knowledge [that is] useful" or "best available science" -- i.e., if the public unavailability of a study's underlying dose response data and models makes the study ineligible to meet these criteria, regardless of whether the study has been peer reviewed, is based on rigorous methodologies, or has been published in a leading journal, and regardless of the reason for the public unavailability. The commenter states that EPA makes no such demonstration -- nor could it. The commenter states that there is simply no support for such a proposition; to the contrary, all the evidence shows that studies may be "best available science," and certainly "other information available" regardless of whether the data underlying them is publicly available.
Response: The EPA clarifies that in this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available or not available in a manner sufficient for independent validation. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
Furthermore, this final rule does not interpret or apply the provisions of any environmental statutes, nor does it modify the Agency's interpretations of "best available science." The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. The EPA will then identify and consider "pivotal science in accordance with the provisions of this rule," unless the implementation of the rule conflicts with statutory requirements and associated implementing regulations. As this rule makes clear, if implementing this procedural rule would result in conflicts with existing environmental statutes, and their implementing regulations, this rule will yield to the EPA statutes and regulations.
9.7.2 Data Quality Act
Comment: Commenter (6137) asserts that EPA's Proposed Rule ignores requirements set forth in the Data Quality Act. The commenter states that, given that the Proposed Rule eliminates from consideration any scientific study where the underlying data cannot be made publicly available, it undoubtedly precludes the use of the best available science in certain situations and, thus, the Proposed Rule is inconsistent with EPA's implementation of the Data Quality Act.
Commenter (6168) states that EPA should provide an analysis of how this rule differs from the Data Quality Act and related policies and then offer a justification for this rule considering that analysis. The commenter states that it is unclear why the proposed rule is necessary given these policies or how the proposed rule would relate to them, especially given that the Data Quality Act and its associated guidelines cover much of the same ground. The commenter states that it is unclear if, or why, this proposed regulation is meant to supersede existing legislation and policies. The commenter states that EPA should provide an analysis of how this rule differs from the Data Quality Act and related policies and then offer a justification for this rule considering that analysis. The commenter states that, without clarity on these issues, the proposed rule appears to not only be an attempt to regulate where legislation has failed (i.e. the HONEST Act), but to use a regulation to supersede legislation that is already on the books.
Commenter (9227) states that, because Congress expressly granted OMB the authority to set guidelines for data quality and instructed agencies like EPA to follow OMB's lead, EPA lacks statutory authority to adopt a regulation that is contrary to OMB's guidelines. The commenter states that, accordingly, EPA's proposed regulation violates the Information Quality Act (IQA) and must be withdrawn. The commenter adds that EPA's proposed rule is unlawful because it exceeds EPA's authority under Section 515(a) of the IQA. The commenter states that the IQA requires EPA promulgate data quality guidelines that are consistent with those promulgated by OMB. The commenter states that, contrary to EPA's assertion in the preamble to the proposal, the Proposed Rule is not consistent with OMB's data quality regulations. 
Response: The EPA clarifies that in this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available or not available in a manner sufficient for independent validation. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
This final rule makes incremental steps towards increasing data transparency by codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation and thereby improving the public's access to the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. Therefore, this rule is consistent with the focus on transparency in OMB Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and Integrity of Information Disseminated by Federal Agencies and OMB M-13-13. It is also consistent with OMB M-19-15, which highlights the need to characterize the sensitivity of an agency's conclusions to analytic assumptions, as well as other federal guidance documents that require greater data transparency. 
Furthermore, this final rule does not interpret or apply the provisions of any environmental statutes, nor does it modify the Agency's interpretations of "best available science." It therefore cannot conflict with the Data Quality Act. The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. The EPA will then identify and consider "pivotal science in accordance with the provisions of this rule," unless the implementation of the rule conflicts with statutory requirements and associated implementing regulations. As this rule makes clear, if implementing this procedural rule would result in conflicts with existing environmental statutes, and their implementing regulations, this rule will yield to the EPA statutes and regulations.
9.7.3 Other
Comment: Commenters (5165, 6125, 6925) assert that EPA suggests in footnote 3 of the proposal that it would exercise its discretionary authority to establish a policy that would exclude studies from consideration, such as the Harvard School of Public Health "Six Cities" epidemiological study. Commenters (5165, 6925) express concern that EPA would discard rigorously vetted scientific literature such as the Six Cities Study, withdrawing from its legal obligation and stated intention to rely on the best available science. Commenter (8272) states that failure to consider the Harvard Six Cities Study and the ACS study. and others like them in setting NAAQS because their data are not publicly available would conflict with the requirement that EPA set standards that protect susceptible subgroups.
Commenter (6125) states that none of the statutes cited in the proposal provide EPA with the discretionary authority to disregard such data. The commenter adds that the cases cited in the footnote involved two of the most consequential EPA regulations, namely the NAAQS for lead and for fine particles. The commenter states that the studies at issue were the Lanphear study for lead and the Harvard Six-City and ACS studies for fine particles. The commenter states that EPA is surely aware of the extensive history involving industry challenges to those studies. The commenter states that in the two cited cases, industry had challenged EPA's reliance on those studies, arguing in each case that the underlying data should have been disclosed. The commenter states that, in each case, EPA addressed the comments on their merits, finding, with support in the record, that the studies were reliable as they stood despite the criticisms. The commenter states that both courts agreed that there was no need for disclosure. The commenter states that the current proposal claims discretionary authority--for which it identifies no basis--to ignore those considered responses in favor of categorically disregarding such repeatedly examined studies. 
Response: The EPA clarifies that in this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available or not available in a manner sufficient for independent validation. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
Comment: Commenter (6133) states that the Proposal would broadly prohibit EPA from considering studies, models, and analyses that are integral to the functioning of EPA regulatory programs, implementation of statutes like the CAA, and protection of public health and the environment. The commenter states that it is unlawful for EPA to refuse or fail to consider these additional studies, models, and analyses. 
Response: The EPA clarifies that in this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA intends to issue implementation guidance and statute-specific rulemakings that will further describe these criteria and how the EPA will identify pivotal science in its assessments and rulemakings. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available or not available in a manner sufficient for independent validation. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
 Chapter 10: Applicability of the Rule
The proposed rule includes 40 CFR 30.3 which states:
      The provisions of this subpart apply to dose response data and models underlying pivotal regulatory science that are used to justify significant regulatory decisions regardless of the source of funding or identity of the party conducting the regulatory science. The provisions of this section do not apply to physical objects (like laboratory samples), drafts, and preliminary analyses. Except where explicitly stated otherwise, the provisions of this subpart do not apply to any other type of agency action, including individual party adjudications, enforcement activities, or permit proceedings.
As described in the preamble, the proposal solicits comment on (1) whether and to what extent the 40 CFR 30.3 requirements should apply to other stages of the rulemaking process, including proposed rules and guidance; (2) whether a narrower scope of coverage would be appropriate and whether certain categories of regulations should be excluded from coverage, such as individual party adjudications, enforcement activities, or permit proceedings; (3) whether the Agency should apply the provisions of the proposed rule to these actions or to specific types of actions within these categories; and (4) whether other agency actions, beyond significant final regulatory actions under Executive Order 12866, should be included, such as site-specific permitting actions or nonbinding regulatory determinations.
Several comments were received regarding the extent to which the 40 CFR 30.3 requirements should apply. In this section these comments are organized according to the following topics:
       Limiting Requirements to "Significant Regulatory Actions"
       Expanding the Scope of the Applicability of the Rule
       Narrowing the Scope of the Applicability of the Rule
       Changes in the Scope of the Science Subject to the Rule
       Scope - Phase-in
       Scope - Effects on Individual EPA Programs and Exceptions
       40 CFR 30.3  -  How do the provisions of this subpart apply?
       "Regulatory Decisions" Definition at 40 CFR 30.2
The specific comments provided in this Chapter reflect the comment letters that provide arguments supported by detailed rationale, examples, and references/documentation. The Docket ID numbers for these commenters are cited throughout this Chapter. In addition, some general comment letters may also include comments related to comments included and responded to in this Chapter. These comments are summarized in Appendix B. Note that points and issues raised in these general comment letters may also be raised by the more detailed comments summarized in this Chapter. The responses provided in this Chapter adequately address these more general comments.
10.1 Limiting Requirements to "Significant Regulatory Actions"
Comment: Commenters (4882, 6102, 6141, 6149, 6175, 6179, 6362, 6363, 6373) support the proposed rule being limited to "significant regulatory actions." 
Commenter (6141) states that, by limiting the application of scientific transparency to this smaller subset of EPA regulatory actions, the Proposal ensures that the enhanced requirements for science transparency will apply in cases where EPA regulatory actions having the greatest impacts on the U.S. economy and regulated industry sectors. The commenter states that enhancing public access to underlying scientific data in the case of the most important and impactful EPA rulemakings will lead to increases in the true net benefits of these EPA regulatory actions by helping to ensure that the policies and requirements established by EPA in those regulatory actions are based on valid science and not on published studies with irreproducible results. 
Response: The EPA agrees that the Agency's significant regulatory actions can potentially impact American lives and livelihoods. Because of this, the American people deserve environmental decisions and policies that are based on the best information, and only through continuous improvement to its procedures, especially those focused on transparency, can the EPA truly demonstrate its commitment to the Agency's mission of protecting public health and the environment through sound policy decisions that are informed by robust scientific and technical research. 
In 2020, the EPA issued an SNPRM and proposed to expand the scope of the rulemaking to include final influential scientific information. Based on a consideration of public comments on the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments.
Comment: Commenter (6102) suggests that, over time, EPA can consider expanding the scope of the types of actions covered by this rulemaking through an amendment to the rule once stronger data sharing mechanisms are in place and operational. 
Response: In 2020, the EPA issued an SNPRM and proposed to expand the scope of the rulemaking to include influential scientific information. Based on a consideration of public comments on the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. By finalizing the applicability of this rulemaking to influential scientific information, the Agency has expanded the scope not only to an important category of scientific assessments but also to a wider range of EPA regulatory actions.
The EPA would like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenter (6149) requests that the category of radiation protection be exempted from the proposed rule, or the proposed definition of "pivotal regulatory science" should be limited solely to regulatory actions that are used as a basis for "significant regulatory action" as defined by Executive Order 12866 or for implementing/modifying a "major rule" as defined by 5 U.S.C. 804, Congressional Review of Agency Rulemaking.
Commenter (6149) states that various federal and state agencies implement radiation protection standards within an interconnected regulatory framework from a common scientific basis. In addition to the EPA, federal authority for radiation protection resides with the U.S. Nuclear Regulatory Commission (USNRC), the U.S. Department of Energy (DOE), the U.S. Department of Defense (DOD), the Occupational Safety and Health Administration (OSHA), and the U.S. Food and Drug Administration (FDA). State radiation control programs exercise regulatory authority for protection from ionizing radiation as defined in the Atomic Energy Act (AEA) through federal and state statutes. Recognizing that the EPA is not the sole regulatory authority for radiation protection, the commenter contends that it is critical that any significant changes in the associated integrated regulatory framework be coordinated and harmonized to ensure regulatory certainty and consistency. The commenter states that this is supported by the guidance in Executive Order 12866 which states, "Each agency shall avoid regulations that are inconsistent, incompatible, or duplicative with its other regulations or those of other Federal agencies."
The commenter (6149) notes that the linear no-threshold (LNT) radiation dose-response relationship is the foundation for the system of radiation protection, as well as federal and state laws and regulations. According to the commenter, any changes in its application would significantly impact radiation protection guidance, medical practice, compensation programs, environmental contamination issues, advances in technology, and communication of risk with profound corresponding safety, compliance, and financial impacts to both the private and public sectors. The commenter asserts that the National Academies, National Research Council, and National Council on Radiation Protection and Measurements (NCRP) have provided effective methods for a rigorous, holistic, and consensus review of the science behind radiation protection from all stakeholders. Thus, based upon these existing processes, the commenter recommends that the category of radiation protection be exempted from the proposed rule. At a minimum, the commenter recommends that the definition of "pivotal regulatory science" be limited to regulatory actions that are used as a basis for "significant regulatory action" as defined by Executive Order 12866 or for implementing/modifying a "major rule" as defined by 5 U.S.C. 804, Congressional Review of Agency Rulemaking.
In summary, the commenter (6149) believes the radiation protection category of regulations should be exempted from the Proposed Rule on Strengthening Transparency in Regulatory Science based upon the unique nature of the extensive - interconnected radiation protection system, the limited availability of radiation epidemiological and scientific expertise, and the demonstrated ability of the Congressionally chartered scientific advisory organizations (the National Academies, National Research Council, and NCRP) to effectively develop and review the scientific bases for radiation protection regulatory actions in an open and transparent manner.
Response: In 2020, the EPA issued an SNPRM and proposed to expand the scope of the rulemaking to include influential scientific information. Based on a consideration of public comments on the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. "Significant regulatory actions" means final regulations determined to be "significant regulatory actions" by the Office of Management and Budget (OMB) pursuant to Executive Order 12866. Influential scientific information means scientific information the Agency reasonably can determine will have or does have a clear and substantial impact on important public policies or private sector decisions. This action establishes how the Environmental Protection Agency (EPA) will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information. The provisions of this part do not apply to physical objects (like laboratory samples), drafts, and preliminary analyses, and influential scientific information or pivotal science that meet one or more of the exemptions identified in Section IX of the OMB Final Information Quality Bulletin for Peer Review. In the event the procedures outlined in this part conflict with statutes the EPA administers, or their implementing regulations, the statutes and regulations will control. Except where explicitly stated otherwise, the provisions of this part do not apply to any other type of Agency action, including individual party adjudications, enforcement activities, site-specific actions, or permit proceedings. Unless specifically exempt, this final rule will apply regardless of the scientific issue considered.
10.2 Expanding the Scope of Applicability of the Rule
Comment: Commenters (4882, 5181, 6112, 6115, 6133, 6141, 6143, 6148, 6161, 6360, 6362, 6366, 6375, 6867, 6875, 6891, 6906, 6921, 6935, 9227) support applying the proposed rule requirements more broadly and offer suggestions on expanding the scope of the rule. Commenter (6891) states that limiting transparency obligations only to significant regulatory actions ignores that EPA's actions outside of the rulemaking context can often have significant impacts on those other regulatory decisions. 
Commenters (6133, 9227) argue that limiting the proposal to "significant regulatory actions" would result in treating a study differently depending on whether it supports regulation or non-regulation in a particular context. Commenter (6133) asserts that it is a hallmark of the Proposal's inherent arbitrariness and capriciousness that the Proposal prohibits EPA from considering the identical regulatory science, studies, and information in some regulatory situations, while allowing EPA to consider the identical regulatory science, studies, and information in other regulatory situations -- based merely upon the type of situation, rather than any differences in availability, replicability, verifiability, or validation concerning the information. 
Specific suggestions provided by commenters for expanding the scope of the rule are summarized in sections 7.2.1 through 7.2.7 of this Chapter and include: other regulatory actions; guidance; proposed rulemaking/advanced notice of proposed rulemaking; individual party adjudications, enforcement activities, permit proceedings; actions relying on records from previous reviews; specific EPA programs, and application of the rule to a limited and special class of laboratory samples.
Response: In 2020, the EPA issued an SNPRM and proposed to expand the scope of the rulemaking to include influential scientific information. Based on a consideration of public comments on the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. By finalizing the applicability of this rulemaking to influential scientific information, the Agency has expanded the scope not only to an important category of scientific assessments but also to a wider range of EPA regulatory actions.
The EPA is committed to its mission of protecting human health and the environment through sound policy decisions that are informed by robust scientific and technical research. Because of the potential impact of the EPA's significant regulatory actions and influential scientific information on American lives and livelihoods, the American people deserve environmental decisions and policies that are based on the best scientific information. Only through continuous improvement to its procedures, especially those focused on transparency, can the EPA fully demonstrate that commitment.
The provisions of final rule do not apply to physical objects (like laboratory samples), drafts, and preliminary analyses, and influential scientific information and/or pivotal science that meet one or more of the exemptions identified in Section IX of the OMB Final Information Quality Bulletin for Peer Review. In the event the procedures outlined in this part conflict with statutes that EPA administers, or their implementing regulations, the statutes and regulations will control. Except where explicitly stated otherwise, the provisions of this final rule do not apply to any other type of agency action, including individual party adjudications, enforcement activities, site-specific actions or permit proceedings.
Finally, because of the potential impact of the EPA's significant regulatory actions and influential scientific information on American lives and livelihoods, this rule focuses on these important Agency actions. The EPA disagrees with commenters who believe that this rule requires the Agency to treat science differently based on the situation. Rather this rule does not categorically exclude any pivotal science, but because of the potentially significant societal impact of significant regulatory actions and influential scientific information requires the Agency to consider the availability of the underlying dose-response data to better ensure the validity of the science.
10.2.1 Application of the Rule to Other Regulatory Actions 
Comment: Commenters (1972, 4882, 6141, 6148, 6362, 6375) support expansion of the rule requirements beyond "significant regulatory actions" to all EPA programs and stages of rule development; scientifically or technically novel actions (2426, 6161, 6375); interpretive rules (6141); regulatory and other key agency decision-making support assessments/studies (4882, 6112, 6115, 6125, 6141, 6143, 6148, 6161, 6360, 6369, 6375, 6867, 6891, 6906, 6911); pre-rulemaking planning activities (6375); Agency policy documents (6366); and models used in studies supporting regulations/policies (4882, 6112, 6115, 6143, 6360, 6375, 6867, 6875, 6891, 6935). In general, these commenters assert that there is no reason that there should be a distinction made between the application of the proposed rules provisions to major rules and other regulatory actions regarding the consideration of the science and modeling done in the development of such actions. 
Commenter (6125) contends that allowing a distinction between using studies when they confer a benefit on industry but disallowing them when used to influence public health protection would be arbitrary and invidious. 
Commenter (6360) argues that, since EPA's application of science always involves judgement and since this judgement is an exercise of government authority, all EPA applications of science should have minimum transparency standards. The commenter contends that there is no objective principle to separate permits, cleanup decisions, regulations, or other uses of EPA authority to arrive at the conclusion that one community deserves the ability to understand the government's judgement and reasoning and another does not. The commenter states that EPA commonly employs fate-and-transport models, route of exposure models, cost estimation models, statistical techniques to project analytic data beyond method detection limits, and numerous other specific applications of science that involve judgement by EPA and by the application developers, and that there is no logical principle to require transparency for one application and not another. As an example, the commenter asserts that a fate and transport model could have as much economic impact as a dose-response model. 
Commenter (6360) states that, when EPA introduces a new applied science tool, the requirements of this rulemaking should apply, no matter the significance level or impact of the decision. The commenter asserts that EPA generally has discretion  -  judgement  -  as to when it introduces a new scientific application. According to the commenter, if the applicability is limited only to economically significant regulatory actions or some other impact threshold, this tiering approach creates a perverse incentive to introduce new, non-transparent tools in minor regulatory actions, permits, or product approvals. The commenter suggests that, once scientific applications are established in this manner, these applications may be granted deference as EPA policy and its precedent.
Response: Based on a consideration of these and other similar comments, the EPA proposed, through a 2020 SNPRM, to expand the scope of the 2018 proposed rulemaking in two ways: (1) expanding "dose-response data and models" to "data and models," and (2) including influential scientific information. As EPA noted in the 2020 SNPRM, transparency of EPA's science should not be limited to dose-response data and dose-response models, because other types of data and models can also drive the requirements and/or quantitative analysis of EPA significant regulatory actions and influential scientific information.
Based on the comments on the 2018 proposed rule and the 2020 SNPRM, including that the number of studies that would be subject to the rule might hinder rulemaking, the EPA determined that the Agency should pursue an incremental approach to maximizing transparency in the science that it relies upon by focusing the final rule requirements on dose-response data and, in particular, only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs).
The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. The provisions of the final rule apply to science that serves as the basis for informing a significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science. 
The provisions of final rule do not apply to physical objects (like laboratory samples), drafts, and preliminary analyses, and influential scientific information and/or pivotal science that meet one or more of the exemptions identified in Section IX of the OMB Final Information Quality Bulletin for Peer Review. In the event the procedures outlined in this part conflict with statutes that EPA administers, or their implementing regulations, the statutes and regulations will control. Except where explicitly stated otherwise, the provisions of this final rule do not apply to any other type of agency action, including individual party adjudications, enforcement activities, site-specific actions or permit proceedings.
Further, the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
10.2.2 Application of the Rule to Agency Guidance
Comment: Commenters (2426, 6141, 6148, 6161, 6366, 6375, 6906, 6911, 6913) urge the Agency to apply the requirements to EPA guidance. 
Commenters (2426, 6366, 6913) assert that guidance documents can have as much impact on the regulated community as the "significant regulatory actions" discussed in the proposal. Commenters (2426, 6913) state that allowing this opportunity for public involvement safeguards the larger goal of regulatory transparency, as operational guidelines and guidance documents can often be just as impactful as official regulatory actions or rules. 
Commenter (6161) states that transparency principles should be applied to all science used in developing guidance. Commenter (2426) states that the proposed data transparency rule should also apply to scientifically or technically novel guidance, and the resulting guidance document should be made available for public comment. 
Response: The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. Except where explicitly stated otherwise, the provisions of the final rule part do not apply to any other type of agency action. By finalizing the scope to include influential scientific information, the Agency is applying the applicability of the rule to an important category of scientific assessments that influence a wide range of EPA actions.
Comment: Commenter (6906) states that, due to the vast range of Guidance that EPA produces and utilizes, it would not be practical to require the Agency to enforce that these Guidelines be met for all Guidance Documents. 
Response: EPA agrees with the commenter and is not expanding the scope of the rule to cover guidance documents. The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. Except where explicitly stated otherwise, the provisions of the final rule part do not apply to any other type of agency action. By finalizing the scope rule to include influential scientific information, the Agency is applying the applicability of the rule to an important category of scientific assessments that influence a wide range of EPA actions. Except in those instances where this rule yields to an existing environmental statute, all future influential scientific information will be subject to the requirements of this rule. Though this final rule does not apply directly to guidance documents, all new guidance will benefit from this rule if those documents rely on new influential scientific information.
Comment: Commenter (5181) expresses opposition to applying the requirements of the proposed rule to promulgation of guidance. The commenter states that applying such strenuous requirements for data early in the development of a rule would restrict initial consultation with other agencies, with stakeholders, and with the public. According to the commenter, in environmental programs, guidance often supports implementation of a rule in a practical manner at the field level. The commenter states that practical, accurate, and efficient application of regulations in the field typically depends upon guidance based not only on published science, but also on professional experience and results of field testing. The commenter asserts that it would be highly impractical to make such information available to the public in the format described by the proposed rule. According to the commenter, failure to develop guidance because of lack of extensive published data would cripple implementation of necessary regulatory programs. 
Response: This final rule does not apply to guidance documents. The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. Except where explicitly stated otherwise, the provisions of the final rule part do not apply to any other type of agency action. By finalizing the scope rule to include influential scientific information, the Agency is applying the applicability of the rule to an important category of scientific assessments that influence a wide range of EPA actions.
10.2.3 Application of the Rule to Proposed Rules, Advanced Notice of Proposed Rulemaking and All Aspects of the Rulemaking Process
Comment: Commenters (4932, 6143, 6148, 6151, 6177, 6360, 6366, 6375, 6867, 6906, 6913, 6921) urge the Agency to apply the requirements of the proposed rule to proposed rulemakings. 
Commenter (6360) states that transparency at the proposal stage benefits EPA by helping build a robust administrative record to support its policy decisions.
Commenter (6143) recommends that requirements apply first to proposed regulations for which rulemaking is already in progress, then prospectively to proposed and final rules alike.
Commenters (4932, 6151, 6177, 6366, 6913, 6921) assert that deferring access to the science until the final rulemaking is issued will be too late for active public engagement. Commenters (6913, 6360, 6921) state that policy decisions and scientific interpretations need to be thoroughly explained in the proposed rule stage, so that the public can understand and provide an informed opinion on them, which allows EPA to benefit from transparent stakeholder input before rendering a final agency action.
Commenter (6366) states that, when impacted stakeholders and the public are provided an opportunity to comment on federal proposals, they should have all relevant data, including cost benefit data, available to ensure they fully understand the justification for the rule and that their comments are clearly in response to the selected regulatory option(s) being proposed. 
Commenters (6177, 6360, 6921) suggest that additional information may become available as a result of broadening the scope to the proposed rule stage. Commenter (6177) states that broadening the scope to the proposed rule stage would allow third parties to review the scientific underpinnings early in the process and possibly provide additional data sets and studies generated that the Agency may not currently possess. Alternatively, according to the commenter, if EPA only allows access to data sets once the rule is final, interested third parties would be more limited in their options to seek redress if a substantial issue is found.
Commenter (6906) states that it would be useful for the Agency to include language that regulators "strongly consider," rather than mandate, the use of these transparency guidelines during other moments of the regulatory process (i.e. the creation of guidance). The commenter asserts that, by requesting consideration of the guidelines at earlier stages in the proposed rule process, enforcement of these provisions for Final Rulemakings should be smoother. 
Response: The EPA sees no need to include the proposed rule stage of significant regulatory actions in the final regulatory text for this rulemaking because, as a practical matter, these proposed rules must comply with this final rule in order to be finalized. The EPA does not introduce the studies and analyses it relies on for a rulemaking at the final rule stage. The basis for a rulemaking is provided for public review and comment in the public docket when the proposed rule is issued or if subsequently added to the docket through a separate opportunity for public comment. The EPA expects to identify pivotal science in proposed significant regulatory actions or external review drafts of influential scientific information, which will allow the public, if they so choose, to independently validate the pivotal science and provide comment to the EPA.
Comment: Commenters (4932, 6867, 6375, 6906) recommend revisions to the regulatory text in support of their positions that the proposed rule apply to both advanced notice of proposed rulemakings and proposed rules.
Commenter (4932) recommends that EPA revise the definitions of "regulatory decisions," "pivotal regulatory science," and "regulatory science" by striking the word "final" from the proposed definitions so that the transparency objectives (i.e., the disclosure of pivotal regulatory science) apply to advanced notices of proposed rulemaking and proposed rulemakings. 
Commenter (6867) states the EPA should identify all studies (or other regulatory science) it will rely upon at the time it proposes any regulation in order for the public to meaningfully comment on the proposed action. The commenter suggests that Proposed 40 CFR 30.4 be amended to state that:
      EPA shall clearly identify all studies (or other regulatory science) relied upon to support any agency action. When EPA proposes any agency action, EPA shall make all such studies (or other regulatory science) available to the public to the extent practicable.
Commenter (6906) recommends that the EPA amend 40 CFR §§ 30.4 (identification of studies), 30.5 (disclosure of data and models), 30.6 (documentation and evaluation of studies) and 30.7 (peer review) to require these steps at an interim phase prior to issuance of the final rule. 
Response: The EPA sees no need to include the proposed rule stage of significant regulatory actions in the final regulatory text for this rulemaking because, as a practical matter, these proposed rules must comply with this final rule in order to be finalized. The EPA does not introduce the studies and analyses it relies on for a rulemaking at the final rule stage. The basis for a rulemaking is provided for public review and comment in the public docket when the proposed rule is issued or if subsequently added to the docket through a separate opportunity for public comment. The EPA expects to identify pivotal science in proposed significant regulatory actions or external review drafts of influential scientific information, which will allow the public, if they so choose, to independently validate the pivotal science and provide comment to the EPA.
The EPA also sees no need to include advance notices of proposed rulemakings in this final rule because advance notices of proposed rulemakings are preliminary in nature and frequently focus on soliciting initial comment on a regulatory issue or approach. Therefore, an expansion to advance notices of proposed rulemaking would be inconsistent with the purpose of this final rule.
Finally, based on a consideration of the public comments on both the 2018 proposed rule and the 2020 SNPRM, the EPA has revised many of the rules definitions (at 40 CFR 30.2), including the definitions of "regulatory decisions," "pivotal regulatory science," and "regulatory science."
Comment: As background, the commenter (5181) explains that the proposed rule language indicates that it is generally applicable prospectively to final agency actions. However, the commenter states that the proposed rule requests comments on applicability to other stages of rulemaking. The commenter states that the EPA is currently engaged in a very extended proposed modification to the definition of waters of the United States (WOTUS) and it is unclear how the proposed rule might affect current rulemaking efforts. According to the commenter, although the definition of "dose response data and models" may not apply directly to the WOTUS rule, the Federal Register notice indicates that EPA is considering expansion of the rule "to cover other types of data and information, such as for example economic and environmental impact data and models that are designed to predict the costs, benefits, market impacts and/or environmental effects of specific regulation interventions on complex economic or environmental systems." The commenter recommends that, given the in-depth scientific analysis undertaken to support development of the 2015 rule regarding WOTUS  -  based heavily on a publicly available analysis of the pertinent peer-reviewed literature, they recommend that the WOTUS rule be exempt from any application of the proposed rule regarding transparency in science and that completion of the WOTUS rule be allowed using the existing scientific analysis. The commenter asserts that applying the proposed rule to the information gathered to support a WOTUS rule is unnecessary given currently availability of the underlying science to the public.
Response: This final rule applies prospectively to significant regulatory actions and influential scientific information developed after the effective date of the final rule; therefore, the rule does not apply to the WOTUS rule. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. 
Comment: Commenters (6112, 6143, 6375) contend that these requirements should apply to all aspects and steps in the pre-rulemaking and rulemaking process. Commenter (6112, 6143) specifically suggests that the proposed rule requirements should apply to assessments (including certain retrospective assessments), models, criteria documents, and regulatory impact analyses that provide the basis for EPA's policies, procedures, guidance, and proposed and final regulatory decisions. Commenter (6375) requests that the scope be expanded to cover, at a minimum, pre-rulemaking activities (e.g., during the planning and/or assessment stages of a National Ambient Air Quality Standard (NAAQS) cycle, EPA should indicate on which studies it plans to rely). 
Commenter (1972) requests that the EPA follow the proposed requirements to the fullest extent practicable for all rulemaking decisions that rely on scientific or economic information as a basis for EPA's decision.
Commenter (4882) asserts that transparency is just as important in all aspects of rule-making as it is in regulatory science.
Response: Based on a consideration of these and other similar comments, the EPA proposed, through a 2020 SNPRM, to expand the scope of the 2018 proposed rulemaking in two ways: (1) expanding "dose-response data and models" to "data and models," and (2) including influential scientific information. The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. The EPA will not expand the applicability of this rulemaking beyond significant regulatory actions and influential scientific assessments to individual party adjudications, permit proceedings, site-specific actions, and enforcement actions because these actions are typically focused on individual regulated entities. Based on the comments on the 2018 proposed rule and the 2020 SNPRM, including that the number of studies that would be subject to the rule might hinder rulemaking, the EPA determined that the Agency should pursue an incremental approach to maximizing transparency in the science that it relies upon by focusing the final rule requirements on dose-response data and, in particular, only those studies that are integral to characterizing dose-reponse relationships (e.g., identifying candidate PODs).
10.2.4 Individual Party Adjudications, Enforcement Activities, or Permit Proceedings
Comment: Commenters (6112, 6115, 6143, 6375) provide arguments that support application of the proposed requirements to individual party adjudications, enforcement activities, and permit proceedings. Commenter (6143) recommends that the proposed requirements apply on a case-by-case basis to individual party adjudications, enforcement activities, and permit proceedings, to the extent that such actions are based or depend on research information. 
Response: The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. The EPA will not expand the applicability of this rulemaking beyond significant regulatory actions and influential scientific assessments to individual party adjudications, permit proceedings, site-specific actions, and enforcement actions because these actions are typically focused on individual regulated entities.
Comment: Commenter (6935) supports proposed provision 40 CFR 30.3 which indicates that the regulation does not apply to Agency action involving individual party adjudications, enforcement activities, or permit proceedings. The commenter states that it is not clear that the proposed new regulations should be applicable to individual party adjudications, enforcement activities, or permit proceedings, unless reliance on a particular study is a central issue. The commenter explains that adjudications and enforcement activities tend to turn on questions of fact and current processes can uncover necessary details without adding a new layer of requirements, which they assert would slow down such actions, while creating uncertainty and adding unnecessary costs for both regulated parties and regulators. The commenter suggests that even if reliance on a particular study is central to a more individualized proceeding, it would be more appropriate for the Agency to conduct a more generic regulatory action to address potential deficiencies in a study -- and the resulting regulatory requirements -- than require a private party to expend its own resources to assess and validate the underlying science. 
Commenter (6137) asserts that, to the extent EPA decides to apply this Proposed Rule in the context of enforcement, adjudicatory, and permit actions  -  which it should not  -  it must first issue a proposal setting out its basis for such a rule and provide opportunity for comment, separate from the April 30, 2018 proposal, under the Administrative Procedure Act (APA) and Clean Air Act (CAA) § 307(d). 
Response: The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. The EPA will not expand the applicability of this rulemaking beyond significant regulatory actions and influential scientific assessments to individual party adjudications, permit proceedings, site-specific actions, and enforcement actions because these actions are typically focused on individual regulated entities.
Comment: Commenter (6191) requests that EPA define what they would consider "practical" with respect to application of the rule to individual party adjudications, enforcement activities, or permit proceedings
Response: The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. The EPA will not expand the applicability of this rulemaking beyond significant regulatory actions and influential scientific assessments to individual party adjudications, permit proceedings, site-specific actions, and enforcement actions because these actions are typically focused on individual regulated entities.
10.2.4.1 Individual Party Adjudications
Comment: Commenter (6133) states that the Proposal arbitrarily and capriciously allows EPA to continue to rely on highly relevant regulatory science and other information supplied to the agency by industry during individual party adjudications, even when that information is not "publicly available in a manner sufficient for independent validation." According to the commenter, the EPA could consider non-peer reviewed, non-transparent industry science to conclude that formaldehyde or asbestos are not carcinogens, or that fine particles (PM2.5) or lead have safe exposure levels, or that carbon dioxide (CO2) does not endanger public health or welfare. The commenter states that, under the proposed 40 CFR 30.3, EPA could conclude, after considering the non-transparent, unavailable information, that the regulations should not apply in ways that would affect an entire industrial sector favorably, while harming the public meant to be protected by those regulations. 
Response: The final rule does not prohibit EPA from relying on studies when the underlying dose-response data is not available for independent validation. Implementation of this rule will not undermine EPA's efforts in protecting human health and the environment. The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. The EPA will not expand the applicability of this rulemaking beyond significant regulatory actions and influential scientific assessments to individual party adjudications, permit proceedings, site-specific actions, and enforcement actions because these actions are typically focused on individual regulated entities.
Comment: Commenter (6137) states that EPA does not explain how or where its Proposed Rule could be relevant in individual party adjudications. The commenter asserts that, though the administrative penalty provisions of the CAA provide for individual party adjudications, EPA's proposed procedures bear no relevance or applicability to such adjudications. 
Response: The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. The EPA will not expand the applicability of this rulemaking beyond significant regulatory actions and influential scientific assessments to individual party adjudications, permit proceedings, site-specific actions, and enforcement actions because these actions are typically focused on individual regulated entities.
Comment: Commenter (6935) states that, to the extent that an individual adjudication, whether in the context of permitting or enforcement, relies on EPA's own models or those provided by intervenors, the Agency should provide complete and transparent access to these models to the private party so that it can assess and critique any outcomes. 
Response: The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. The EPA will not expand the applicability of this rulemaking beyond significant regulatory actions and influential scientific assessments to individual party adjudications, permit proceedings, site-specific actions, and enforcement actions because these actions are typically focused on individual regulated entities.
10.2.4.2 Enforcement Activities
Comment: Commenter (6133) asserts that the proposal arbitrarily and capriciously allows EPA to continue to rely on highly relevant regulatory science and other information supplied to the agency by industry during enforcement proceedings, even when that information is not "publicly available in a manner sufficient for independent validation." The commenter provides for consideration an example where a company that receives a notice of violation from EPA meets with the agency to make the case that EPA and the Department of Justice should not file a complaint. The commenter states that a company may submit regulatory science, confidential business information (CBI) (CBI), or other non-confidential information that is not "publicly available in a manner sufficient for independent validation." The commenter asserts that the EPA could consider non-peer reviewed, non-transparent, erroneous industry science to conclude that formaldehyde or asbestos are not carcinogens, or that PM2.5 or lead have safe exposure levels, or that CO2 does not endanger public health or welfare. According to the commenter, considering this and other non-transparent information, EPA could conclude that prosecution is not warranted, or that the information represents mitigating factors for penalties or injunctive relief, notwithstanding that the non-transparent, unavailable information is scientifically erroneous and even absurd.
Response: The final rule does not prohibit EPA from relying on studies when the underlying dose-response data is not available for independent validation. The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. The EPA will not expand the applicability of this rulemaking beyond significant regulatory actions and influential scientific assessments to individual party adjudications, permit proceedings, site-specific actions, and enforcement actions because these actions are typically focused on individual regulated entities.
Comment: Commenter (6137) states that, with respect to enforcement actions, EPA has no authority to bar the courts or administrative adjudicators from considering relevant evidence merely because that evidence has not passed an arbitrary test for transparency and peer review. The commenter contends that, whether a scientific study is sufficiently transparent or has been peer reviewed provides no bearing on administrative enforcement of statutes and is wholly irrelevant to determining administrative civil penalties. The commenter provides that the CAA, for example, vests authority over judicial enforcement actions to the courts, not EPA, and EPA has no authority to dictate to the courts what evidence they can and cannot consider, or what weight to give such evidence. See Nat. Res. Def. Council v. EPA, 749 F.3d 1055, 1063-64 (D.C. Cir. 2014). The commenter asserts that courts and administrative law judges are equipped with tools to separate out credible and non-credible scientific evidence, see, e.g., Fed. R. Evid. 702; Daubert, 509 U.S. 579. According to the commenter, whether a scientific study is sufficiently transparent or has been peer reviewed provides no bearing on administrative enforcement of these statutes, which provide enforcement authority over whether there has been a violation of specified requirements and prohibitions, see, e.g., 42 U.S.C. § 7413(a)(1)-(5). The commenter argues that it is also wholly irrelevant to determining administrative civil penalties. See, e.g., id. § 7413(e). Though the CAA does allow consideration of "such other factors as justice may require" in setting civil penalties, the commenter asserts that the EPA does not and cannot explain how its transparency and peer review tests could possibly be relevant to whether "justice" requires a different penalty for violation of a prohibition or requirement. The commenter asserts that, if a defendant feels that a standard is unjust because of a lack of data transparency or peer review in supporting studies, the defendant can seek review in the Court of Appeals within 60 days of the standard's publication and that the validity of standards cannot be questioned in an enforcement proceeding. See, e.g., 42 U.S.C. § 7607(b)(2). 
Response: Nothing in the final rule bars courts or administrative adjudicators from considering relevant evidence. With this final rule, the EPA is including regulatory text in 40 CFR 30.3 stating that the statutes and regulations will control in the event of any conflicts, and clarifying that the requirements in this final rule set the overarching structure and principles for science transparency in the EPA's significant regulatory actions and influential scientific information. The EPA intends to develop internal implementation guidance and promulgate either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule, which will clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes that the EPA administers.
The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. The EPA will not expand the applicability of this rulemaking beyond significant regulatory actions and influential scientific assessments to individual party adjudications, permit proceedings, site-specific actions, and enforcement actions because these actions are typically focused on individual regulated entities.
Comment: Commenter (6356) recommends that the rule not apply to litigation and settlement/consent or to EPA enforcement actions, consent decrees, or administrative settlements because of the prosecutorial and confidentiality of discussions in the context of litigation and settlement agreements. 
Response: The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. The EPA agrees with the commenter and will not expand the applicability of this rulemaking beyond significant regulatory actions and influential scientific assessments to individual party adjudications, permit proceedings, site-specific actions, and enforcement actions because these actions are typically focused on individual regulated entities.
Comment: Commenter (6362) supports the application of the rule to enforcement action. The commenter states that in civil judicial enforcement programs, EPA routinely makes discretionary decisions targeting cases to pursue on the basis of scientific data on exposure of humans and ecological resources to pollutants and that the scientific data used in such cases should be publicly available to be verified and replicated. 
Response: The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. The EPA will not expand the applicability of this rulemaking beyond significant regulatory actions and influential scientific assessments to individual party adjudications, permit proceedings, site-specific actions, and enforcement actions because these actions are typically focused on individual regulated entities.
10.2.4.3 Permit Proceedings
Comment: Commenters (6180, 6362) state that the requirements should apply to permit proceedings. Commenter (6180) states that permit proceedings (and related water quality standards and total maximum daily loads (TMDLs)) have potentially long-lasting, widespread, and costly implications for permittees and therefore should not be excluded from this rule as they meet the requirements of pivotal regulatory science. Commenter (6362) contends that it should apply to permit proceedings because, in administrative programs, EPA routinely makes discretionary decisions targeting cases to pursue on the basis of scientific data on exposure of humans and ecological resources to pollutants and that the scientific data used in such cases should be publicly available to be verified and replicated. 
Response: The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. The EPA will not expand the applicability of this rulemaking beyond significant regulatory actions and influential scientific assessments to individual party adjudications, permit proceedings, site-specific actions, and enforcement actions because these actions are typically focused on individual regulated entities.
Comment: Commenters (5181, 6356) oppose application of the proposed requirements to individual permit actions. 
Commenter (5181) asserts that such requirements would be inconsistent with sound and timely authorizations under the section 404 dredge and fill permit program. The commenter states that regulations that subject all data to identical requirements regarding data collection, evaluation, and release would interfere with the permitting process in an unacceptable manner. According to the commenter, the exclusion of non-peer reviewed data - which is typically collected at the time of a permit application for the purpose of clarifying both the extent and limitations of adverse impacts  -  would undermine the accuracy of individual permit decisions. 
Commenter (6356) recommends that the rule, if adopted, should not apply to information that forms the basis of permits under any of the statutes such as the CAA, the Clean Water Act (CWA), or the Safe Drinking Water Act (SDWA) because separate public participation requirements provide under each of the respective statutes that the permit itself, including its requirements and conditions and their respective bases, already are subject to public comment and review. The commenter states that, while it is not clear what this rule would apply to other than these provisions, it seems possible that the applicability of this transparency regulation to other information underlying an environmental permit could open the door to confidential information about manufacturing processes and other business information and also third-party technology, which would be irrelevant to the permit's requirements and conditions and otherwise not required to be in the public's purview.
Response: The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. The EPA will not expand the applicability of this rulemaking beyond significant regulatory actions and influential scientific assessments to individual party adjudications, permit proceedings, site-specific actions, and enforcement actions because these actions are typically focused on individual regulated entities.
10.2.5 Application of the Rule to Actions Relying on Records from Previous Reviews
 10.2.5.1 Support Expanding the Scope to Prior Studies
Comment: Commenters (0555, 4882, 6143, 6362, 6375, 6867) support an expansion of the scope of the Proposal to prior studies. 
Commenter (0555) states that, when possible, EPA should apply the proposed new guidelines to existing regulations and the regulatory science that supports them. The commenter recognizes that this may not always be feasible, especially for regulatory science that was developed under conditions that would limit disclosure, and EPA should develop clear criteria for the types of research that would be eligible for exemptions to its transparency policy. The commenter states that EPA has an opportunity, when conducting retrospective evaluation of regulatory impacts (as required by Executive Orders 13563 and 13771), to reexamine the studies, data, models and assumptions that generated ex-ante estimates of regulatory benefits and costs in order to ensure transparency and opportunities for independent analysis. 
Commenter (6143) also recommends that the proposed requirements apply on a case-by-case basis to retrospective assessment of research information. 
Commenter (6362) suggests that, in cases where EPA has developed analytical tools and models in the past that incorporate dose response data, it may be valuable to apply this rule retrospectively. 
Commenter (6375) requests that a standard based on pivotal regulatory science that was not made publicly available, which has ongoing or upcoming economically significant costs, be subject to this rule. The commenter states that it would be inappropriate to reaffirm an existing standard without first reaffirming the underlying science. According to the commenter, if the underlying science was based on pivotal regulatory studies which have not been made publicly available, there would be no way to know if the standard was set correctly.
Response: This final rule applies prospectively to significant regulatory actions and influential scientific information developed after the effective date of the rule; the final rule has no retrospective effect on either final significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created.
10.2.5.2 Oppose Expanding the Scope to Prior Studies
Commenters (1008, 1011, 1012, 1281, 2006, 2426, 2477, 2876, 3971, 4595, 4838, 4931, 5165, 5170, 5172, 5181, 6109, 6114, 6116, 6120, 6126, 6133, 6361, 6362, 6373, 6447, 6449, 6868, 6875, 6895, 6905, 6925, 6935, 9224) oppose expanding the scope of the proposal with respect to prior studies and recommend that the rule only apply prospectively. Detailed comments supporting their positions are provided below.
10.2.5.2.1 Retrospective Implementation Would Lead to Significant Delays and Costs
Comment: Commenters (1008, 5181, 6895, 6935) assert that retrospective implementation could lead to delays and significant new costs that would lead to the EPA diverting substantial resources from its mission and a delay to the process of decision making. Commenter (6935) adds that, if previously implemented regulatory determinations were invalidated by a retrospective evaluation of the transparency in underlying scientific studies, then the investments made based on those regulatory decisions would be undermined. The commenter asserts that companies and their customers already will have made the investment to comply with a regulation and may have made corollary investments (e.g., if a regulation caused plant retirements and replacement generation is needed) that cannot be undone.
Response: This final rule applies prospectively to significant regulatory actions and influential scientific information developed after the effective date of the final rule; the final rule has no retrospective effect on either final significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created.
10.2.5.2.2 Retroactive Application of the Proposed Rule is Not Practical
Comment: Commenters (1011, 2426, 4595, 6126, 6447, 6895) assert that retroactive application of this proposed rule to existing rules is not practical. Commenter (2426) states that the Proposal is infeasible considering the volume of significant environmental rules already in place and the time and expertise it would take to reanalyze the technical and legal aspects of these rules while maintaining oversight of ongoing agency business. Commenter (6895) states that applying the proposed rule retroactively to health studies, potentially initiated decades ago, may be impossible. 
Commenter (6126) states that EPA will not likely have the resources or capability to assess the entire suite of regulations and the evidence-base retrospectively. The commenter states that, for example, EPA is technically required to evaluate every regulation promulgated under the Resource Conservation and Recovery Act (RCRA) on a triennial basis; and recent research suggests that statutory requirement has been largely unfulfilled due to a lack of resources to comply with the decades-old mandate. The commenter asserts that, in the absence of a major influx of new resources, EPA would be unlikely to engage in such meaningful and transparent reviews while also complying with statutory obligations.
Response: This final rule applies prospectively to significant regulatory actions and influential scientific information developed after the effective date of the final rule; the final rule has no retrospective effect on either final significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. 
10.2.5.2.3 Application of the Proposal to Prior Studies Would Dismiss Important Studies
Comment: Commenters (1011, 1281, 2477, 4838, 4931, 5172, 6109, 6114, 6126, 6868, 6895) recommend against applying the rule retrospectively because the outcome of such action would be to dismiss scientific studies and weaken the arguments that lay the foundation for public health protections. 
Commenter (1011) states that it may not be possible for EPA to acquire and release, in a manner consistent with the proposed rule, the data generated from studies previously incorporated into the administrative record. The commenter asserts that omitting these data from the administrative record would unnecessarily weaken the body of evidence EPA can consider in its regulatory actions.
Commenter (2477) states that retroactively trying to apply even a reasonable open-science standard to research done before that was the norm and before funding agencies were providing funds to do the work to make data publicly accessible, is not sound science; retroactively applying the proposed requirements comes across as defining an ad hoc metric to allow people to ignore prior work with which they disagree. The commenter recommends using the best previous scientific studies we have while at the same time increasing funding so future studies can be more open, including funding replication studies of any past work that is in question and is critical to current policy. 
Response: This final rule applies prospectively to significant regulatory actions and influential scientific information developed after the effective date of the final rule; the final rule has no retrospective effect on either final significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. The final rule does not require EPA to obtain, store or publish the dose-response data underlying pivotal science.
The EPA acknowledges, and agrees with commenters, that there may be pivotal science where the underlying dose-response data are not available in a manner sufficient for independent validation. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use these studies in significant regulatory actions and influential scientific information after either giving lesser consideration to the study or studies or if the Administrator grants an exemption under 40 CFR 30.7.
Comment: Commenters (4838, 4931, 5172, 6114, 6868) express concern that, if the proposed rule is applied retroactively, the EPA Administrator could reconsider regulations already in place, thereby allowing existing health protections to be revoked if the studies they were based upon are no longer allowed to be used by the EPA. Commenter (4931) states that, while significant progress in improving air quality has resulted from a number of federal, state, and local rules that were adopted over the years, the proposed rule runs the very real risk of reversing these gains, resulting in worse air and significant negative health effects. 
Commenter (6868) expresses concern that EPA plans to apply the proposed requirements to rulemakings that are already in progress or recurring policy review  -  such as the periodic review of the NAAQS, potentially applying the proposed new data policy to current standards retroactively. 
Response: This final rule applies prospectively to significant regulatory actions and influential scientific information developed after the effective date of the final rule; the final rule has no retrospective effect on either final significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created.
10.2.5.2.4 Grandfathering of Prior Studies/Regulatory Actions
Comment: Commenters (1012, 2006, 4595, 5170, 6449, 6875, 9224) oppose applying the proposed requirements to prior studies and, instead, suggest exempting or grandfathering certain past studies and data. Commenter (2006) asserts that it is not fair or wise to disregard studies with important results by holding them to a standard that the scientists producing the studies could have no knowledge of at the time. Commenter (4595) states that all prior studies should be exempt from this rule, as many foundational studies regarding air quality and asthma and exposure to mercury and lead were conducted decades ago and many of these studies have been reanalyzed, reassessed and/or reproduced by other institutions and countries. Commenter (5170) states that grandfathering certain past studies and data would be similar to what EPA did for pesticides that EPA approved for use in the past that were approved under less current research protocols and have continued to be "grandfathered in" for use. Commenter (6449) suggests that EPA grandfather existing analyses, unless being updated or challenged. 
Commenters (6875, 9224) state that trying to apply this proposal to models, rules, and research that has already begun or has concluded would only serve to set current work back and complicate work already done. The commenters state that it makes sense to "grandfather" what has already been completed and to implement this rule in stages in order to not compromise or delay EPA's work. Commenter (1012) states that regulatory actions that are based on administrative records from previous reviews should be excepted from the proposed rule to prevent introducing needless procedural hurdles into the EPA's regulatory review process. 
Response: This final rule applies prospectively to significant regulatory actions and influential scientific information developed after the effective date of the final rule; the final rule has no retrospective effect on either final significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created.
The EPA acknowledges, and agrees with commenters, that there may be pivotal science where the underlying dose-response data are not available in a manner sufficient for independent validation. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use these studies in significant regulatory actions and influential scientific information after either giving lesser consideration to the study or studies or if the Administrator grants an exemption under 40 CFR 30.7.
10.2.5.2.5 Applying the Proposed Rule Retrospectively Could Lead to Regulatory Uncertainty/Litigation Risks/Legal Challenges
Comment: Commenters (4931, 5165, 5170, 6905, 6935) express concern that retrospective implementation could lead to significant regulatory uncertainty, litigation risks, and legal challenges. Commenters (4931, 5165) contend that retrospective implementation would call into question existing regulatory standards as well as the permits, state implementation plans and other decisions that are based on those standards. Commenter (6905) states that retrospective implementation presents increased litigation risk for states who rely on existing rules and standards set by the EPA. Commenter (5170) states that retroactive application of this proposed rule would make the Agency vulnerable to an onslaught of legal challenges.
Response: This final rule applies prospectively to significant regulatory actions and influential scientific information developed after the effective date of the final rule; the final rule has no retrospective effect on either final significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created.
Comment: Commenters (6116, 6133, 6361) assert that retrospective application of the Proposal is not legally supportable.
Commenters (6116, 6361) state that it would be improper and contrary to the APA for EPA to reopen prior regulations for review due to alleged lack of transparency in past scientific studies. 
Commenter (6133) states that the Proposal does not identify a single provision in a single statute that EPA administers, or any other federal law, that requires or even authorizes any final rule based on the Proposal to have retroactive effect. The commenter states that there has been no power conveyed by Congress in express terms to promulgate retroactive rules related to any element of the Proposal. 
Commenter (6133) further states the Proposal ignores an entire body of case law that has considered and roundly rejected both retroactivity in rulemakings and limiting data that underlies rulemakings to "publicly available data." The commenter states that the Supreme Court strongly disfavors retroactive application of rules: 
      "Retroactivity is not favored in the law. Thus, congressional enactments and administrative rules will not be construed to have retroactive effect unless their language requires this result. [] By the same principle, a statutory grant of legislative rulemaking authority will not, as a general matter, be understood to encompass the power to promulgate retroactive rules unless that power is conveyed by Congress in express terms. [] Even where some substantial justification for retroactive rulemaking is presented, courts should be reluctant to find such authority absent an express statutory grant." Bowen v. Georgetown Univ. Hosp., 488 U.S. 204, 208 - 09 (1988) (internal citations omitted). 
Response: This final rule applies prospectively to significant regulatory actions and influential scientific information developed after the effective date of the final rule; the final rule has no retrospective effect on either final significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created.
10.2.5.2.6 Conditional Support for Application of Proposed Rule to Retroactive Studies/Actions
Comment: Commenters (1011, 6141, 6151), while generally supporting a forward-looking approach for the rule, suggest that there may be circumstances in which EPA needs to re-address past regulatory decisions or rely on data previously submitted for a new decision-making activity. 
Commenter (1011) states that EPA should avoid reviewing data previously incorporated into the administrative record unless new scientific data necessitates the review or EPA is periodically required to do so by law. 
Commenter (6141) recommends that, instead of imposing an absolute ban on the use of past studies, EPA develop reasonable rules and guidelines on when EPA may consider these past studies in future rulemakings. The commenter states that one such situation where Agency guidance will be needed is for health-based standards (e.g., NAAQS) that EPA set based on restricted data in past rulemakings but must review those standards in the future on a periodic basis.
Commenter (6151) states that, whether or how to implement the transparency rule should be considered on a case-by-case basis, recognizing the considerable challenges that will be faced. 
Response: This final rule applies prospectively to significant regulatory actions and influential scientific information developed after the effective date of the final rule; the final rule has no retrospective effect on either final significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created.
The EPA acknowledges, and agrees with commenters, that there may be pivotal science where the underlying dose-response data are not available in a manner sufficient for independent validation. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use these studies in significant regulatory actions and influential scientific information after either giving lesser consideration to the study or studies or if the Administrator grants an exemption under 40 CFR 30.7.
Comment: Commenters (2426, 6375) state that, for the most part, retroactive application is likely unnecessary. Commenters state that many of the EPA's significant rules have already been implemented and are on a statutorily mandated review cycle. Commenter (2426) states that, once the EPA solidifies the requirements of this proposed data transparency rule, they should commit to applying the new transparency standards to pivotal scientific studies, both old and new, that justify agency actions on rules as they are formally reviewed on their existing cycle. Commenter (2426) states that, in a separate analysis, the EPA should evaluate whether any rules that are not on a review cycle should be re-evaluated through the lens of this data transparency rule and the EPA should receive feedback on their determinations from the public and any applicable review boards or entities. 
Response: This final rule applies prospectively to significant regulatory actions and influential scientific information developed after the effective date of the final rule; the final rule has no retrospective effect on either final significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created.
Comment: Commenter (6356) states that, to the extent that future regulations rely on old data and/or dose-response relationships, then the proposed regulation should be transparent on both sets of information--the data and/or dose-response models must be available for review and independent validation in the context of the new rulemaking. Commenter states that the information could not form the basis for a challenge of a standard or pollutant limitation that EPA promulgated prior to adoption of this final agency action. 
Response: This final rule applies prospectively to significant regulatory actions and influential scientific information after the effective date of the final rule; the final rule has no retrospective effect on either final significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created.
The EPA acknowledges, and agrees with commenters, that there may be pivotal science where the underlying dose-response data are not available in a manner sufficient for independent validation. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use these studies in significant regulatory actions and influential scientific information after either giving lesser consideration to the study or studies or if the Administrator grants an exemption under 40 CFR 30.7.
Comment: Commenter (3971) states that "looking backward" to rehash regulations that are already in place would be disruptive to the environmental sciences and would generate significant push-back from many stakeholders. The commenter suggests that, as EPA reviews the appropriateness and protectiveness of past regulations during their required re-evaluation process, EPA should do so under the guidelines of this "transparency" regulation.
Response: This final rule applies prospectively to significant regulatory actions and influential scientific information developed after the effective date of the final rule; the final rule has no retrospective effect on either final significant regulatory actions or influential scientific information.
10.2.6 Application of the Rule to Specific EPA Programs
 10.2.6.1 Application to Prior Reviews Under the NAAQS Program
Comment: Commenters (0555, 4882, 6102, 6857, 6921, 9243) support expanding the scope of the rule to include application to prior reviews under the NAAQS program. 
Commenter (0555) states that for regulatory programs like the NAAQS that periodically review and update standards based on a record that has been built over decades, application of the proposed transparency and integrity procedures to that record, when feasible, could allow a broader group of experts access to pivotal regulatory science, including data, models, and assumptions. The commenter states that such review would be consistent with EPA's statutory mandate for review under the CAA and allow EPA to make future decisions on better data. 
Commenter (4882) states that NAAQS reviews need to use the most current, and rigorous standards, not carry forward less rigorous standards used in prior reviews. The commenter states that, if no scientific information is publicly available in a manner suitable for independent validation, the last standard that did use scientific information meeting the requirements of this rule should be used until such time new information becomes available. According to the commenter, past poor decisions in setting standards should not be carried forward in new standards. 
Commenter (6185) suggests that without clearly articulated and appropriate bounds, the Proposed Rule can restrict the scientific literature that is the basis for reviews of the NAAQS. The commenter adds that, along with the changes to the NAAQS review process outlined in EPA's May 9, 2018 "Back to Basics" memo and the recent requirement that members of the CAA Science Advisory Committee and related panel members not receive any current EPA funding, the Proposed Rule would constrain both the range of expertise and body of scientific literature that is available to be considered in these reviews, undermining the NAAQS.
Commenters (6102, 6857, 6921, 9243) assert that increased transparency is critically important in NAAQS rulemakings due to the magnitude of the potential impact of the rulemakings on public health and the economy. 
Commenters (6857, 9243) state that transportation conformity review is an example of the need for transparency in regulatory science. The commenters state that, where appropriate, the EPA should consider science demonstrating the consequences of proposed NAAQS changes on other federal activities that promote public health and economic stability. The commenters state that a complete analysis of potential NAAQS revisions should include the effects of the potential for increased unemployment, reduced congestion relief and weakened public safety.
Response: The EPA agrees that the Agency's significant regulatory actions can potentially impact American lives and livelihoods. Because of this, the American people deserve environmental decisions and policies that are based on the best information, and only through transparency can the EPA truly demonstrate its commitment to the Agency's mission of protecting public health and the environment through sound policy decisions that are informed by robust scientific and technical research.
The EPA clarified in the final rule that this rulemaking is not intended to remedy a particular instance in which the EPA may have relied on a poor-quality study or a study for which the conclusions could not be independently validated. Rather, the rulemaking is intended to meet a broader, compelling public need for increased transparency in the EPA's decision-making. 
The final rule applies prospectively to significant regulatory actions and influential scientific information developed after the effective date of the final rule; the final rule has no retrospective effect on either final significant regulatory actions or influential scientific information. The EPA would like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the rule does not require the EPA to collect, store, or publicly disseminate data and models underlying pivotal science.
As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and finalizing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available for independent validation. The EPA acknowledges, and agrees with commenters, that there may be pivotal science where the underlying dose-response data are not available in a manner sufficient for independent validation. In these cases, the EPA may still use these studies in significant regulatory actions and influential scientific information after either giving lesser consideration to the study or studies or if the Administrator grants an exemption under 40 CFR 30.7.
With this final rule, the EPA is including regulatory text in 40 CFR 30.3 stating that the statutes and regulations will control in the event of any conflicts and clarifying that the requirements in this final rule set the overarching structure and principles for science transparency in the EPA's significant regulatory actions and influential scientific information. The EPA intends to develop internal implementation guidance and promulgate either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule, which will clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes that the EPA administers.
Comment: Commenters (1012, 6116, 6133, 6356, 6362, 6379, 6868, 6925) oppose retroactive application of the rule to the NAAQS program. 
Commenters (1012, 6925) assert that retroactive application of this proposed rule would open up a process that could undermine existing public health protections. Commenter (6925) disagrees with any retroactive application of this proposed rule that would result in a revisionist and scientifically invalid assessment of the NAAQS. Commenter (6925) states that epidemiological studies form an essential basis of our understanding of how air pollution impacts human health; stripping these studies from consideration would undermine EPA's ability to establish NAAQS that remain sufficiently protective of public health. 
Commenters (1012, 6362) argue that retroactive application of this proposed rule would cause needless delays in the already beleaguered NAAQS review process. Commenter (6362) states that, because each NAAQS review under the CAA is based on a substantial amount of scientific and policy information used to inform EPA's determinations of appropriate levels for each standard, the retroactive application of this proposal to those administrative records would only serve to confuse, distress, and impede a NAAQS review process that is already severely overburdened. Commenter (6362) states that, while it is unclear which administrative NAAQS records would be covered by the proposal, the proposal could potentially cover more than a decades' worth of NAAQS administrative records and scientific analyses. 
Commenter (6356) states that prior regulations should not be affected by this rulemaking, even if EPA has not made their basis available for independent validation. The commenter states that EPA should explain, in the context of this rulemaking, that many environmental statutes, including but not limited to the CAA, contain anti-backsliding provisions regarding the relaxation of standards. The commenter states that courts have interpreted this CAA provision to mean that while EPA can revoke a NAAQS for a criteria pollutant, the non-backsliding provisions of the CAA prohibit the Agency from revoking regulatory requirements that were required for implementation of a prior standard. 
Response: The final rule applies prospectively to significant regulatory actions and influential scientific information developed after the effective date of the final rule; the final rule has no retrospective effect on either final significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created.
Comment: Commenters (6116, 6133) identify legal issues with retrospective application of the Proposal to the NAAQS program. Commenter (6133) states that the suggestion in the Proposal that EPA may invoke the Proposal's approach to review all prior health and scientific studies underlying the NAAQS is illegitimate, arbitrary and capricious, and contrary to caselaw. Commenter (6133) states that Bowen and its progeny do not permit agency rules to have retroactive effect to disallow health studies and regulatory science generated prior to, or relied upon by EPA prior to, adoption of any final rule based on the Proposal. 
Response: The final rule applies prospectively to significant regulatory actions and influential scientific information developed after the effective date of the final rule; the final rule has no retrospective effect on either final significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created.
Comment: Commenter (6379) urges EPA to avoid taking actions that would result in stranded costs of compliance with existing NAAQS and reduce ongoing regulatory certainty. The commenter states that companies have made and will continue to make investments in physical and operational changes in response to the existing NAAQS. The commenter states that any implementation of the Proposed Rule should ensure the certainty of these existing standards for states and businesses by applying only to future rules, if at all.
Response: The final rule applies prospectively to significant regulatory actions and influential scientific information developed after the effective date of the final rule; the final rule has no retrospective effect on either final significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created.
Comment: Commenters (5165, 6362) suggest the rule be applied prospectively to the NAAQS program. Commenter (5165) states that the rule should not be applied to data and models underlying studies that have already been completed or are currently underway. Commenter (6362) recommends the final rule be applied prospectively in a manner that integrates its application within the periodic review schedule established for each criteria air pollutant. 
Response: The final rule applies prospectively to significant regulatory actions and influential scientific information developed after the effective date of the final rule ; the final rule has no retrospective effect on either final significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. The provisions of this final rule apply to significant regulatory actions for which a proposed rule was published in the Federal Register after the effective date of the rule and influential scientific information submitted for peer review after the effective date of the rule.
10.2.6.2 Integrated Risk Information System (IRIS)
Comment: Commenters (4932, 6143, 6148, 6151, 6362, 6364, 6369, 6375, 6891, 6911) urge EPA to clearly apply the proposed requirements to risk evaluations performed in the Agency's IRIS program. 
Commenters (6362, 6364, 6891) suggest EPA incorporate the requirements of the Transparency rule during the development of the IRIS process for a chemical rather than wait until the IRIS assessment is used to support a rulemaking. Commenter (6362) strongly recommends that the Agency apply the proposed rule to any IRIS assessment that could be used as the basis for significant regulation.
Commenter (6362) states that the IRIS Program has been plagued for years by its slow pace generating IRIS assessments and lack of scientific transparency and reproducibility and provides several examples where they contend that IRIS assessments failed to reflect best available science (see APPENDIX C of their comment letter for examples). The commenter suggests that in cases such as IRIS assessments, where the Agency has yet to articulate a periodic review schedule for updating scientific assessments dating back 10-20 years or longer, EPA should develop appropriate mechanisms for application of the rule. 
Response: In 2020, the EPA issued an SNPRM and proposed to expand the scope of the rulemaking to include influential scientific information. Based on a consideration of public comments on the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. IRIS assessments are normally designated as influential scientific information.
This final rule applies prospectively to final significant regulatory actions and influential scientific information developed after the effective date of the final rule ; the final rule has no retrospective effect on either final significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created.
Comment: Commenter (6362) recommends that the Agency apply this rule to any IRIS assessment that could be used as the basis for significant regulation, including ethylene oxide. The commenter asserts that the final IRIS assessment for ethylene oxide, posted in December 2016, uses unsupportable and un-reviewed conservative risk assessment modeling. 
Commenter (6362) provides that the IRIS assessment concludes that the one-in-a-million lifetime cancer risk value associated with exposure to ethylene oxide is less than 1 part per trillion (ppt). The commenter states that this value is far below both ethylene oxide background levels in the environment and ethylene oxide levels naturally converted from ethylene in humans through breathing. According to the commenter, this conclusion is not plausible and not scientifically supportable. The commenter contends that it is based on an inadequate evaluation of a body of evidence from human studies that include historical exposure levels to ethylene oxide that are far higher than current occupational exposure limits. The commenter states that other, more accurate data sources are available, and alternative scientific risk assessment modeling approaches could have been used, but the IRIS Program did not systematically integrate all the evidence. The commenter notes that public comments on the ethylene oxide IRIS assessment can be found in Docket No. EPA-HQ-ORD-2006-0756.
Commenter (6362) states that EPA's Scientific Advisory Board (SAB) 2007 review concluded that substantial revisions were needed to the draft IRIS assessment including:
 Acquiring and using individual data for modeling rather than grouping populations, which results in overly conservative estimated cancer risks. 
 Considering using both linear and non-linear approaches to estimate cancer risk due to the distribution of and questionable association with certain cancer types; and
 Providing more transparency and correcting flaws associated with inappropriately grouping lymphohematopoietic cancers and combining genders for the dose-response analysis.
Commenter (6362) provides that meeting materials, including public comments, can be found at https://yosemite.epa.gov/sab/sabproduct.nsf/MeetingCal/7E3E313F627541D78525711400470D01.  
Commenter (6362) asserts that they did not believe that the 2015 Science Advisory Board (SAB) Committee that reviewed the revised 2013 ethylene oxide draft IRIS assessment conducted an independent, unbiased review. Problems identified by the commenter included:
 Several SAB members made inaccurate public statements indicating industry produced scientific studies should be not be considered due to potential industry influence, although no evidence of biased data sponsored by industry was ever presented.
 SAB members did not understand new evidence-based medicine concepts regarding mutagenicity of cancer cells and the contribution of naturally occurring ethylene oxide in DeoxyriboNucleic Acid (DNA) repair mechanisms.
 The SAB recommended using epidemiology data sets with questionable or scientifically unsound characteristics to estimate cancer risk and rejected alternative data sets that are as or more robust than those selected.
Commenter (6362) notes that the EPA still did not use individual data for modeling as recommended by the SAB in 2007 and did not adequately explore alternatives to the linear low dose modeling approach. The commenter provides that meeting materials, including public comments, can be found at https://yosemite.epa.gov/sab/sabproduct.nsf/MeetingCal/17F305EC43EB1A6585257E2D0050255F.
Commenter (6362) states that the IRIS Program used a spline approach (piecewise linear model that was not presented during either SAB review) for exposure-response analyses for each of the lymphoid and breast cancer endpoints and ultimately combined the results. According to the commenter, this approach results in higher risk at lower exposure levels and leads to proposed regulatory levels that are orders of magnitude lower than what the epidemiologic and genotoxicity scientific evidence would support.
Further, commenter (6362) asserts that the IRIS Program did not fully consider all available evidence in finalizing the ethylene oxide assessment. According to the commenter, scientific evidence clearly indicates that ethylene oxide is a weak mutagen and a unit risk factor of less than 1 ppt is not realistic or reliably measurable and is orders of magnitude lower than levels of ethylene oxide in ambient air and the normal, endogenous levels of ethylene oxide present in human bodies. Moreover, the commenter contends that the assessment fails to consider the difference between exposures to ethylene oxide produced outside the human body and exposure to ethylene oxide produced within the human body as a normal metabolic product.
Response: In 2020, the EPA issued an SNPRM and proposed to expand the scope of the rulemaking to include influential scientific information. Based on a consideration of public comments on the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. IRIS assessments are normally designated as influential scientific information.
This final rule applies prospectively to significant regulatory actions and influential scientific information developed after the effective date of the final rule; the final rule has no retrospective effect on either final significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created.
The EPA would like to emphasize that the final rule is not intended to remedy a particular instance in which the EPA may have relied on a poor-quality study or a study for which the conclusions could not be independently validated. Rather, the rulemaking is intended to meet a broader, compelling public need for increased transparency in the EPA's decision-making.
Comment: Commenter (6362) recommends that the Agency apply this rule to any IRIS assessment that could be used as the basis for significant regulation, including 1,2,3-Trimethylbenzene (TMBs). The commenter asserts that, in its review of TMBs, the EPA fell far short in meeting its obligations to improve its IRIS processes and assessment reports. The commenter adds that EPA failed to respond to public comments on the draft TMBs assessment, even though the IRIS process for developing assessments explicitly includes a response to comments element and that EPA failed to rely on available guideline studies on commercial complex C9 aromatic mixtures that industry conducted under EPA's TSCA program. According to the commenter, guideline studies on the commercial complex of aromatic mixtures are highly relevant to assessing the toxicology of TMBs. The commenter provides that EPA's Office of Pesticide Programs (OPP) has also reviewed the toxicology of TMBs and determined that the health effects of TMBs can be efficiently assessed by relying on C9 aromatic mixture studies. The commenter notes that OPP reached different scientific conclusions, including different quantitative health effect numbers, than that of EPA's IRIS Program. The commenter states that EPA, however, did not resolve these differences during the IRIS assessment of TMBs.
Response: In 2020, the EPA issued an SNPRM and proposed to expand the scope of the rulemaking to include influential scientific information. Based on a consideration of public comments on the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. IRIS assessments are normally designated as influential scientific information.
This final rule applies prospectively to significant regulatory actions and influential scientific information developed after the effective date of the final rule; the final rule has no retrospective effect on either final significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created.
The EPA would like to emphasize that the final rule is not intended to remedy a particular instance in which the EPA may have relied on a poor-quality study or a study for which the conclusions could not be independently validated. Rather, the rulemaking is intended to meet a broader, compelling public need for increased transparency in the EPA's decision-making.
Comment: Commenter (6364) urges EPA to apply the Transparency Rule to the IRIS program from the time any public announcement is made concerning development of a compound-specific assessment through the publication of a final assessment. The commenter states that significant regulatory decisions that incorporate an IRIS analysis can take place at any time, and those proceedings may or may not provide adequate opportunities to evaluate the quality of the IRIS assessment itself. According to the commenter, applying the Transparency Rule throughout the development process will help ensure that IRIS documents are scientifically sound and balanced, because IRIS assessments "provide a critical part of the scientific foundation" for EPA decision-making. 
Commenter (6891) states that supplying the underlying data/models during the IRIS process and providing sufficient opportunity to the public to comment and participate in the process is far preferable to waiting until those reference doses are employed in a later significant rulemaking, which could occur years or decades later. 
Response: In 2020, the EPA issued an SNPRM and proposed to expand the scope of the rulemaking to include influential scientific information. Based on a consideration of public comments on the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. IRIS assessments are normally designated as influential scientific information.
This final rule applies prospectively to significant regulatory actions and influential scientific information developed after the effective date of the final rule; the final rule has no retrospective effect on either final significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created.
Comment: Commenters (6151, 6364) note that EPA views IRIS assessments as "the preferred source of toxicity information used by [the agency]," and are used in "set[ting] national standards and clean[ing] up hazardous sites." Commenter (6364) states that, because the development of risk assessments through the IRIS Program is a vital component of the rulemaking process, EPA should expressly include in the final Transparency Rule the publication of the underlying IRIS assessment data as a regular and early step in the development of IRIS assessments. Commenter (6891) states that, in the IRIS assessment process, EPA may develop reference concentrations or reference doses that will be critical for later rulemakings regarding those chemicals.
Response: In 2020, the EPA issued an SNPRM and proposed to expand the scope of the rulemaking to include influential scientific information. Based on a consideration of public comments on the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. IRIS assessments are normally designated as influential scientific information.
This final rule applies prospectively to significant regulatory actions and influential scientific information developed after the effective date of the final rule; the final rule has no retrospective effect on either final significant regulatory actions or influential scientific information.
Comment: Commenters (6151, 6364) express concern over a lack of transparency in the IRIS program. Commenter (6151) states that, despite recent agency efforts, the lack of full transparency in the IRIS program's development of toxicity values, as well as the inability to independently reproduce some of those values, continue to present obstacles to ensuring sound and balanced scientific decisions. 
Commenter (6364) states that extending the Transparency Rule to IRIS assessments is particularly appropriate in light of the long history of criticism of the IRIS program from the U.S. Government Accountability Office (GAO), the National Academy of Sciences (NAS) and other expert bodies. The commenter states that the IRIS Program's most recent Report to Congress touts the benefits of making information pertaining to assessments available for public comment "earlier in the assessment development process" and advances the goal of developing systematic review methods that ensure reproducibility. The commenter asserts that providing public access to the science and methods relied upon in IRIS assessments meaningfully advances this agenda.
Response: In 2020, the EPA issued an SNPRM and proposed to expand the scope of the rulemaking to include influential scientific information. Based on a consideration of public comments on the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. IRIS assessments are normally designated as influential scientific information.
This final rule applies prospectively to significant regulatory actions and influential scientific information developed after the effective date of the final rule; the final rule has no retrospective effect on either final significant regulatory actions or influential scientific information.
Comment: Commenters (6369, 6911) state that EPA's formaldehyde assessment illustrates why application of appropriate scientifically based exposure-response modelling and transparency in the use of science by the Agency is so important. Commenter (6369) states that, using data obtained from the National Cancer Institute (NCI)[,][,], in combination with data provided in the draft 2010 IRIS assessment for formaldehyde, further detailed analyses were conducted to validate pivotal scientific findings used by EPA's IRIS program to draw conclusions. Commenter (6369) states that the results have been found to disagree with the original study findings and/or the findings cannot be replicated due to methodological issues.
Response: In 2020, the EPA issued an SNPRM and proposed to expand the scope of the rulemaking to include influential scientific information. Based on a consideration of public comments on the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. IRIS assessments are normally designated as influential scientific information.
This final rule applies prospectively to significant regulatory actions and influential scientific information developed after the effective date of the final rule; the final rule has no retrospective effect on either final significant regulatory actions or influential scientific information.
The EPA would like to emphasize that the final rule is not intended to remedy a particular instance in which the EPA may have relied on a poor-quality study or a study for which the conclusions could not be independently validated. Rather, the rulemaking is intended to meet a broader, compelling public need for increased transparency in the EPA's decision-making.
Comment: Commenter (4932) recommends that EPA alter the definition of "regulatory decisions" to include Agency "rules," in addition to regulations, to make the Transparency Proposal fully consistent with Executive Order 12866. The commenter asserts that the adoption of this proposed change would ensure that the Transparency Proposal applies to the derivation of all inhalation reference concentrations, reference doses, oral slope factors, and inhalation unit risk values as part of EPA's IRIS program. The commenter states that EPA's IRIS values are not typically thought of as regulations and better fit the definition of a rule. According to the commenter, as the Transparency Proposal appears to have been intended to apply to derivation of reference values by virtue of the definition of "dose response data and models," EPA should eliminate any potential confusion about the application of the Transparency Proposal to derivation of IRIS values by making such applicability explicit.
Response: In 2020, the EPA issued an SNPRM and proposed to expand the scope of the rulemaking to include influential scientific information. Based on a consideration of public comments on the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. IRIS assessments are normally designated as influential scientific information. 
10.2.6.3 Toxic Substances Control Act (TSCA)
Comment: Commenters (6112, 6115, 6143, 9227) recommend that the Proposal's requirements apply to risk evaluations performed under section 6(b) of TSCA. 
Commenter (9227) objects to the Proposal in part because, as drafted, it will not apply to EPA's New Chemicals Review Program under TSCA. The commenter states that EPA has effectively exempted the TSCA new chemicals program where industry seeks expeditious actions allowing market access and EPA regularly fails to disclose its own analyses and the studies and materials supporting those decisions, much less any underlying data. The commenter states that, in these proceedings industry seeks affirmative authorization from EPA to commercialize chemicals, so industry has a vested interest in expeditious government action. The commenter asserts that EPA's exclusion of the new chemicals program clearly favors industry, allowing industry to conceal information and evade regulation. According to the commenter, the EPA cannot rationally impose stringent new disclosure requirements that exclude extensive peer-reviewed, high-quality studies in some contexts while simultaneously authorizing the commercial distribution of new chemicals with almost no disclosure and no peer-review. 
Moreover, commenter (9227) states that the New Chemicals Program is more opaque than the rulemakings EPA is currently targeting with its Proposal, often in direct violation of law. The commenter provides that the EPA does not make the public files for new chemicals electronically available, and when a person does obtain a copy of the public file from EPA, the files generally reveal almost none of EPA's analyses supporting its decisions or the information submitted to support those decisions, with massive amounts of data redacted or concealed as CBI. The commenter asserts that, if EPA extended the rule articulated in proposed §30.5 to the new chemicals program, it would seem that EPA would either have to make much of the information in the public files available or EPA would be precluded from using this information. Without this information, EPA generally would not be able to find that the new chemical "is not likely to present an unreasonable risk of injury to health or the environment," the finding that allows unregulated manufacture of the chemical. Thus, excluding the information would require EPA to regulate the new chemicals before they could enter the market.
Commenter (6143) specifically urges the EPA to clarify that the proposed requirements will also apply to risk evaluations performed under section 6(b) of TSCA, and to assessments performed by the Agency's IRIS Program.
Response: In 2020, the EPA issued an SNPRM and proposed to expand the scope of the rulemaking to include influential scientific information. Based on a consideration of public comments on the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. Influential scientific information means scientific information the Agency reasonably can determine will have or does have a clear and substantial impact on important public policies or private sector decisions. 
As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and finalizing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available for independent validation.
With this final rule, the EPA is including regulatory text in 40 CFR 30.3 stating that the statutes and regulations will control in the event of any conflicts, and clarifying that the requirements in this final rule set the overarching structure and principles for science transparency in the EPA's significant regulatory actions and influential scientific information. The EPA intends to develop internal implementation guidance and promulgate either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule, which will clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes that the EPA administers.
10.2.6.4 Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA) and Resource Conservation and Recovery Act (RCRA)
Comment: Commenter (6365) states that the language in the regulatory proposal should be read to apply EPA's proposed scientific transparency policy to CERCLA remedial action levels and RCRA corrective actions. According to the commenter, however, for clarity, in the final rule EPA should expressly state that its scientific transparency policy applies to the process of setting CERCLA remedial action levels and in establishing RCRA corrective actions. The commenter asserts that this clarification will provide certainty to potentially responsible parties and agency officials as they undertake these processes, including assessing risk and considering remedial alternatives. 
Response: With this final rule, the EPA is including regulatory text in 40 CFR 30.3 stating that the statutes and regulations will control in the event of any conflicts, and clarifying that the requirements in this final rule set the overarching structure and principles for science transparency in the EPA's significant regulatory actions and influential scientific information. The EPA intends to develop internal implementation guidance and promulgate either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule, which will clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes EPA administers.
10.2.6.5 Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA) and Federal Food, Drug, and Cosmetic Act (FFDCA)
Comment: Commenters (6363, 6374) argue that the proposed rule requirements should not apply to pesticide registration decisions under FIFRA, and tolerance setting under FFDCA. These commenters assert that EPA pesticide product registration actions are not "significant regulatory actions" that meet the definition of "significant regulatory action" as set forth in Executive Order 12866/OMB).
Commenters (6263, 6374) state that EPA needs to ensure compliance with the statutory framework established by Congress to carefully balance the requirements of public notice and transparency (which are accomplished through several statutory provisions under FIFRA and FFDCA) with the need to incentivize pesticide product innovation through the protection of proprietary data rights (also accomplished through specific provisions of FIFRA and FFDCA).
Response: With this final rule, the EPA is maintaining language in 40 CFR 30.3 stating that the statutes and regulations will control in the event of any conflicts, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for science transparency in the EPA's final significant regulatory actions and influential scientific information. The EPA intends to develop internal implementation guidance and promulgate either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule, which will clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes EPA administers.
10.2.6.6 CWA
Comment: Commenter (5181) states that application of the proposed requirements to individual permit actions under the CWA would be inconsistent with sound and timely authorizations under the §404 dredge and fill permit program. The commenter states that proposed requirements that would subject all data to identical requirements regarding data collection, evaluation, and release would interfere with the permitting process and would undermine the accuracy of individual permit decisions that rely on non-peer reviewed information. 
Response: The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. The EPA will not expand the applicability of this rulemaking beyond significant regulatory actions and influential scientific assessments to individual party adjudications, permit proceedings, site-specific actions, and enforcement actions because these actions are focused on individual regulated entities.
Comment: Commenters (6180, 6362, 6891) favor application of the transparency requirements in specific CWA programs. Commenter (6180) states that National Pollutant Discharge Elimination System (NPDES) permit proceedings (and related water quality standards TMDL decisions have potentially long-lasting, widespread, and costly implications for permittees and therefore should not be excluded from this rule as they meet the requirements of pivotal regulatory science. Commenter (6362) states that each of those decisions requires analyzing complex environmental/engineering data on a case-specific basis.
Commenter (6180) supports application of the Proposed Rule to technical CWA criteria development and related NPDES permitting activities, including the derivation and imposition of NPDES permit limits, TMDLs, and water quality standards (WQS). The commenter provides that these legally binding NPDES permits, often driven by WQS and TMDLs, include effluent limits for wastewater treatment facilities for pollutants such as lead, mercury, or phosphorus. According to the commenter, slight alterations in these permit limits can cost a single wastewater facility tens of millions of dollars, the cost of which is eventually passed on to individual local rate payers. The commenter notes that these permit limits are supposed to be derived in a manner similar to dose-response relationships, as mentioned in the Proposed Rule, where, for example, a lower level of a pollutant in the discharge will result in a measurable improvement in receiving water quality or human health. As such, the commenter asserts that these criteria development and permitting activities fall under the Proposed Rule's scope and definition of "pivotal regulatory science." The commenter expresses that they are aware of numerous instances throughout the country where environmental agencies have (1) based regulations on outdated science, or (2) refused to consider available data or studies which indicate stringent permit limits imposed by these agencies are not anticipated to result in any quantifiable environmental or human health benefit despite disproportionately high costs. (The commenter refers to Attachment 1 of their comment letter  -  March 20, 2013 House Letter). The commenter suggests that expanding the Proposed Rule to CWA criteria development would remedy shortcomings by applying, at a minimum, to the following:
 Water quality criteria derivation and interpretation, particularly for narrative nutrient criteria
 Impairment listing determinations
 NPDES permit limit derivations
 Statistical methods used to claim ecological significance
Commenter (6891) states that, under the Proposed Rule, they are concerned that the EPA's water quality criteria would not be covered as they are not considered "regulations." The commenter asserts that the federal water quality criteria are important because they initiate a chain of actions that ultimately result in water quality-based effluent limits (WQBELs). The commenter provides that EPA requires states and tribes to consider the federal water quality criteria during triennial review of their own water quality standards and they often form the basis of those standard and that these state and tribe standards, in turn, are incorporated into the wastewater discharge permits. The commenter expresses concern that they have not been able to adequately comment on EPA's draft water quality criteria because some of the key data or models justifying them were not provided. The commenter provides several recent examples of draft water quality criteria where some of the necessary underlying data and models were not provided for review (e.g., Draft Aquatic Life Ambient Water Quality Criteria for Aluminum, Draft Aquatic Life Ambient Estuarine/Marine Water Quality Criteria for Copper, Draft Updated National Recommended Water Quality Criteria for the Protection of Human Health (Notice of Availability)).
Response: With this final rule, the EPA is including regulatory text in 40 CFR 30.3 stating that the statutes and regulations will control in the event of any conflicts, and clarifying that the requirements in this final rule set the overarching structure and principles for science transparency in the EPA's significant regulatory actions and influential scientific information. The EPA intends to develop internal implementation guidance and promulgate either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule, which will clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes EPA administers.
10.2.6.7 Endangerment Finding
Comment: Commenter (6137) states that the proposed procedures cannot lawfully or rationally be required in the context of EPA decisions to abate "imminent and substantial endangerments" ("ISEs") to human health and the environment. See, e.g., 42 U.S.C. §§ 300(i)(a) (SDWA § 1431(a)), 6973(a) (RCRA § 7003(a)), 7603 (CAA § 303), and 9606(a) (CERCLA § 106(a)). The commenter states that none of these provisions set or authorize limitations on the studies that can be relied upon in identifying ISEs. The commenter states that, to the contrary, both courts and the EPA have interpreted these statutes as precautionary in nature, allowing abatement action where there is only a risk of harm. See, e.g., EPA, Guidance On the Use of Section 303 of the CAA, EPA-R08-OAR-2013-0556-0015, at 2-4 (1991) ("Section 303 Guidance"); see also United States v. Vertac Chemical Corp., 453 F.3d 1031, 1045 (8th Cir. 2006) (describing § 9606(a)'s ISE standard as "cautionary" in sanctioning EPA's decision to issue a unilateral administrative order requiring cleanup of former manufacturing site). 
Response: As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and finalizing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available for independent validation.
With this final rule, the EPA is including regulatory text in 40 CFR 30.3 stating that the statutes and regulations will control in the event of any conflicts, and clarifying that the requirements in this final rule set the overarching structure and principles for science transparency in the EPA's significant regulatory actions and influential scientific information. The EPA intends to develop internal implementation guidance and promulgate either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule, which will clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes EPA administers.
10.2.7 Application of the Rule to a Limited and Special Class of Laboratory Samples
Comment: Commenter (6143) suggests that the EPA apply the proposed requirements to a limited and special class of laboratory samples, the analytical results of which (from non-destructive testing) may become an essential element of the research that underpins a given regulation. The commenter states that, not only can different laboratory analysts reach different conclusions by analyzing the same sample, it is common for analytical methods to improve over time. The commenter notes that improved analytical methods provide researchers with the opportunity to re-analyze properly archived samples, potentially yielding more accurate and even different results. The commenter notes that it should become part of the researchers' protocol to retain and properly archive this special class of samples when the analytical results are likely to inform or inspire the regulatory process. The commenter states that is particularly true for research conducted by or for EPA and its contractors; and by any party conducting research with the intent to inform or inspire a specific regulatory action. (While it is true that certain types of samples cannot be retained because they are inherently unstable chemically and/or physically, other types of samples can remain stable for decades when properly archived.)
Response: The EPA does not agree that this rulemaking should apply to physical objects, and the provisions of final rule do not apply to physical objects (like laboratory samples). Physical objects like lab samples often have a shelf-life, are not permanent nor is it practical to retain physical objects for a lengthy period of time, which then makes it unreasonable for the provisions of this rule to apply. The provisions of the final rule also do not apply to drafts, and preliminary analyses, and influential scientific information and/or pivotal science that meet one or more of the exemptions identified in Section IX of the OMB Final Information Quality Bulletin for Peer Review. 
10.3 Narrowing the Scope of Applicability of the Rule
10.3.1 Narrowing the Scope of the Applicability of the Rule
Comment: Commenters (5181, 6099, 6449, 6155) express support for narrowing the scope of the applicability of the rule.
Commenter (5181) requests that, in their support for narrowing the applicability of the rule, EPA define a more specific category of decisions that would demand the level of public access to data defined by this rule, and more fully explain how the benefit of greater public access to raw data justifies the cost of implementation. 
Commenter (6155) supports limiting the scope of the proposed rule to "dose response data and models" as defined in §30.2 of the proposed rule. 
Commenter (6449) recommends that, since the "significant regulatory action" threshold of $100M was established by Congress in 1996, the threshold that would apply to this regulation should be adjusted to current and/or future year dollars, as appropriate.
Response: In 2020, the EPA issued an SNPRM and proposed to expand the scope of the 2018 proposed rulemaking in two ways: (1) expanding "dose-response data and models" to "data and models," and (2) including influential scientific information. As EPA noted in the 2020 SNPRM, transparency of EPA's science should not be limited to dose-response data and dose-response models, because other types of data and models can also drive the requirements and/or quantitative analysis of EPA final significant regulatory actions and influential scientific information. Based on the comments on the 2018 proposed rule and the 2020 SNPRM, including that the number of studies that would be subject to the rule might hinder rulemaking, the EPA determined that the Agency should pursue an incremental approach to maximizing transparency in the science that it relies upon by focusing the final rule requirements on dose-response data and, in particular, only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs).
The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. The provisions of the final rule apply to science that serves as the basis for informing a significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science. Significant regulatory actions means final regulations determined to be "significant regulatory actions" by the OMB pursuant to Executive Order 12866. The Agency will continue to follow the $100M economic impact consideration found in the Executive Order.
Finally, as defined in Part 30.2 of the final rule, the Agency has narrowed the definition of data. Data means the set of recorded factual material commonly accepted in the scientific community as necessary to validate research findings in which obvious errors, such as keystroke or coding errors, have been removed and that is capable of being analyzed by either the original researcher or an independent party. This stage of data is not raw data. In addition, this is a rule of internal procedure that does not impose requirements (such as the provision of data) on any party other than the EPA. This rule imposes no costs on researchers or their institutions, and the EPA will consider and evaluate all relevant and appropriate science in its significant regulatory actions and influential scientific information.
10.3.2 Limiting to "Major" Issues Under the Congressional Review Act or "Economically Signification" Actions Under Executive Order 12866 
Comment: Commenters (2426, 6099, 6149, 6449) support limiting coverage of this regulation to "major" issues under the Congressional Review Act (CRA) or "economically significant" under Executive Order 12866. Commenter (2426) contends that this will better ensure timely regulatory action, as well as balance the need for confidentiality with regulatory transparency. The commenter states that regulatory actions outside of this scope (e.g., site-specific permitting actions) may still impact the public, though the impact is much less than that of major or economically significant rules.
Response: The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. For this rule, "significant regulatory actions" means final regulations determined to be "significant regulatory actions" under Executive Order. 12866. 
The EPA will not expand the applicability of this rulemaking beyond significant regulatory actions and influential scientific assessments to individual party adjudications, permit proceedings, site-specific actions, and enforcement actions because these actions are typically focused on individual regulated entities.
Comment: Commenters (6102, 6362) state that narrowing the definition to only "major" issues under the CRA or "economically significant" under Executive Order 12866 would omit rulemakings that could significantly impact specific sectors or industries, or that could have significant precedential impact. Commenter (6362) asserts that either of these approaches would lose the efficiency and predictability benefits of using the Executive Order 12866 definition -- and would increase work for both EPA and Office of Information and Regulatory Affairs (OIRA). 
Response: The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. For this rule, "significant regulatory actions" means final regulations determined to be "significant regulatory actions" under E.O. 12866. 
Comment: Commenters (5181, 6144) request clarification of, and distinction between, "major" issues under the CRA, and "economically significant" under Executive Order 12866. 
Response: The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. For this rule, "significant regulatory actions" means final regulations determined to be "significant regulatory actions" under E.O. 12866. The Agency believes the scope of the final rule is appropriate as the Agency makes incremental progress towards its transparency goals. 
10.3.3 CAA
Comment: Commenters (6155, 6179) suggest that EPA more narrowly focus the final rulemaking on improving transparency in the use of dose-response studies that establish regulations under the CAA. Commenter (6155) states that two important uses of dose response studies are to help determine NAAQS levels and which hazardous air pollutants to regulate under the CAA. Commenter (6179) states that air regulations are the most likely to incur substantial costs and benefits and are therefore most appropriate for the Proposal's high standard of robustness. 
Response: The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. The provisions of the final rule apply to science that serves as the basis for informing a significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science.
With this final rule, the EPA is including regulatory text in 40 CFR 30.3 stating that the statutes and regulations will control in the event of any conflicts, and clarifying that the requirements in this final rule set the overarching structure and principles for science transparency in the EPA's significant regulatory actions and influential scientific information. The EPA intends to develop internal implementation guidance and promulgate either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule, which will clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes EPA administers.
 10.3.4 Other Comments on Narrowing the Scope of Applicability of the Rule
Comment: Commenter (6924) states that EPA should not implement this proposal for any Agency decision, whether major or minor. The commenter asserts that doing so would hinder scientific inquiry and lead to inaccurate results. 
Response: As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available for independent validation. The EPA does not believe that this final approach will hinder scientific inquiry or lead to inaccurate results.
Comment: Commenter (6955) asserts that, if the rule were narrowed to apply only to studies funded by the US government and to studies funded by stakeholders participating in regulation, it would be an improvement, but, then, the public access part of the proposed rule would be largely moot, given current Freedom of Information Act (FOIA) access. 
Response: The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. The provisions of the final rule apply to science that serves as the basis for informing a significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science.
The EPA would like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the rule does not require the EPA to collect, store, or publicly disseminate data and models underlying pivotal science.
Finally, as stated in the preamble to the final rule, in part, the purpose of this final rule is to consider the availability of the underlying dose-response data in pivotal science. When this information is available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Make underlying dose-response data available is not a waste of time. Besides, for pivotal science where there is no access to dose-response data, or access is limited, the Agency may still consider these studies but will give them lesser consideration unless the Administrator grants an exemption under 40 CFR 30.9.
 10.4 Changes in the Scope of the Science Subject to the Rule
Comment: Commenter (6867) states that the replication crisis and public confidence rationales that underlie the Proposed Rule suggest that data, methodology, and computational code should be disclosed for all science relied upon to support agency actions. The commenter suggests, as a first step, the phrase "dose response data and models" should be replaced with "data and models, including dose response data and models" throughout the Proposed Rule. Commenter (6891) suggests that data and models provided should include all supporting quality assurance/quality control data and be accompanied with adequate explanatory materials so that the public can readily understand all models and all inputs and formulas to the models. 
Commenter (4882) requests that the disclosure requirements in this rule should be applied to information from environmental impact studies, models used to predict cost/benefit analysis, and environmental effects. 
Response: Based on a consideration of these and other similar comments, the EPA proposed, through a 2020 SNPRM, to expand the scope of the 2018 proposed rulemaking in two ways: (1) expanding "dose-response data and models" to "data and models," and (2) including influential scientific information. As EPA noted in the 2020 SNPRM, transparency of EPA's science should not be limited to dose-response data and dose-response models, because other types of data and models can also drive the requirements and/or quantitative analysis of EPA significant regulatory actions and influential scientific information.
Based on the comments on the 2018 proposed rule and the 2020 SNPRM, including that the number of studies that would be subject to the rule might hinder rulemaking, the EPA determined that the Agency should pursue an incremental approach to maximizing transparency in the science that it relies upon by focusing the final rule requirements on dose-response data and, in particular, only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs).
The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. The provisions of the final rule apply to science that serves as the basis for informing a significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science.
And as defined in Part 30.2 of the final rule, Data means the set of recorded factual material commonly accepted in the scientific community as necessary to validate research findings in which obvious errors, such as keystroke or coding errors, have been removed and that is capable of being analyzed by either the original researcher or an independent party. Depending on the data in question, this may or may not include QA/QC information as well as other explanatory information.
The EPA would also like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the rule does not require the EPA to collect, store, or publicly disseminate data and models underlying pivotal science.
 10.5 Scope  -  Phase-In
 10.5.1 Proposed Rule Does Not Provide for Any Phase-In Time
Comment: Commenter (6188) provides that the Proposed Rule would be effective immediately upon adoption, rather than including any sort of implementation schedule that recognizes the significant practical challenges of applying the rule. According to the commenter, putting the Proposed Rule into practice would mean that EPA would need to gather and make available enormous amounts of data, requiring the Agency not only to acquire that data, but also to devise a system to manage it; secure its storage; appropriately redact data; ensure data subject to Health Insurance Portability and Accountability Act (HIPAA) and other privacy laws or otherwise considered to be confidential is not disclosed; and implement "feasibility" exemptions to the rule. The commenter complains that the Proposed Rule does not provide for any phase-in time to establish the complicated procedures that will be necessary, meaning that any ongoing or planned rulemaking processes will be stalled or thwarted upon the rule's adoption as EPA struggles to take practical steps towards rule implementation. The commenter contends that this would not only stymie necessary regulation, but would also, as discussed above, leave the scientific community in the dark as it attempts to design and carry out studies to provide high quality information EPA can use to make informed decisions.
Response: The EPA does not find that a phase-in approach for implementing the final rule is necessary because of the limited application of this rule to pivotal science underlying significant regulatory actions and influential scientific information. The EPA would also like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the rule does not require the EPA to collect, store, or publicly disseminate data and models underlying pivotal science. 
The EPA disagrees with commenters that requirements of the final rule will result in any meaningful delay in promulgating regulations.
 10.5.2 Support Phasing-In of Transparency Requirements
Comment: Commenters (6875, 9224) support EPA seeking to phase-in transparency requirements, regardless of the strategy used. The commenters provide that the scientific community has just recently begun data access requirements. The commenters assert that, while the scientific community has been moving towards the idea of "open science", the policies are still new and phasing in requirements would give the scientific community sufficient time to respond and prepare for the implications of this rule. The commenters recommend that EPA ensure that there are explicit and clear milestones to be achieved throughout the process.
Commenter (6955) suggests that if the rulemaking goes forward, EPA should limit the first introduction to a single regulatory proceeding, as an experiment.
Commenter (6126) suggests that in lieu of a regulatory approach to changing scientific transparency practices at EPA, the Agency could begin with a targeted voluntary pilot effort that aims to address open scientific activities in a thoughtful, intentional manner. The commenter states that such a pilot could also enable the agency to better assess implications for individual programs, costs of implementation, and whether the anticipated benefits are realized relative to the activities described in the proposed rule.
Commenter (6448) recommends that the effective date of this regulation reflect significant phase-in times commensurate with the timeframes of the impacted regulatory actions.
Response: The EPA does not find that a phase-in approach for implementing the final rule is necessary because of the limited application of this rule to pivotal science underlying significant regulatory actions and influential scientific information. The EPA would also like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the rule does not require the EPA to collect, store, or publicly disseminate data and models underlying pivotal science. 
 10.5.3 Conditional Phase-In Implementation Recommendations
Comment: Commenter (0068) notes that if the Agency decides to impose a requirement that dose-response and similar data on which regulatory science is based be made "publicly available in a manner sufficient for independent validation," this requirement should apply only to new data developed after some future date, and only after the Agency has clearly demonstrated that safe and effective methods exist to anonymize sensitive personal data in epidemiological and other studies. According to the commenter, failing to impose such limitations would have the effect of discarding decades of excellent (and expensive) peer-reviewed science on which the agency has depended for years to protect the health and welfare of all Americans. For example, the commenter states that meta-analyses such as those used to determine the odds ratios for the effect of maternal smoking on lower respiratory illnesses during infancy can rely on scores of studies that cover decades. 
Commenter (2171) recommends that, given that the proposed rule offers no instruction on how "independent validation" would be conducted, existing EPA programs evaluating human health studies should continue under current practices until such time as implementation elements are completed, including providing sufficient funding for developing open access to databases, establishing guidance as to how and when EPA is to assess or use unsupervised validation studies submitted by the public, as well as determining and resolving public and nonpublic data issues (securing permissions if necessary).
Commenter (2426) suggests that, once the EPA solidifies the requirements of the data transparency rule, EPA should commit to applying the new transparency standards to pivotal scientific studies, both old and new, that justify agency actions on rules as they are formally reviewed on their existing cycle. In a separate analysis, the commenter recommends that the EPA evaluate whether any rules that are not on a review cycle should be re-evaluated through the lens of this data transparency rule and receive feedback on their determinations from the public and any applicable review boards or entities. The commenter suggests that EPA consider using an upcoming major or significant rule-making as a test case to help solidify how the intent of the data transparency rule could be thoughtfully implemented before phasing in the final data transparency rule. According to the commenter, because the proposed data transparency rule offers limited implementation details, it is difficult to provide meaningful comment on the potential intentional and unintentional consequences of its enactment. The commenter provides that, rather than promulgate an untested final rule that could potentially impact all agency actions, the EPA should consider choosing a major upcoming rule-making as a test case. According to the commenter, ideally, the test case would help sketch out more specific guidelines for data transparency rule implementation that could then be vetted with an external, cross-disciplinary work group and provided for public review and comment. 
Commenter (6360) believes that that requirements of this rule can be implemented immediately if the scope and applicability are defined appropriately. However, the commenter acknowledges that there could be practical considerations that could make this infeasible. The commenter recommends that criteria be employed during the transition period to full EPA compliance to address past instances of non-transparency consistent with existing law and policy.
Commenter (6375) states that the proposed rule should apply to all significant regulatory actions promulgated after the effective date of the rule. According to the commenter, if a phase-in is desired, another approach would be to prioritize implementation of this rule for any significant regulatory actions related to establishment, review, and/or revision of NAAQS. Subsequently, the commenter states that the Agency could then apply this to other programs on a set schedule laid out in the final rule.
Commenter (6356) notes that, if this proposed rule is adopted, a less acceptable alternative would be to phase the rule's requirements in over the next ten years. The commenter provides, however, that the phase-in cannot be based on a rule's regulatory impact assessment because a rule's cost is generally not known until a final rule is adopted based on cost-benefit studies done after the basis for a regulation has been generated.
Response: The final rule applies prospectively to significant regulatory actions and influential scientific information developed after the effective date of the final rule. The provisions of the final rule apply to science that serves as the basis for informing a significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and finalizing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available for independent validation. The EPA acknowledges, and agrees with commenters, that there may be pivotal science where the underlying dose-response data are not available in a manner sufficient for independent validation. In these cases, the EPA may still use these studies in significant regulatory actions and influential scientific information after either giving lesser consideration to the study or studies or if the Administrator grants an exemption under 40 CFR 30.7.
The EPA does not find that a phase-in approach for implementing the final rule is necessary because of the limited application of this rule to pivotal science underlying significant regulatory actions and influential scientific information. The EPA would also like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the rule does not require the EPA to collect, store, or publicly disseminate data and models underlying pivotal science. 
 10.5.4 Implement Existing Guidance More Fully Before Considering Phasing-In of Rule
Comment: Commenter (6126) recommends that EPA seek to implement its existing guidance more fully to encourage open data and data availability using administrative means and policy guidance as a logical first step before considering phasing-in of the rule. The commenter suggests that EPA could pilot an opportunity for a targeted statutory area at the Agency to develop the practices and protocols for voluntary submission of relevant de-identified data that fulfill the stated intent of the rule. The commenter provides that, importantly, this would include developing the protocols, practices, and infrastructure needed to fulfill the expectations for privacy and confidentiality protections. The commenter states that the EPA should first identify how to implement these aspects of the proposed rule before specifically connecting the use of those data or availability of data to the decision-making process.
Response: The EPA does not find that a phase-in approach for implementing the final rule is necessary because of the limited application of this rule to pivotal science underlying significant regulatory actions and influential scientific information. The EPA would also like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the rule does not require the EPA to collect, store, or publicly disseminate data and models underlying pivotal science.
The EPA also disagrees that existing guidance needs to be implemented before proceeding with the final rule. The transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency. The EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. For example, in 2002 the Office of Management and Budget (OMB) released its Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies, which includes discussion of the importance of the reproducibility of analyses underlying influential information. The EPA's 2016 Plan to Increase Access to Results of EPA-Funded Scientific Research noted that "transparency is a core EPA value" and that increased availability of research data would accelerate scientific breakthroughs that support the Agency's mission and policymaking efforts. The EPA's Open Government Plan 5.0 also details the EPA's progress in implementing the tenets of the numerous data transparency initiatives in the federal government prior to 2018, including the OMB M-10-06, the Office of Science and Technology Policy Memorandum of February 22, 2013, and OMB M-13-13. In 2019, Congress passed the Foundations for Evidence-Based Policymaking Act of 2018 (or OPEN Government Data Act, Pub. L. 115-435) into law, which included requirements for federal agencies to prioritize making their data available to the public, and OMB has released additional guidance for implementing the act.
 10.5.5 Phase-In for Use of Past Studies Lacking in Sufficient Transparency
Comment: Commenter (6141) requests that, instead of imposing an absolute ban on the use of past studies that are lacking in sufficient transparency in all cases, EPA develop reasonable rules and guidelines on when EPA may consider these past studies in future rulemakings. The commenter states that one such situation where such careful Agency guidance will be needed is those health-based standards (e.g., NAAQS) that EPA set based on restricted data in past rulemakings but must review those standards in the future on a periodic basis. In the case of such requirements, the commenter suggests that it may be appropriate for EPA to apply a phase-in of the transparency rules or establish a future effective date that would not require the transparency rules to apply until after the completion of the current five-year review cycle of the NAAQS for each criteria pollutant.
Response: As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and finalizing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available for independent validation. The EPA acknowledges that there may be pivotal science where the underlying dose-response data are not available in a manner sufficient for independent validation. In these cases, the EPA may still use these studies in significant regulatory actions and influential scientific information after either giving lesser consideration to the study or studies or if the Administrator grants an exemption under 40 CFR 30.7.
With this final rule, the EPA is clarifying that the requirements in this final rule set the overarching structure and principles for science transparency in the EPA's significant regulatory actions and influential scientific information. The EPA intends develop internal implementation guidance and promulgate either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule, which will clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes EPA administers.
 10.5.6 States Involvement in Identifying Priorities
Comment: Commenter (6104) asks whether states will have any role in helping EPA identify priorities if EPA decides to phase in the requirements or prioritize certain specific actions.
Response: The EPA does not find that a phase-in approach for implementing the final rule is necessary because of the limited application of this rule to pivotal science underlying significant regulatory actions and influential scientific information. The EPA would also like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the rule does not require the EPA to collect, store, or publicly disseminate data and models underlying pivotal science.
10.5.7 Transparency Rule Should Never Be Implemented
Comment: Commenter (0671) states that the Agency should not implement the proposed rule or any of its components at any time, now or ever.
Response: The EPA disagrees with the commenter. Because the EPA's significant regulatory actions can potentially impact American lives and livelihoods, the American people deserve environmental decisions and policies that are based on the best information, and only through transparency can the EPA truly demonstrate its commitment to the Agency's mission of protecting public health and the environment through sound policy decisions that are informed by robust scientific and technical research. 
10.6 Scope  -  Effects on Individual EPA Programs and Exceptions
10.6.1 Proposed Rule Would Have Effects on All/Several EPA Programs
Comment: Commenter (1012) states that if this rule is implemented as currently written, many relevant studies that rely on personal health information or CBI, may be excluded from the Agency's decision-making process. According to the commenter, this would reduce the total number of studies that utilize the best available science being used in the EPA's regulatory process. The commenter (1012) notes that restricting the number and type of scientific studies used would lower the overall quality of regulatory review at the Agency and would also impact the scientific integrity of the regulatory work conducted by the Agency. The commenter provides that, a non-comprehensive list of EPA programs that may be negatively impacted by the proposed rule include: the IRIS; the Integrated Science Assessment (ISA) for the NAAQS; Provisional Peer Reviewed Toxicity Values (PPRTV); the TSCA; pesticide reviews under FIFRA; other chemical review programs; and programs under the jurisdiction of the SDWA and CWA.
Commenter (4878) states that the proposed rule stands to affect every program and statute that the EPA administers. 
Response: In response to this comment, and other similar comments, the EPA clarified the relationship between this rulemaking and the environmental statutes, and their implementing regulations, in a 2020 SNPRM by adding language to proposed 40 CFR 30.3, stating that the statutes and regulations would control in the event of any conflicts. 
With this final rule, the EPA is maintaining language in 40 CFR 30.3 stating that the statutes and regulations will control in the event of any conflicts, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for science transparency in the EPA's significant regulatory actions and influential scientific information. The EPA intends to develop internal implementation guidance and promulgate either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule, which will clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes EPA administers.
As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and finalizing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available for independent validation. The EPA acknowledges, and agrees with commenters, that there may be pivotal science where the underlying dose-response data are not available in a manner sufficient for independent validation. In these cases, the EPA may still use these studies in significant regulatory actions and influential scientific information after either giving lesser consideration to the study or studies or if the Administrator grants an exemption under 40 CFR 30.7.
 10.6.2 Evaluating Effects of Proposal on Environmental Programs and their Implementation is Unnecessary
Comment: Commenter (6356) states that it is unnecessary for EPA to evaluate the effects of this rulemaking on individual environmental programs and their implementation. According to the commenter, this rulemaking addresses whether the data, model, or causal relationship (such as a dose and a response), is available to the public and whether the information on which a final agency action is based is replicable. The commenter notes that neither requires or warrants a particularized analysis of each statutory program that EPA administers, and such review would delay unnecessarily a much-needed reform of EPA's overall regulatory agenda. The commenter asserts that while each environmental statute has a distinctive basis for regulatory actions and some delineate what facts may or may not be considered, the proposed rule only attempts to ensure that facts and means of obtaining them, which establish the foundation for implementation of a particular statutory authority, are available to stakeholders and the public for review and comment. 
Response: As described in the final rule, the EPA is including regulatory text in 40 CFR 30.3 stating that the environmental statutes and regulations will control in the event of any conflicts. The EPA also clarified in the preamble of the final rule that the requirements in the final rule set the overarching structure and principles for science transparency in the EPA's significant regulatory actions and influential scientific information. The EPA intends to develop internal implementation guidance and promulgate either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule, which will clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes EPA administers.
 10.7 40 CFR 30.3 - How Do the Provisions of this Subpart Apply? 
 10.7.1 Unlimited Discretion and Authority to Decide What "Any Other Type of Agency Action" is or is Not 
Comment: Commenter (6133) asserts that 40 CFR 30.3 is unlawfully vague, open-ended and arbitrary due to the capacious and unlimited way that EPA has drafted the exclusion from the Proposal's prohibitions. The commenter states that 40 CFR 30.3 indicates that "the provisions of this subpart do not apply to any other type of agency action." According to the commenter, this grants EPA capacious and effectively unlimited discretion and authority to decide what "any other type of agency action" is and is not, without providing the public or regulated entities any criteria, understanding or advance notice as to how EPA will exercise that discretion and authority. The commenter contends that is the essence of arbitrary and capricious agency action. Indeed, according to the commenter, the Proposal is structured in such a way that EPA will be exercising that discretion and authority -- to decide what "any other type of agency action" does and does not cover -- in secret, with no public input and no public awareness, concerning the situations in which EPA will and will not consider non-transparent, unavailable information. In addition to this being perversely ironic, the commenter believes that considering the "transparency" title of the Proposal, this fact renders the Proposal even more arbitrary and capricious and unlawful.
Response: The Agency took comment on the applicability of this rule ant the EPA does not agree with the commenter and finds that 40 CFR 30.3, as finalized, clearly identifies the applicability of the final rule. As stated in 40 CFR 30.3, the provisions of 40 CFR part 30 apply to science that serves as the basis for informing a significant regulatory action or influential scientific information, as well as to data and models underlying pivotal science regardless of the source of funding or identity of the party conducting the science. Significant regulatory action and influential scientific information are both clearly defined in Part 30.2 of the final rule. 
 10.7.2 Applicability Covers Studies, Models and Analyses Integral to the Functioning of EPA Regulatory Programs
Comment: Commenter (6133) provides that the Proposal states that "[t]he provisions of this subpart apply to dose response data and models underlying pivotal regulatory science that are used to justify significant regulatory decisions regardless of the source of funding or identity of the party conducting the regulatory science." 83 Fed. Reg. at 18,773/3 (proposed 40 CFR 30.3). The commenter then states that the proposal defines "dose response data and models" and "pivotal regulatory science" and "regulatory science" in such a way that it appears to cover studies, models, and analyses that are integral to the functioning of EPA regulatory programs, implementation of statutes like the CAA, and protection of public health and the environment. The commenter lists and describes the following studies, models and analyses and their use in the support of existing regulations (pages 105  -  110 of their comment letter):
 Integrated Planning Model (IPM)
 National Electric Energy Data System (NEEDS) 
 Co-Benefits Risk Assessment (COBRA)
 Avoided Emissions and Generation Tool (AVERT)
 Community Multi-scale Air Quality (CMAQ) Modeling System 
 EPA U.S. Nine-Region MARKet ALlocation (MARKAL) Database
 Emissions & Generation Resource Integrated Database (eGRID)
 National Emissions Inventory (NEI) 
The commenter contends that it would be harmful, unlawful, arbitrary and capricious, and an abuse of EPA's discretion to include these materials within the sweep of the Proposal's prohibitions. Alternatively, the commenter states that if EPA disagrees that the listed examples are covered by the Proposal, then continuing to consider these materials that have the same hallmarks as the prohibited materials, and that raise the same issues and concerns that cause EPA to prohibit their consideration, demonstrates that the Proposal is arbitrary and capricious, biased, and internally inconsistent and contradictory. Moreover, the commenter provides that in this case, the Proposal would suffer from fatal failures to explain why EPA may consider these materials, while the Proposal would prohibit EPA from considering other materials.
Response: In 2020, the EPA issued an SNPRM and proposed to expand the scope of the 2018 proposed rulemaking in two ways: (1) expanding "dose-response data and models" to "data and models," and (2) including influential scientific information. As EPA noted in the 2020 SNPRM, transparency of EPA's science should not be limited to dose-response data and dose-response models, because other types of data and models can also drive the requirements and/or quantitative analysis of EPA significant regulatory actions and influential scientific information.
Based on the comments on the 2018 proposed rule and the 2020 SNPRM, including that the number of studies that would be subject to the rule might hinder rulemaking, the EPA determined that the Agency should pursue an incremental approach to maximizing transparency in the science that it relies upon by focusing the final rule requirements on dose-response data and, in particular, only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs).
The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. The provisions of the final rule apply to science that serves as the basis for informing a significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science.
 The 2018 proposed rule included proposed definitions for "dose-response data and models," "pivotal regulatory science," "regulatory decisions," "regulatory science," and "research data." Some commenters stated that several of the proposed definitions were unclear, including some that seemed to overlap (e.g., "pivotal regulatory science" and "regulatory science"). Some commenters also stated that certain terms used in the proposed regulatory requirements were not clear and should be defined. In response to these comments on the 2018 proposed rule, the EPA proposed in the 2020 SNPRM definitions for "capable of being substantially reproduced," "data," "independent evaluation," "models," "publicly available," and "reanalyze." In the 2020 SNPRM, the EPA also proposed a definition of "influential scientific information" to comport with the proposed expansion of the applicability of the rulemaking to influential scientific information. Based on a consideration of the public comments on both the 2018 proposed rule and the 2020 SNPRM, the EPA is finalizing the definitions at 40 CFR 30.2.
The final rule covers only dose-response data underlying pivotal science used for significant regulatory actions and influential scientific information and thus the scope of this final rule does not cover the models noted in the comments.
10.7.3 Drafted Exclusions from Proposal's Prohibitions
Comment: Commenter (6133) provides that §30.3 states that: 
      Except where explicitly stated otherwise, the provisions of this subpart do not apply to any other type of agency action, including individual party adjudications, enforcement activities, or permit proceedings. 
 83 Fed. Reg. at 18,773. 
Response: The Agency took comment on the applicability of this rule and the EPA does not agree with the commenter and finds that 40 CFR 30.3, as finalized, clearly identifies the applicability of the final rule. As stated in 40 CFR 30.3, the provisions of 40 CFR part 30 apply to science that serves as the basis for informing a significant regulatory action or influential scientific information, as well as to dose-reponse data and models underlying pivotal science regardless of the source of funding or identity of the party conducting the science. Significant regulatory action and influential scientific information are both clearly defined in Part 30.2 of the final rule.
The final rule does not prohibit EPA from relying on studies when the underlying dose-response data is not available for independent validation. Implementation of this rule will not undermine EPA's efforts in protecting human health and the environment. The scope of the rule is clear and EPA has articulated that its provisions will not apply to other agency actions. The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. The EPA will not expand the applicability of this rulemaking beyond significant regulatory actions and influential scientific assessments to individual party adjudications, permit proceedings, site-specific actions, and enforcement actions because these actions are typically focused on individual regulated entities.
Finally, as described in the preamble to the final rule, the requirements in this final rule set the overarching structure and principles for science transparency in the EPA's significant regulatory actions and influential scientific information. The EPA intends to develop internal implementation guidance and promulgate either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule, which will clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes EPA administers.
 10.7.4 Applicability to Government-Funded and/or Beyond Government-Funded Scientific Research 
Comment: Commenter (6362) provides that the EPA is proposing to add this rule to 40 CFR Part 30, contained in Chapter 1, Subchapter B, dedicated to "Grants and Other Federal Assistance," without explaining how or why this rule fits within this subchapter, thereby creating potential confusion regarding its applicability. According to the commenter, the potential for confusion was enhanced by the fact that EPA's public website currently contains information regarding the content that was formerly within 40 CFR Part 30 but was repealed on December 19, 2014, i.e., general terms and conditions applicable to grant recipient and sub-recipients. In addition, the commenter asserts that several questions on which EPA seeks comment relate solely to EPA cooperative agreements and grants or access to EPA-funded data.
In contrast, the commenter (6362) provides that 40 CFR 30.3 of the proposed regulatory text states that "the provisions of this section apply to dose-response data and models underlying pivotal regulatory science regardless of who funded or conducted the underlying data, models, or other regulatory science." The commenter asserts that stakeholders would benefit greatly from EPA providing clarification regarding the applicability of Subchapter B and whether and to what extent this rule applies to government-funded and/or beyond government-funded scientific research. The commenter believes the broader approach is warranted.
Commenter (1972) also strongly supports proposed 40 CFR. 30.3, stating that EPA's transparency requirements will apply "regardless of the source of funding or identity of the party conducting the regulatory science." Too often, according to the commenter, studies funded in whole or in part by industry are dismissed or given diminished credibility solely due to the source of the funding. Although the commenter recognizes the common presumption that the funding source may influence a study's results, these presumptions are rarely applied to studies performed or funded by nongovernmental organizations, academic institutions, or government agencies even though each of them may exert a significant influence over study results as well. Further, the commenter contends that industry may be the only party interested in funding the study of certain aspects of environmental or public health issues, allowing for a more robust scientific review of these issues than if the matter were left solely to the decision of others. Of course, the commenter states that the quality of scientific studies stand or fall on their underlying data, methodologies, and assumptions. The commenter provides that a transparent and open process that allows for outside validation or criticism of studies will reveal the quality of those studies regardless of the funding source.
Response: The EPA would like to clarify that 40 CFR part 30 is contained in Chapter 1, Subchapter A, which is titled, "General." The EPA finds this to be the appropriate subchapter for the requirements in the final rule. 
The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. The provisions of the final rule apply to science that serves as the basis for informing a significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science.
Comment: Commenter (2911) states that the federal government already has rules requiring that data be made publicly available for studies funded by the federal government. The commenter states that, if a need were to be demonstrated that additional regulation is needed to promote more transparency in scientific studies, then such regulation should apply to the studies themselves, not to the users of the studies.
Response: The transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency. The EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. The EPA also agrees with commenters that government agencies are making great strides in data transparency; however, improvements can still be made over existing policies and mechanisms. 
The rule will allow for greater scientific scrutiny as EPA decision makers are developing significant regulatory actions and influential scientific information and increases the likelihood that any errors in pivotal science can be identified and corrected through independent validation. Greater transparency is also inherently valuable as a principle of good government and provides benefits to the public at large, including reducing the risk of errors in EPA analyses and in the science such analyses rely upon. The ability for independent subject matter experts to validate pivotal science will facilitate more effective comment and engagement with the public during development of future significant regulatory actions and influential scientific information.
The EPA would like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the rule does not require the EPA to collect, store, or publicly disseminate data and models underlying pivotal science.
10.7.5 Data Pivotal to the Basis of a Regulatory Decision
Comment: Commenter (6356) states that proposed 40 CFR 30.3 and 30.4 appear to be inconsistent. The commenter states that proposed §30.3 would require EPA to identify all studies or other regulatory science that it relied upon when taking any final agency action and that such studies be available to the public for review. The commenter then provides that 40 CFR30.4 appears to presume that only dose response models and data are pivotal regulatory science. The commenter asserts that other research or data may be pivotal to the basis of a regulatory decision. The commenter states that this inconsistency appears to be at cross-purposes with the regulation and unnecessarily imbues the rule with a political purpose that detracts from its primary purpose.
Response: In the final rule, the EPA has revised the requirements of 40 CFR 30.4 such that they are consistent with the EPA's existing practice of making science that serves as the basis for informing a significant regulatory action available in the public docket as part of the rulemaking. As a result, the final regulatory text at 40 CFR 30.4 reads, "The EPA shall clearly identify all science that serves as the basis for informing a significant regulatory action. The EPA shall make all such science that serves as the basis for informing a significant regulatory action publicly available to the extent permitted by law."
In the final rule, the EPA has also revised the requirements of 40 CFR 30.3 to clarify how the provisions of the 40 CFR part 30 apply to EPA activities. As stated in 40 CFR 30.3, "The provisions of this part apply to science that serves as the basis for informing a significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science."
10.7.6 No Clear Information on How the Rule will Apply
Comment: Commenter (2169) states that the question "[h]ow do the provisions of this subpart apply" is not answered by the regulatory proposal. That is, according to the commenter, no clear information is stated in the Notice as to HOW the provisions of the subpart will be applied.
Response: In response to this comment, and other similar comments, the EPA has clarified how the provisions in the final rule will be applied. As described in the final rule, specifically Part 30.3, the provisions of this part apply to science that serves as the basis for informing a significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science. The provisions of this part apply to significant regulatory actions for which a proposed rule was published in the Federal Register after the publication date of the final rule and influential scientific information submitted for peer review after the publication date of the final rule. 
In addition, the provisions of this part do not apply to physical objects (like laboratory samples), drafts, and preliminary analyses, and influential scientific information or pivotal science that meet one or more of the exemptions identified in Section IX of the OMB Final Information Quality Bulletin for Peer Review. In the event the procedures outlined in this part conflict with statutes the EPA administers, or their implementing regulations, the statutes and regulations will control. Except where explicitly stated otherwise, the provisions of this part do not apply to any other type of Agency action, including individual party adjudications, enforcement activities, site-specific actions, or permit proceedings.
Finally, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and finalizing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider dose-response data availability when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available for independent validation. There may be pivotal science where the underlying dose-response data are not available in a manner sufficient for independent validation. In these cases, the EPA may still use these studies in significant regulatory actions and influential scientific information after either giving lesser consideration to the study or studies or if the Administrator grants an exemption under 40 CFR 30.7.
The EPA intends for the final rule to be implemented in a transparent manner. The final rule requires the EPA to document the rationale for exemptions granted by the Administrator to the requirements included in the final rule in the significant regulatory action or influential scientific information. Additionally, the EPA expects to identify pivotal science in proposed significant regulatory actions or external review drafts of influential scientific information, which will allow the public, if they so choose, to independently validate the pivotal science and provide comment to the EPA.
10.7.7 Physical Objects, Drafts and Preliminary Analyses Exclusion
Comment: Commenter (6880) states that, in describing the data to which the proposal applies, EPA states in 40 CFR 30.3 "[t]he provisions of this section do not apply to physical objects (like laboratory samples), drafts, and preliminary analyses." According to the commenter, as these are already excluded from the definition of "research data," it would be redundant for EPA to exclude them if they were using said definition. Also, the commenter asserts that, by mentioning these exclusions while remaining silent on trade secrets and privacy information that are excluded from the definition of "research data," EPA clearly signals that those data are not excluded from the "data" to which EPA refers in the proposal. The commenter contends that the proposal is clear that it applies to a much broader range of data than that included in the cited definition of "research data." The commenter states that this would allow EPA to ignore research findings if it were not possible to make public all the underlying data, including data that is excluded from the definition of "research data."
Response: Based on a consideration of this and other similar comments, the EPA issued a SNPRM in 2020, which included clarifications, modifications and additions to certain provisions proposed in the 2018 NPRM. In the 2020 SNPRM, the EPA deleted the 2018 proposed 40 CFR 30.2 definition of ``research data,'' because, as noted by the commenter, this definition excludes ``trade secrets, commercial information, materials necessary to be held confidential by a researcher until they are published, or similar information which is protected under law'' and ``[p]ersonnel and medical information and similar information the disclosure of which would constitute a clearly unwarranted invasion of personal privacy, such as information that could be used to identify a particular person in a research study.'' The EPA clarified in the 2020 SNPRM that these types of data would not be disclosed in violation of any statutory prohibitions. 
In the final rule, the EPA has defined "data" to mean "the set of recorded factual material commonly accepted in the scientific community as necessary to validate research findings in which obvious errors, such as keystroke or coding errors, have been removed and that is capable of being analyzed by either the original researcher or an independent party."
In the final rule, the EPA has also revised the requirements of 40 CFR 30.3 to clarify how the provisions of the 40 CFR part 30 apply to EPA activities. As stated in 40 CFR 30.3, "The provisions of this part apply to science that serves as the basis for informing a significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science." Because the provisions of 40 CFR part 30 apply to science that serves as the basis for informing a significant regulatory action or influential scientific information, the EPA finds it appropriate to further clarify that this science does not include physical objects (like laboratory samples), drafts, and preliminary analyses, and influential scientific information and/or pivotal science that meet one or more of the exemptions identified in Section IX of the OMB Final Information Quality Bulletin for Peer Review.
10.8 "Regulatory Decisions" Definition at 40 CFR 30.2
10.8.1 Amend Definition to Remove Limitation of Applicability to Only Final Regulations
Comment: Commenter (4932) recommends that EPA alter the "regulatory decisions" definition by striking the word "final" to make the Transparency Proposal fully consistent with Executive Order 12866. The commenter states that the necessary (and seemingly unintended) implication of including the word "final" in the definition is that the transparency objective (i.e., the disclosure of pivotal regulatory science) does not apply to advanced notices of proposed rulemaking or proposed rulemakings. The commenter asserts that the public must have access to pivotal regulatory science at these earlier steps in the rulemaking process in order to assess effectively the merit of the Agency's regulatory science. The commenter concludes that EPA's adoption of their recommendation would address this timing issue.
Response: The 2018 proposed rule included proposed definitions for "dose-response data and models," "pivotal regulatory science," "regulatory decisions," "regulatory science," and "research data." Some commenters stated that several of the proposed definitions were unclear, including some that seemed to overlap (e.g., "pivotal regulatory science" and "regulatory science"). Some commenters also stated that certain terms used in the proposed regulatory requirements were not clear and should be defined. The final rule no longer uses the term "regulatory decisions" and thus the term is not defined.
The preamble to the final rule also notes that EPA sees no need to include the proposed rule stage of significant regulatory actions in the final regulatory text for this rulemaking because, as a practical matter, these proposed rules must comply with this final rule in order to be finalized. The EPA does not introduce the studies and analyses it relies on for a rulemaking at the final rule stage. The basis for a rulemaking is provided for public review and comment in the public docket when the proposed rule is issued or if subsequently added to the docket through a separate opportunity for public comment. The EPA expects to identify pivotal science in proposed significant regulatory actions or external review drafts of influential scientific information, which will allow the public, if they so choose, to independently validate the pivotal science and provide comment to the EPA.
The EPA also sees no need to include advance notices of proposed rulemakings in this final rule because advance notices of proposed rulemakings are preliminary in nature and frequently focus on soliciting initial comment on a regulatory issue or approach. Therefore, advance notices of proposed rulemaking are not consistent with the purpose of this final rule. 
 10.8.2 Proposal is Inconsistent with Regards to What Types of Regulatory Actions Would be Covered
Comment: Commenter (6133) states that in some places the Proposal suggests that the rule would apply only to final rulemakings (e.g., "regulatory decisions" definition). See 83 Fed. Reg. at 18771 ("EPA solicits comment on whether and to what extent these requirements, or other provisions and policies, should apply to other stages of the rulemaking process . . .."). The commenter asserts that applicability of the rule to only final rulemakings is inconsistent with OMB guidance on Executive Order 12866 that states that the definition is intended to cover "any policy document of general applicability and future effect, which the agency intends to have the force and effect of law, such as guidance, funding notices, manuals, implementation strategies, or other public announcements, designed to implement, interpret, or prescribe law or policy or to describe the procedure or practice requirements of an agency." OMB, Memorandum for Heads of Executive Departments and Agencies, and Independent Regulatory Agencies, at 5 n.3 (Oct. 12, 1993). Therefore, according to the commenter, there is an inconsistency between what EPA says it is doing, and what it is really proposing. The commenter states that without knowing what types of agency actions would be covered, the public cannot understand the Proposal's true impact.
Response: The EPA agrees that the use of "regulatory decisions" is confusing, given that the term was only intended to apply to a subset of regulations. To clarify the definition, the EPA has changed the term from "regulatory decisions" to "significant regulatory actions" in the final rule. The EPA finds that the definition of "significant regulatory actions" used in the final rule (i.e., "final regulations determined to be `significant regulatory actions' by the Office of Management and Budget pursuant to Executive Order 12866") sufficiently identifies the applicability of the final rule. 
10.8.3 Regulatory Decisions Should Also Apply to Permitting Activities
Comment: Commenter (6180) recommends that the definition for "regulatory decisions" be revised to apply to permitting activities.
Response: The EPA does not agree with this comment and will not expand the applicability of this rulemaking beyond significant regulatory actions and influential scientific assessments to individual party adjudications, permit proceedings, site-specific actions, and enforcement actions because these actions are typically focused on individual regulated entities.
 10.8.4 Recommends that the Regulatory Decisions Definition Include EPA-Focused Criteria
Comment: Commenter (6906) provides that, throughout the proposed rule, EPA establishes that the guidelines for strengthening the transparency in regulatory science would only apply to "significant regulatory actions" as described by Executive Order 12866. The commenter understands why the Agency uses Executive Order 12866 as the basis for determining whether a regulatory action is significant; as its' four criteria have long been the standard to which the "significance" of a regulatory action is judged. The commenter states that they do not want to discount the decades-long use of this Executive Order with nearly every regulatory action but recommend that it would be useful for the Agency to develop more specific EPA-focused criteria. The commenter states that the EPA-focused criteria could take into account the range of regulatory and policy activities that the Agency conducts. According to the commenter, one possible way to meet this suggestion is by having each Program Office establish a list of rulemaking categories that would not be deemed significant, and thus minimizing any potential confusion. The commenter contends that using this approach could aid future regulators and stakeholders in future determinations as to when these transparency guidelines need to be accounted for. Furthermore, the commenter states that this could be one potential method of "narrowing the scope" of this proposed rulemaking without sacrificing the integrity of what the rule intends to do.
Response: The EPA is finalizing the applicability of this final rule to significant regulatory actions as defined in Executive Order 12866, and influential scientific information because of the broad impact of these actions and assessments. Influential scientific information means scientific information the Agency reasonably can determine will have or does have a clear and substantial impact on important public policies or private sector decisions. Following the Agency's Peer Review Guidance, each Program determines what science produced will be designated as influential scientific information. Also, as described in the preamble to the final rule, the requirements in this final rule set the overarching structure and principles for science transparency in the EPA's significant regulatory actions and influential scientific information. The EPA intends to develop internal implementation guidance and promulgate either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule, which will clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes EPA administers.
10.8.5 Basis for Regulatory Decisions
Comment: Commenter (0039) recommends that EPA require all regulatory decisions, meaning "significant regulatory actions," as defined by the OMB pursuant to Executive Order 12866, be based on:
 Scientific research that meets the "best available reproducible science" standard;
 At least one study on the effect of publication bias on this scientific research;
 At least one meta-analysis of this scientific research; and
 Explicit consideration of whether the corpus of scientific research has accounted for different aspects of the irreproducibility crisis, including flawed statistics, arbitrary research methods, publication bias, and disciplinary and/or political groupthink.
Response: As described in the preamble to the final rule, consistent with existing Agency practice (Ref. 35), the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information. The EPA will continue to use the following, established factors to assess the quality of studies used to develop significant regulatory actions and influential scientific information: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. 
When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence (consistent with Agency guidelines on hazard identification and dose-response assessment) to support a relationship between exposure and effect, the EPA will identify a subset of those studies for use in characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect. This will be based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both will be identified as pivotal science (see Section III.E of the final rule preamble). Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of personally identifiable information (PII), CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator (see Section III.G). See Section III.E for a description of the factors the EPA will consider when determining the consideration to afford to pivotal science when the underlying dose-response data are not available for independent validation.
The EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
The specific request found in these comments regarding the scientific basis for or other concerns associated with a significant regulatory action will be in part addressed through the availability of dose-response data underlying pivotal science.
 Chapter 11: Availability of Data and Models
The proposal Federal Register preamble states that the proposed rule is intended to strengthen the integrity, transparency, and validity of the scientific information relied upon by EPA (83 FR 18769). The proposal notes that, in applying the best available science to its regulatory decision-making, EPA must comply with federal transparency and data integrity laws (83 FR 18769). Several comments were received regarding the integrity and transparency of regulatory decision making. In this section these comments are organized as follows:
Section 11.1: NPRM Transparency and Integrity of Regulatory Decision Making vs. Availability of Raw Data 
       Basis for EPA Decision Making Regarding Use/Non-Use of Studies Not Required to be Transparent; and 
       EPA Actions in Other Areas.
Section 11.2: SNPRM Data Availability-Related Comments 
       Confidential Business Information, Proprietary Data, or Personally Identifiable Information
The specific comments provided in this Chapter reflect the comment letters that provide arguments supported by detailed rationale, examples, and references/documentation. The Docket ID numbers for these commenters are cited throughout this Chapter. In addition, some general comment letters may also include comments related to comments included and responded to in this Chapter. These comments are summarized in Appendix B. Note that points and issues raised in these general comment letters may also be raised by the more detailed comments summarized in this Chapter. The responses provided in this Chapter adequately address these more general comments.
 11.1. NPRM Transparency and Integrity of Regulatory Decision Making vs. Availability of Raw Data 
 11.1.1 Basis for EPA Decision-Making Regarding Use/Non-Use of Studies Not Required to be Transparent
Comment: Commenter (5176) states that the proposed rule does not enhance transparency in science. The commenter states that the proposed rule undermines its claim of promoting transparency within the EPA regarding environmental policymaking. According to the commenter, the proposed rule does not say that a scientist has to make her study data publicly available, if the EPA is to consider using that study to inform policy; rather, it says that her study will not be considered as a basis for EPA decision-making. The commenter adds that the proposed rule requires no transparency regarding when and under what circumstances the EPA might decide unilaterally that a rigorous, independently validated, peer-reviewed study cannot be used as a basis for regulatory policymaking, solely on the grounds that its underlying data are not publicly available. Under the proposed rule, the commenter notes that the public would never know that a decision not to use a particular study had been made. The commenter contends that, in that sense, the proposed rule will foster a lack of transparency about EPA decisions and the science that serves as the basis for its regulatory policy.
Commenter (9224) asserts that, in order to increase transparency, it would be important to have the agency give a thorough explanation as to why certain studies and data sets were chosen and not others, rather than relying on public access to the individual data. The commenter states that the public does not need the individual data in order to determine if there are inherent issues within the study itself. The commenter states that the analysis of the data is already done multiple times throughout the publication process, via peer review. The commenter states that, while it is useful to have the data EPA uses, it is perhaps more important to understand the methodology and reasoning behind why EPA chooses the data it does. 
Response: The EPA disagrees that this rule will foster decreased transparency in regulatory decision-making. In this final rule, the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. When characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect, the EPA will identify pivotal science and give greater consideration to pivotal science for which the underlying dose-response data are available in a manner sufficient for independent validation. As stated in 40 CFR 30.5(c), the Agency will identify the pivotal science that was given lesser consideration and provide a short description of why lesser consideration was given.
As codified in this rule in 40 CFR 30.4, EPA will continue its practice of making public the science that serves as the basis for informing a significant regulatory action available in the public docket. The EPA has established various guidelines for how it assesses scientific evidence in its regulatory actions and influential scientific information, such as the Framework for Human Health Risk Assessment to Inform Decision Making. The Agency will continue to follow this and other guidelines under this rule. Consistent with the intent of this rulemaking, the EPA also intends to clearly identify the studies considered pivotal in the documentation at the proposed rule stage for significant regulatory actions and when influential scientific information is disseminated for peer review.
Comment: Commenter (6134) asserts that the proposed rule puts EPA in the position of defining "transparent" and allows them to decide if particular studies should be included in their rule making process. The commenter states that it is not designed to make an already robust review process better. 
Response: The EPA disagrees with the commenter that this rule allows the Agency to decide if particular studies should be included in the rulemaking process. Under this final rule, no studies are categorically excluded from consideration. Rather, the purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. Also see the response to the preceding comment. 
11.1.2 EPA Actions in Other Areas 
Comment: Commenter (6168) states that the litany of ways the EPA under the current administration has been secretive and has sought to limit information and evidence of chemical harms suggests that the idea of "transparency" grows not out of commitment to openness and public participation. The commenter states that, rather, it is being used here strategically, in bad faith, to limit the evidence used in regulatory decision-making and to undermine moves toward protective regulations. The commenter lists the following examples:
 EPA has made less accessible or removed extensive information about climate change that had been available on EPA's website.
 EPA has failed to provide the information necessary for independent evaluation of its decisions under Toxic Substances Control Act (TSCA), e.g. EPA's recent decision to green light a new fragrance chemical despite a range of health concerns.
 EPA has pushed to delay releasing reports documenting harms of chemicals, e.g. formaldehyde and Per- and Polyfluoroalkyl Substances (PFAS). 
 EPA has reduced opportunities for public participation by creating short comment periods and trying to circumvent public comment entirely.
 EPA's procedure for implementing TSCA is designed to limit the range of information included in regulatory decisions by limiting the sorts of uses/exposures that will be analyzed. 
 EPA has drastically shifted influence away from the broader public and toward business interests, for example in private meetings and in appointments to science advisory boards.
 Especially under Scott Pruitt -- who was Administrator when this proposed rule was written and published -- EPA has been secretive, for example denying Freedom of Information Act (FOIA) requests until required to release documents by courts. Pruitt's secrecy was famous: not telling reporters where he was speaking, not letting the press into events, hiding, and even changing his calendars, etc.
Response: The EPA disagrees that this final rule is meant to limit the evidence used in regulatory decision-making and to undermine moves toward protective regulations. Rather, the purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. Under this final rule, no studies are categorically excluded from consideration. Where the underlying data is available, EPA will give greater consideration to the pivotal science. Where the underlying data is unavailable, EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator under 40 CFR 30.7.
Comment: Commenter (6868) suggests several discrete actions the Agency could take to increase transparency and public trust in how EPA develops policy other than the proposed rule:
       Making the schedule of the EPA Administrator/Acting Administrator available to the public,
       Granting media representatives full access to public meetings,
       Ensuring public access to factual information on the environmental and public health effects of climate change,
       Minimizing the role regulated industry representatives play on EPA advisory committees, and
       Meeting court ordered deadlines for promulgation of major regulatory actions and providing full cooperation with any pending federal investigations of agency staff misuse of office.
Response: These suggestions are outside the scope of this rule. The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions.
11.2 SNPRM Data Availability-Related Comments
11.2.1 Confidential Business Information, Proprietary Data, or Personally Identifiable Information
11.2.1.1 40 CFR 30.5 Requirements
11.2.1.1.1 Deletion of Research Data Definition
The supplemental proposal states the EPA is deleting the 2018 proposed 40 CFR 30.2 definition of research data because this definition excludes trade secrets, commercial information, materials necessary to be held confidential by a researcher until they are published, or similar information which is protected under law and personnel and medical information and similar information the disclosure of which would constitute a clearly unwarranted invasion of personal privacy, such as information that could be used to identify a particular person in a research study. The EPA proposes that these types of data are intended to be subject to this rulemaking. 
Comment: Commenters (10038, 10740, 11361, 12710) oppose the proposal and express concern over the deletion of the definition of "research data."
Commenter (10740) suggests that it is the intent of this proposed rule to preclude scientific evidence based on data and models informed by raw data, including medical records, that is required to be held confidential and concludes that this is not transparency, nor is it science.
Commenter (11361) asserts that the effect of the proposed deletion is readily apparent: it will dramatically shift the balance of the kinds of studies that will be utilized in Agency decision-making, reducing the proportion of public health studies and increasing the proportion of assay-based and computational modeling studies. The commenter states that this is likely to make the link between regulations and health protections less clear, not more. 
Commenter (11361) asserts it is an ill-conceived move by the EPA to put current researchers in the position of either complying with privacy laws or allowing their research to have real-world impacts by informing regulations. The commenter notes that in this supplemental notice of proposed rulemaking (SNPRM or supplemental proposal), the EPA explicitly bucks the use of the term "research data" in the 2018 proposed rule because "research data" excluded data that was protected under various privacy laws, such as confidential business information (CBI) and personally identifiable information (PII), and the EPA is requiring that that protected information be made public or be disregarded. 
Commenter (12710) states that "research data" excludes the "physical objects (like laboratory samples), drafts, and preliminary analyses" that proposed § 30.3 will exclude. The commenter states that research data also excludes trade secrets and personal and medical information that if disclosed would constitute an invasion of privacy. The commenter states that this is not excluded from EPA's new definition of data; instead, throughout the proposal studies that allow "tiered access" to PII are mentioned and yet offer very little detail in the preamble (no sources) and none in the proposed regulation as to what constitutes acceptable "tiered access."
Response: While the EPA maintains the approach of the 2020 supplemental proposed rule, that the types of data included in the proposed definition for "research data" should be subject to this final rule, the EPA emphasizes that this final rule does not require that data be made publicly available. Rather, this final rule of internal agency procedure requires that the EPA give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. 
The EPA acknowledges, and agrees with commenters, that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. This may include studies containing protected information from the proposed definition of "research data." In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
11.2.1.1.2 Protected Data 
Comment: Commenter (12472) states that the data access requirements described in the supplemental proposal may pose difficulties for emissions inventories used in modeling. The commenter states that, when developing emissions inventories for modeling, there are instances where CBI, PII, or proprietary data is used but cannot be shared in raw form due to access agreements with originators of that data. The commenter states that, for instance, on-road mobile, commercial marine vessels, rail, area, or oil and gas inventories are often constructed with PII, CBI, or proprietary data. The commenter states that the economic growth factors used in forecasting emissions may be constructed with code or models that are considered CBI or proprietary data. The commenter states that the Texas Commission on Environmental Quality is legally prohibited from releasing some of these datasets, but it is necessary to collect such information in order to prepare emissions inventories for air quality modeling. The commenter states that, in these instances, as allowed in the supplemental proposal, the EPA should consider studies and other data even if the underlying data and models are not publicly available.
Response: The EPA acknowledges that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. This may include modeling studies that rely on underlying CBI, PII, or proprietary dose-response data. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
Comment: Commenter (12718) expresses concern that the supplemental proposal offers no meaningful articulation of privacy and confidentiality safeguards for the American people. The commenter states that the Agency has no established statistical Agency or unit designated under the Confidential Information Protection and Statistical Efficiency Act of 2018 (Title 3 of PL 115-435) to apply civil and criminal penalties for violations of protection. The commenter states that this is disappointing in that these protections were unanimously upheld by the Republican and Democratic appointees to the U.S. Commission on Evidence-Based Policymaking as a model for federal agencies, including EPA, then reauthorized with strong bipartisanship in Congress. 
Response: The EPA clarifies that this rule is not intended to establish enforcement of civil and criminal penalties for violations of data protections. Rather, this internal rule establishes procedures for the EPA to give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. 
11.2.1.2 Confidential Business Information (CBI)
11.2.1.2.1 Transparency Standards for CBI and Public Health Data
Comment: Commenters (10339, 11403, 11479, 12403, 12442, 12455, 12702) express concern that, under the proposed rule, CBI may not be held to the same standards of transparency as public health information. Commenter (12442) states that the rule is arbitrary and capricious because this rule creates a double standard by allowing specific industry data to be kept confidential while exposing sensitive data underlying health-based studies.
Commenter (10339) states that the rule and supplemental proposal still fail to explain whether the Agency will ensure that CBI will also be held to the same standards of transparency, disclosure, and access. Commenter (12403) asserts the proposal's approach to CBI favors private sector research over independent institutions. 
Commenter (11403) asserts that the proposal would result in EPA policy being based on secret industry CBI that has been protected under this proposed rule. The commenter notes that the proposed rule restricts the use of the exemption to circumstances where "compliance is impracticable because technological barriers render sharing of data or models infeasible." The commenter asserts that, in practice, this exemption would result in EPA policy being based exclusively on secret industry CBI that has been protected under this proposed rule and potentially outdated studies that have been exempted by the Administrator under this section 30.9. The commenter states that, under this exemption, data that is not publicly available because of a technological barrier would arbitrarily be considered prioritized over health data that would be blocked from consideration by this proposed rule.
Commenter (11479) suggests the EPA appears to be more concerned with protecting industry data and CBI than with protecting public health information. The commenter states that this disparity is exemplified by the fact that the Agency requests commenters to provide feedback on how to balance protections for copyrighted and CBI but does not seek similar advice on how to protect patient health data. The commenter states that emails show that top EPA officials responsible for drafting the proposed rule struggled with how to incorporate exemptions and limit the rule's impact on industry data. The commenter states that Dr. Beck's comments on protecting CBI also resemble her testimony in front of a Senate Homeland Security and Government Affairs subcommittee, where she represented her former employer, the American Chemistry Council, whose members have a significant financial interest in EPA regulatory matters. The commenter states that the proposed rule and supplemental proposal's focus on protecting industry over protecting the public health research community illustrates the Agency's disinterest in seeking actual transparency in decision making; instead, the EPA is seeking an opportunity to protect data for the industries it is supposed to regulate, at the expense of science-based public health safeguards. The commenter adds that this bias is affirmed by Dr. Yamada's response to an email from Dr. Beck, in which he writes that he "didn't know about the intricacies of [confidential business information]" and that the Agency would have to "thread this one real tight!"
Commenter (12442) contends that, while the proposed rule would subject studies to additional scrutiny by exposing the privacy of participants and raise the standards by which they can be considered, the rule would allow for the continued confidentiality of equally critical and often much more opaque business information such as economic cost modeling or industry labeled proprietary data, which is rarely subject to verification at all, let alone rigorous peer review. The commenter states that this approach would only serve to eliminate the strong and well-founded basis for health and environmental regulations millions of Americans depend on while keeping information deemed confidential by for profit industry interests out of the public eye. The commenter states that the dividing line drawn by this proposal creates two camps: the privileged, industry data kept out of the public eye and often used to avoid protections like pollution controls for heavily emitting sources such as coal-fired power plants and oil refineries; and the epidemiological data prejudiced by EPA's proposed transparency rule that is used to support protections for the benefit of public health and welfare. The commenter states that this is a text book example of an arbitrary and capricious rulemaking.
Commenter (12442) states that, in the prevention of significant deterioration (PSD) air permitting process, permitting decisions often rely on CBI. The commenter states that, as a result, the public is barred from commenting on, let alone knowing, the information on which an effective emission control may be bypassed in favor of a lax permit that would result in negative air quality impacts that could otherwise be avoided. The commenter states that a rule truly designed to increase transparency should require the data of source operators and owners to be released to the public.
Commenter (12455) contends the supplemental proposal allows companies to manipulate which studies would be considered by declaring them CBI, or by seeking to publish the study in a journal that prohibits sharing data pre-publication. The commenter is also concerned that studies published in subscription journals vs. open source may be treated differently.
Commenter (12702) states that, under the proposed rule and supplemental proposal, the EPA would be required to establish an exemption process to utilize CBI or alternative process to give higher weight to data that did not include CBI. The commenter is concerned that either approach could decrease the consistency with which data are treated and increase the discretion of EPA in making such determinations. The commenter states that CBI from regulated or effected entities is in some cases critical to determine the most appropriate standard, control technology, and regulatory requirements. The commenter states the EPA should not finalize a rule, such as this one, that decreases transparency of the rulemaking process by obscuring the basis of and process for taking regulatory actions.
Response: To take into consideration studies that involve PII, CBI or proprietary data, the EPA is finalizing an approach that takes gives greater consideration to pivotal science whose underlying dose-response data involve PII, CBI or proprietary data are available through restricted or tiered access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
11.2.1.2.2 Plan for CBI
Comment: Commenters (10974, 11378, 11491, 11496, 11894, 12455, 12465) contend that the EPA does not sufficiently describe how the proposed rule would cover CBI. Commenter (11378) contends the criteria for evaluation and plans for long-term management regarding research utilizing confidential and proprietary data remain unclear in the supplemental proposal. Commenter (11496) states the EPA has not described how pivotal peer-reviewed science from other countries would be incorporated, including CBI.
Commenter (11491) asserts that, under the proposal as supplemented, there would be uncertainty about how CBI submissions would be treated. The commenter states that understanding the costs and practicability of compliance the EPA often relies upon regulated industry to submit CBI. The commenter notes that, where CBI is excluded, compliance costs are based on outdated numbers or technology. The commenter states that CBI might be at increased risk of public release, creating a disincentive for companies to voluntarily submit information, and Agency use of legally protected CBI might be disfavored, depriving the rulemaking process of important information. 
Commenter (11894) states that, much, if not most, of the information used to determine if a chemical may present an unreasonable risk to human health or the environment or would result in substantial human exposure or release to the environment is claimed as CBI and, thus, it could not comply with the transparency rule. The commenter states that how the EPA will deal with this remains to be seen as the proposed rule merely states that it is an issue to be worked out. The commenter states that, under the 2016 TSCA amendments, the EPA must make an affirmative finding on the safety of a new chemical or significant new use of an existing chemical before it is allowed into the marketplace.
Commenter (12455) states that the supplemental proposal needlessly complicates the use of pesticide reviews, which are traditionally based on CBI studies. The commenter states that it is unclear whether older studies, when they come up for review, will be completely excluded from consideration as they would not qualify under the supplemental proposal.
Commenter (12465) is concerned with the lack of details around appropriate access to CBI. The commenter notes that there will be situations in which CBI cannot be shared publicly, but this should not preclude the inclusion of the CBI in regulatory decisions. The commenter states that, while noting that "[n]othing in the proposed rule compels the disclosure of any confidential or private information in a manner that violates applicable legal and ethical protections," the lack of specificity and applicability is found wanting and the Agency should expand this guidance at great length should the rule go forward.
Response: Under this rule there is no categorical exclusion of pivotal science if the underlying dose-response data are not available for independent validation. Thus, dose-response data involving PII or CBI that cannot be made legally or ethically available or proprietary information that cannot be made available through either law or contractual agreement could still be considered as pivotal science. Availability of dose-response data would be done in accordance with existing laws and contractual agreements.
To take into consideration studies that involve PII, CBI or proprietary data, the EPA is finalizing an approach that takes gives greater consideration to pivotal science whose underlying dose-response data involve PII, CBI or proprietary data are available through restricted or tiered access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Even if the underlying dose-response data are not available for independent validation, the pivotal science including these dose-response data may still be considered as pivotal science, although it would be given lesser consideration than pivotal science where the underlying dose-response data are available, including through restricted or tiered access, for independent validation. 
Comment: Commenter (12453) appreciates that the EPA has continued in the supplemental proposal to confirm that it will apply the rule consistent with statutory requirements to protect proprietary data and CBI. 
Response: EPA thanks the commenter.
11.2.1.3 Proprietary Data
11.2.1.3.1 General
Comment: Commenter (12430) expresses concern that the term "proprietary data" in 40 CFR 30.5 is not defined.
Response: EPA is not providing a definition of proprietary data because this rule does not regulate such data or treat such data categorically. This rule describes the extent to which EPA will give consideration to pivotal science in which the underlying dose-response data are available or are not available for independent validation. 
Comment: Commenter (11378) asserts that the proposal to make information publicly available would potentially violate legally binding proprietary data agreements. The commenter states that deemphasizing geospatial research in which proprietary data agreements necessitate restrictions to public data access would severely limit the EPA's mission to protect human health and the environment, with consequent deleterious effects on environmental and health policy among state and local government agencies.
Commenter (12430) expresses concern that restricting models to those that are publicly available, or that meet the vaguely described exceptions in proposed 40 CFR 30.5, would severely limit the science that the EPA may consider or reference. The commenter states that, while models are critical to help humans assess and analyze large quantities of data, they are nearly always proprietary. The commenter adds that most scientists and agencies purchase access to and use these models for data.
Response: This rule does not require any dose-response data be made publicly available. Thus, there is no categorical exclusion for dose-response data that includes proprietary data. Availability of dose-response data would be done in accordance with existing laws and contractual agreements.
To take into consideration studies that involve PII, CBI or proprietary data, the EPA is finalizing an approach that takes gives greater consideration to pivotal science whose underlying dose-response data involve PII, CBI or proprietary data are available through restricted or tiered access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Even if the underlying dose-response data are not available for independent validation, the pivotal science including these dose-response data may still be considered as pivotal science, although it would be given lesser consideration than pivotal science where the underlying dose-response data are available, including through restricted or tiered access, for independent validation.
11.2.1.3.2 Plan for Proprietary Data
Comment: Commenter (11378) expresses concern that the criteria for evaluation and plans for long-term management regarding research utilizing proprietary data remain unclear in the supplementary proposal.
Response: The rule imposes no requirements on the criteria for evaluation and plans for long-term management regarding research utilizing proprietary data.
Comment: Commenter (12453) appreciates that the EPA has continued in the supplemental proposal to confirm that it will apply the rule consistent with statutory requirements to protect proprietary data, including the Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA) and Federal Food, Drug, and Cosmetic Act (FFDCA) provisions. Commenter (12465) recommends that the EPA ensure that any new guidelines comply with applicable laws and encourages the use of all relevant scientific information, regardless of whether the underlying data can be publicly disclosed, especially with respect to data and information developed individually or via consortia under the requirements of FIFRA or TSCA.
Response: EPA will comply with all applicable laws. It will continue to consider all relevant pivotal science in its significant regulatory actions and influential scientific information. 
11.2.1.3.3 Tribes
Comment: Commenter (12423) states that, while the proposed rule addresses public health studies, for tribal peoples many of these studies are intertwined with proprietary lifeways including subsistence activities, ceremonies, and spiritual beliefs. The commenter notes that, for example, consumption patterns might reveal locations or numbers of wildlife and plant harvest areas that tribes wish to keep private or might reveal information about how and what a tribe eats. The commenter states that, in the case of the retrospective cohort study for waste disposal practices, self-reported daily subsistence food consumption was included to determine whether these foods provided a protective benefit for the surrogate exposures studied. 
Commenter (12423) states that fish consumption is a primary pathway of exposure for multiple chemicals that are released to aquatic systems from a multitude of source and, for many tribes, fish is consumed in far greater quantities than the general population. The commenter notes that it is therefore critical to document the extent of fish consumption in tribes when setting water quality standards and developing water quality policies, either at a national level, state level, or project-specific level in the case of contaminated site cleanups. The commenter further notes that because the fish consumed is not from the grocery store but obtained through traditional practices on tribal lands, community level studies must be carried out. The commenter states that in doing so, not only is the small-scale population problem tripped, but the reluctance of tribal members to share their practices is substantial. The commenter states that tribal members have specific locations in which they harvest fish and some tribes have had conflicts with local and state officials on their treaty rights to harvest the fish or other species. The commenter provides that some tribal members are reluctant to reveal that they may eat less fish than they could or fear that they may have violated some ordinance through subsistence and ceremonial fishing. The commenter states that subsistence is intricately tied to tribal values, beliefs, and lifeways, and is often a spiritual exercise.
Commenter (12423) asserts that, while the EPA is not in the business of collecting and reviewing data on the religious practices and beliefs of American citizens, the proposed rule essentially asks just that of tribes. The commenter states that, by placing tribes in the position of either revealing aspects of their spiritual lifeways that are not relevant to the study purpose or not being protected by the nation's environmental laws, the proposed rule may have implications for the American Indian Religious Freedom Act of 1978 (AIRFA) (42 U.S.C. § 1996.).
Response: This final rule does not require data revealing proprietary lifeways or personal religious practices be made publicly available. In this final rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA acknowledges that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. This may be the case for exposure studies evaluating tribal fish consumption patterns. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. Where the Agency is making dose-response data publicly available, it shall do so in a fashion that is consistent with law, protects privacy, confidentiality, CBI, and is sensitive to national security.
11.2.1.4 Personally Identifiable Information (PII)
11.2.1.4.1 Publicly Available Consistent with Law
Legal Protections Could Result in Exclusion of Studies
Comment: Commenters (9372, 10040, 10865, 10866, 10975, 11359, 11361, 11378, 11184, 11396, 11492, 11576, 12715) express concern that studies will be excluded from consideration by the EPA because legally mandated privacy protections prohibit the proposed requirement to make data publicly available.
Commenter (9372) states that pollution can affect cognitive outcomes, like academic test scores and cognitive disabilities, in children. The commenter states that the Family Educational Rights and Privacy Act requires that any educational records be kept private on secure servers because the release of such information could harm students. The commenter states that this is a ridiculous request that's designed to remove research from policymaking with no benefit. The commenter states that most of the best papers use restricted data because almost no individual data are publicly available anymore. The commenter states that only aggregated data can be released publicly, and aggregated data is often not as good as individual-level data, particularly if there is heterogeneity in the outcomes. 
Commenter (10040) asserts data privacy laws largely prevent information from studies involving humans from being made public in the manner being considered by the EPA. The commenter states that, by limiting EPAs ability to use rigorously conducted peer reviewed literature simply because they do not meet transparency standards stands to block consideration of research that is critical to decision making to protect the public's health.
Commenter (10865) states that there are many credible scientific studies where the exposure of raw data to the public is infeasible or would reveal confidential patient information.
Commenter (10866) notes that many studies are conducted on large datasets such as those that draw on hospital records. The commenter states that the proposal, by requiring that the entire dataset be made available, would expose to public scrutiny data that would allow the identification of individual persons included in the study. The commenter notes that, because this would violate legally mandated privacy protections, this step would result in the de facto exclusion from EPA consideration of valuable and significant research studies.
Commenter (11378) notes that after reviewing the supplementary proposal associated with Strengthening Transparency in Regulatory Science Proposed Rulemaking (EPA-HQ-OA-2018-0259-9322), they are concerned that by giving primacy to research in which data and models are made publicly available the proposed rule will limit the use of valid scientific research for promulgating environmental regulations and finalizing scientific information. The commenter contends that the integrity of much environmental and health research requires the tradeoff that certain business, proprietary, and PII remain confidential. The commenter adds that making such information publicly available would potentially violate long-standing human subjects research protocols (i.e. the 1974 National Research Act), as well as legally binding proprietary data agreements.
Commenter (11184) asserts that the supplemental proposal does not address the fact that many past and ongoing public health studies simply cannot lawfully, ethically, or efficiently publicly release the underlying data. The commenter states that, even if applied only to future research, the supplemental proposal would limit participation, limit the information that can be collected, and waste limited resources. The commenter states that proposed 40 CFR 30.5 remains a poorly formed scheme in search of a problem that would ultimately force the EPA to arbitrarily make public health and environmental decisions based on incomplete or improperly weighted information.
Commenter (11396) asserts that the EPA's proposal would, despite any clarifications and modifications made in the supplemental proposal, block the use of studies that rely on confidential patient information from being used in policymaking. The commenter states that many studies, including older studies, depend on or have historically used such data that legally cannot be made public. The commenter states that the fact that this information must be kept confidential to protect patients does not make the data any less valid.
Commenter (11360) states that it is understandable that when issuing regulations, scientific information helps to provide the rationale for decisions; however, not being able to publicly disclose all sensitive data does not justify the EPA dismissing the use of important scientific work that could very well inform regulatory and policy decisions.
Response: In this final rule the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
In this final rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA acknowledges that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. This may be the case, for example, for studies that rely on health records containing PII. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. Where the Agency is making dose-response data publicly available, it shall do so in a fashion that is consistent with law, protects privacy, confidentiality, CBI, and is sensitive to national security.
Comment: Commenter (12596) suggests the EPA must balance the rule's sound policy goals with the Agency's legal obligation to be a good steward of private, sensitive, and confidential information. The commenter states that statements in the supplemental proposal "clarifying" that the EPA does not intend to make data and models public for which the Agency lacks access or does not have the authority to provide access. The commenter appreciates reassurances in the supplemental proposal concerning sensitive information and anticipates that the Final Transparency Rule can be implemented in a manner that moots disclosure concerns. For the sake of clarity, the commenter reiterates the request from their proposal comments that the EPA explicitly state that the rule does not assert authorization under the law to disclose protected sensitive or confidential information, and that any such claim must be independently based on a statutory grant of authority from Congress.
Response: The EPA confirms that this final rule does not assert authorization under the law to disclose protected, sensitive, or confidential information. This is a rule of internal agency procedure and does not require the EPA or any entity outside the federal government to make data available. Where the Agency is making dose-response data publicly available, it shall do so in a fashion that is consistent with law, protects privacy, confidentiality, CBI, and is sensitive to national security. Finally, as this rule makes clear, if implementing this procedural rule would result in conflicts with existing statutes, and their implementing regulations, this rule will yield to the statutes and regulations.
Comment: Commenter (11496) asserts the EPA does not provide the tools necessary for this rule to function without causing a conflict with established Federal data privacy rules. 
Commenter (12403) contends the proposal undermines privacy protections for participants in human subject research. Commenter (12458) states that confidentiality of health data in research is often ethically, if not legally required. 
Response: The EPA disagrees with the commenters that this rule would conflict with either Federal data privacy laws or regulations, or the ethics of patient confidentiality. This is a rule of internal agency procedure and does not require the EPA or any entity outside the federal government to make data available. Where the Agency is making dose-response data publicly available, it shall do so in a fashion that is consistent with law, protects privacy, confidentiality, CBI, and is sensitive to national security. Finally, as this rule makes clear, if implementing this procedural rule would result in conflicts with existing statutes, and their implementing regulations, this rule will yield to the statutes and regulations.
Common Rule
Comment: Commenters (9372, 11182, 12422) express concern that the supplemental proposal may not be consistent with the Common Rule.
Commenter (9372) asserts that the requirement that researchers post data publicly violates the Common Rule guidelines for Human Subjects research. 
Commenter (11182) states that a key duty in the American College of Epidemiology's ethical guidelines is the protection of confidentiality and privacy., The commenter states that this duty echoes the obligation to maintain patient confidentiality in Occupational and Environmental Medicine (ACOEM) Code of Ethics, as it does for all professional medical and scientific organizations. The commenter states that the 1979 Belmont report laid the groundwork for privacy and confidentiality in human research. The commenter states that the ethical principles guiding the report and underlying the ethical practice of science and medicine are autonomy (respect for persons), beneficence (do no harm), and distributive justice (equitable risks and benefits). The commenter states that the Belmont report led directly to the 1991 Common Rule, which set the regulations for the ethical conduct of human-subjects research., The commenter states that in the 2018 updated Common Rule, confidentiality protections for people who volunteer to participate in research were strengthened. The commenter states that the public must feel confident that their private information is protected, and researchers must be able to assure their research subjects that confidentiality is maintained. The commenter states that the supplemental proposal rejects this basic ethical tenet. The commenter states that even with the unexplained and untested "tiered access" proposition, researchers will be unable to provide that reassurance.
Commenter (12422) states that, depending on how it is implemented and interpreted by the Administrator, under this regulation, human studies could only be used by EPA if the investigators surrender confidential data that the investigators had promised the study subjects and institutional review boards (IRBs) would never be released. The commenter states that, while the EPA might agree to redact these data before public posting, they could be given to anyone who signed a confidentiality agreement designed by the agency. The commenter states that implications for the protections of human subjects and informed consent under the Common Rule (the Federal Policy for the Protection of Human Subjects) have yet to be evaluated.
Response: The EPA disagrees with the commenters that this final rule would either require the release of human subject research data in conflict with the Common Rule, or cause the EPA to categorically exclude human health data from consideration in EPA significant regulatory actions or influential scientific information. The EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
In this final rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA acknowledges that there may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. This may be the case, for example, for studies that rely on human subject research data. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. Where the Agency is making dose-response data publicly available, it shall do so in a fashion that is consistent with law, protects privacy, confidentiality, CBI, and is sensitive to national security.
Health Insurance Portability and Accountability Act (HIPAA) and Institutional Review Boards (IRBs)
The HIPAA Privacy Rule establishes national standards to protect individuals' medical records and other personal health information and applies to health plans, health care clearinghouses, and those health care providers that conduct certain health care transactions electronically. The Rule requires appropriate safeguards to protect the privacy of personal health information, and sets limits and conditions on the uses and disclosures that may be made of such information without patient authorization. The Rule also gives patients control and rights over their health information, including rights to examine and obtain a copy of their health records, and to request corrections.
Comment: Commenters (9372, 11183, 11190, 11404, 11479, 11496, 12718, 12715, 12720) express concern that the supplemental proposal may not be consistent with HIPAA and/or IRB requirements. Commenter (11404) states that HIPAA protections must be honored. Commenter (11183) asserts that the proposed rule violates HIPAA standards.
Commenter (9372) states that researchers cannot make their data publicly available because there are laws against that type of thing to protect the privacy of health and education information such as HIPAA.
Commenter (11190) contends the proposed rule is impractical to implement, because there are good and justifiable regulations protecting private health information, described by HIPAA. The commenter states that human subjects in medical studies participate in these studies with the assurance that their private medical information will never be revealed.
Commenter (11479) states that, while we can assume that the agency believes that a simple anonymization process is enough to fully and completely protect people's confidential data, a simple anonymization process is at odds with the legal and ethical frameworks developed to protect human participants in scientific studies. The commenter states that HIPAA requires that medical information be protected by a set of administrative, physical, and technical safeguards, and a scientist who releases raw anonymized data to the public could be in violation of their IRB approval, which governs a study's informed consent process. The commenter states that the release of underlying human health data may include participants who cannot consent to such a process, like participants who have died. In a rare joint statement on the proposed rule, the editors of six prominent scientific journals wrote that some data "cannot be shared openly; even anonymized personal data can be subject to re-identification, and it has been a longstanding practice for agencies and journals to acknowledge the value of data privacy adjustments."
Commenter (11496) states that the EPA has not described how future studies could comply with the proposed data access requirements while maintaining compliance with other applicable statutes, such as HIPAA, while assuring participants they will continue to prevent disclosure of personal health information to unknown third parties in the future. 
Commenter (12718) states that the EPA's proposed approach in the supplemental proposal presumes that access to data or information must be  -  as an absolute  - accessible to EPA officials, even when that level of access might be inconsistent with IRB requirements, consent statements, or other ethical procedures that should be paramount in government's commitment to its citizens for protecting data and confidentiality. The commenter states that the EPA's supplemental proposal offers no meaningful articulation of privacy and confidentiality safeguards for the American people. The commenter states that the Agency has no established statistical agency or unit designated under the Confidential Information Protection and Statistical Efficiency Act of 2018 (Title 3 of PL 115-435) to apply civil and criminal penalties for violations of protection. The commenter states that this is disappointing in that these protections were unanimously upheld by the Republican and Democratic appointees to the U.S. Commission on Evidence-Based Policymaking as a model for federal agencies, including the EPA, then reauthorized with strong bipartisanship in Congress.
Commenter (12715) notes that the SNPRM does not specify how confidential health information covered by the HIPAA Privacy Rule can be protected outside of HIPAA-covered entities but, again, designing and implementing such protections would be expensive and time consuming.
Commenter (12720) asserts that the proposal's requirement for the public sharing of underlying data of these studies contradicts HIPAA's legal protections for private medical data and requirements researchers adhere to under IRBs, which typically require investigators to ensure participant confidentiality and data security. The commenter states that underlying sensitive health data cannot be released without obtaining individual patient consent, or consent from the next responsible party for study participants who have died.
Response: The final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science. The purpose of the final rule is to codify internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. If dose-response data underlying pivotal science contains PII, the EPA will evaluate whether these data are available in a manner sufficient for independent validation (potentially through restricted access). For pivotal science where there is no access to dose-response data, or access is limited, the Agency may still consider these studies but will give them lesser consideration unless the Administrator grants an exemption under 40 CFR 30.7.
The EPA would like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedure and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. 
Comment: Commenter (11496) states that the EPA has not explained how GLP2 studies will be addressed by this rule, particularly those that are conducted through a process that follows HIPAA rules. The commenter provides that King County is concerned that pivotal regulatory decisions will increasingly be made based on rodent and other guideline studies rather than peer-reviewed science that describes the direct impacts of chemical exposures on people's lives.
Response: The purpose of the final rule is to codify internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. Under the final rule, the EPA is not categorically excluding any studies from consideration with promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. 
As described in the preamble to the final rule, the transparency provisions of the final rule apply to dose-response data underlying pivotal science. Under this rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule under 40 CFR 30.7.
11.2.1.4.2 Deidentification 
General
Comment: Commenter (11900) states that a primary concern about data transparency is that study participants will be re-identified when identifiable information sufficient to contact a person (such as name or address) is matched to "de-identified" data. The commenter states that the supplemental proposal defines PII by reference to the "requirements of the Common Rule, the HIPPA [sic], the 21st Century Cures Act, the Privacy Act, and other relevant laws and regulations, and EPA privacy policies." The commenter states that this definition of PII does not capture all of the data types that can contribute to re-identification risk. 
Response: The final rule does not require the de-identification of PII. The purpose of the final rule is to codify internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. If dose-response data underlying pivotal science contains PII, the EPA will evaluate whether these data are available in a manner sufficient for independent validation (potentially through restricted access). For pivotal science where there is no access to dose-response data, or access is limited, the Agency may still consider these studies but will give them lesser consideration unless the Administrator grants an exemption under 40 CFR 30.7.
The EPA would like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Adverse Consequences of Deidentification
Comment: Commenter (9372) contends that, if employers, colleges or others found out about people's health, it could cause them to get fired or suffer in other ways. Commenter (11576) asserts that the potential consequences of reidentification are severe.
Commenter (11479) re-identifying people from an anonymized database could prove particularly damaging for research on issues like child abuse, sexual assault, and mental illness. And data that include genetic information could reveal sensitive information about the relatives of the research participants, including those not yet been born.
Commenter (11480) states that the release of private health data that could be used to identify individual workers serving as subjects in a study puts those workers at risk of discrimination or retaliation by their employers, insurers, and others. 
Commenter (11576) refers to a study which identifies a number of adverse consequences due to re-identification and expresses concern that these consequences have not been addressed by the EPA. The commenter identifies the following adverse consequences:
      Loss of privacy from re-ID could result in stigma for individuals and communities; affect property values, insurance, employability, and legal obligations; or reveal embarrassing or illegal activity.... It could damage trust in research, harming the study and research more generally. Because ... [environmental health] studies often focus on groups with the highest exposures, privacy risks potentially compound harms faced by the most vulnerable communities. Entities that might be motivated to re-identify ... [environmental-health] data include, for example, employers or insurance companies (who may wish to discriminate against individuals or properties on the basis of environmental exposures) and corporations affected by environmental regulations (who may wish to discredit litigants or studies demonstrating ... [environmental-health] harms, or to discourage participation in ... [environmental-health research). Other parties might leverage the environmental variables to gain access to other parts of the data set, such as
sensitive health information.
Commenter (11920) states that, because public disclosure of sensitive biological data can cause irreparable harm to imperiled species populations, there are established norms for the application and use of such data without full disclosure. The commenter states that applying the proposed rule regarding public disclosure of data would severely limit the ability of the agency to properly address endangered species and wildlife issues in its work, including pesticide labeling and use restrictions under FIFRA, and Section 7 consultations under the Endangered Species Act. The commenter states that access to raw data at the stage described in the supplemental proposal is typically not a requirement even for peer-review, as scientists can evaluate the robustness and accuracy of findings based on the presentation of processed data and description of the methods and analysis used. 
Commenter (11926) states that the uses of sensitive personal data in the wrong hands can range from personal embarrassment through release of identified information on social media, to impacts on an individual's employment and insurance when health data are involved. 
Response: The final rule does not require the de-identification of PII. The purpose of the final rule is to codify internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. If dose-response data underlying pivotal science contains PII, the EPA will evaluate whether these data are available in a manner sufficient for independent validation (potentially through restricted access). For pivotal science where there is no access to dose-response data, or access is limited, the Agency may still consider these studies but will give them lesser consideration unless the Administrator grants an exemption under 40 CFR 30.7.
The EPA would like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Deidentifying and Reidentifying Data
Comment: Several commenters (9372, 11360, 11396, 11186, 11192, 11361, 11395, 11473, 11479, 11576, 11900, 11921, 12405, 12464, 12546, 12715, 12720, 12727) express concern that the deidentification of PII required by the proposed rule is infeasible due to issues of cost, the type of data essential to health studies, identification technology, and/or the risk of identification. Other commenters (10042, 11381, 11899) contend the anonymization process maintains the privacy of the individual.
Response: The final rule does not require the de-identification of PII. The purpose of the final rule is to codify internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. If dose-response data underlying pivotal science contains PII, the EPA will evaluate whether these data are available in a manner sufficient for independent validation (potentially through restricted access). For pivotal science where there is no access to dose-response data, or access is limited, the Agency may still consider these studies but will give them lesser consideration unless the Administrator grants an exemption under 40 CFR 30.7.
The EPA would like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenters (11192, 11361) state that the EPA fails to acknowledge the substantial costs of data de-identification. Commenter (11192) states that thorough data de-identification is not trivial, and if it was not included in an original project scope and budget, it is highly unlikely any academic researcher would have the capacity to undertake the de-identification process. The commenter states that, for existing studies, it is likely to be financially impracticable to de-identify data to make any sort of public access possible, thus excluding them or devaluing them in Agency consideration, except through the sole discretion of the Administrator.
Response: The EPA disagrees that this rule would impose costs on third-party researchers. This is a rule of internal procedure that does not impose requirements on any party other than the EPA. This rule imposes no costs on researchers or their institutions, and the EPA will consider and evaluate all relevant and appropriate science in its significant regulatory actions and influential scientific information. If dose-response data underlying pivotal science contains PII or CBI, the EPA will evaluate whether these data are available through restricted access in a manner sufficient for independent validation. For pivotal science where there is no access to dose-response data, or access is limited, the Agency may still consider these studies but will give them lesser consideration unless the Administrator grants an exemption under 40 CFR 30.7.
Comment: Commenter (11921) asserts that it is not possible to conduct a high-quality air pollution and health study without knowing the locations of those being studied, i.e. where they live, the sources and levels of their air pollution exposure, as well as their specific covariates, and specific health outcomes. The commenter states that, should such information be made available at smaller spatial scale, it is possible to disclose extensive personal and medical information for individual study subjects, raising privacy and ethical concerns. The commenter states that taken together, these characteristics  -  which have in general enhanced the quality and the sensitivity of the studies  -  increase the difficulty of providing a fully "de-identified" data set while also enabling a different investigator to conduct a full replication and sensitivity analysis of the original study results. The commenter states that analyses using more limited data sets may well therefore result in incomplete and misleading results. The commenter states that, since the goal should be to find ways to share data that enables full replication and sensitivity analysis of original studies, it is valuable to consider several aspects of large population air pollution studies that have moved them towards utilizing data at smaller spatial scales:
      First, in response to valid criticisms that earlier air pollution studies relied only on central air quality monitoring data to estimate exposure, investigators have increasingly sought to improve exposure estimates by employing land use regression and other methods that can account for the distance of a subject's home from roadways, industrial facilities, and other sources of air pollution. They have also applied increasingly finer-grained community-level covariates (e.g. at the zip code level). While in the largest locations the application of these finer-grained data may not allow for identification of individuals, the national analyses in some of these studies include subjects from a range of community sizes, including smaller communities where identification could be possible.
      Second, as these types of studies have been reviewed intensively by the HEI Review Committee, the Committee has identified two potentially significant sources of uncertainty in their results: the so-called "ecological confounding" and "spatial
autocorrelation." To address both of these issues, one of the first steps that investigators have taken has been to use data at smaller scales which, while enhancing their ability to address these two sources of uncertainties, also poses the potential in smaller communities for individuals and their personal information to be identified.
Response: The final rule does not require the release of PII or sensitive information. The purpose of the final rule is to codify internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. If dose-response data underlying pivotal science contains PII, the EPA will evaluate whether these data are available in a manner sufficient for independent validation (potentially through restricted access). For pivotal science where there is no access to dose-response data, or access is limited, the Agency may still consider these studies but will give them lesser consideration unless the Administrator grants an exemption under 40 CFR 30.7.
The EPA would like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenters (9372, 10040, 11360, 11396, 11186, 11395, 11473, 11479, 11576, 11900, 12405, 12464, 12715, 12720, 12727) assert that, although "publicly available data" would be anonymized, it is now possible to re-identify people in these datasets. Commenter (9372) states that with new machine learning techniques it is easier and easier to identify people in data even when the data is anonymized.
Commenter (11186) states that newer and more advanced technological methods are being developed to re-identify participants from small and large studies which were once thought "fully de-identifiable". Commenter (11473) states that even anonymized personal data can be subject to reidentification. Commenter (11479) contends that a simple anonymization process is inadequate to protect personal data. Commenter (12727) states that advances in computing have made it possible to identify participants in studies and details about their personal and medical histories, even where researchers have taken steps to mask that information or otherwise "de-identify" the data before disclosing it. As described below, these commenters cite various studies that have documented that de-identification techniques to render data anonymous is not "simple," despite what the Proposal suggests, and can lead to the publication of protected confidential or private data. 
Commenters (11360, 11396) state that stripping out personal identifiers does not solve the problem as the identity of individuals often can be inferred by using data sets from multiple sources. 
Commenters (11576, 11479, 11900, 12464) refer to the Boronow et al. study that discusses how even limited information made available from multiple studies involving the same dataset allowed researchers to successfully re-identify study participants. Commenter (11576) states that researchers in the study were able to re-identify study participants despite the exclusion and redaction of information that could not be shared under the safe-harbor provision of the HIPAA. Commenter (12464) states that the study demonstrates that it is possible to successfully re-identify environmental health study participants using information that is publicly or commercially available such as voter lists, tax and real estate information, or information that is available from data brokers.
Commenter (11900) states that the Boronow et al. study shows that linkage re-identification can occur when de-identified data is matched to externally available, identifiable data using data fields that overlap between both datasets. The commenter states that examples of external public or commercial datasets that can be used in linkage re-identification include tax assessor data, registries of licensed professionals, hospital discharge records, or advertising lists. The commenter states that because environmental health studies often link protected health information and non-protected fields, seemingly non-sensitive data can compromise the security of the entire dataset and lead to breaches of sensitive personal information. The commenter states that the study also demonstrated how chemical exposure measurements, a distinguishing data type of environmental health studies, could pose an unanticipated privacy risk. The commenter states that this analysis demonstrated that chemical data could be used to infer a characteristic of people in a study, even if that characteristic is excluded when the study data are shared. 
Commenters (11395, 12715) assert that the issue of protecting confidentiality of data has been well recognized. Commenters refer t NAS report which states that, "[i]n an experiment to discover whether confidentiality could be preserved while opening the data for public review, the study investigators attempted to disguise the identity of the study participants. They deleted as many features as possible from the questionnaires, such as the name, the state file number, the mother's maiden name, and the name of the person providing the information. However, the investigators needed to retain a minimum set of features if other scientists were to be able to replicate the basic findings of the study... and found that even this minimum set of features could allow for identification of research participants." Commenter (12715) contends this demonstrates that, even after going through an arduous, time consuming process, the removal of personal identifying information would be difficult to achieve.
Commenters (11473, 11479, 12727) refer to the Rocher et al., study in which 99.98% of Americans could be correctly re-identified using a combination of just 15 demographic attributes. The commenters assert that the study demonstrates that anonymized personal data can be subject to reidentification. Commenter (12727) notes that the authors state, their results "reject the claims that, first, re-identification is not a practical risk and, second, sampling or releasing partial datasets provide plausible deniability." 
Commenters (12464, 12727) refer to the Science Advisory Board (SAB) report on the risk of re-identification and state that "[a]lthough the Proposed Rule suggests that privacy and confidentiality issues can be addressed through anonymization or de-identification, this is not always the case;" anonymization is not always effective because "even de-identified datasets present significant risks of reidentification given modern techniques for combining these datasets with other sources of individual information (also known as a `mosaic effect')." Commenter (12464) asserts that, in general, the supplemental proposal expresses too much confidence that de-identification can adequately address privacy and confidentiality concerns.
Commenters (11186, 11395, 11900) refer to the 2017 Sweeney et al., study which tested the ability to re-identify participants from a small de-identified environmental health study on air pollution. The commenters state that in the study, using data considered by the HIPAA to be sufficiently de-identified to be made public and which involved far fewer variables than would be required for the reanalysis proposed by EPA, the researchers were able to correctly identify over 25% of the study participants. 
Commenter (11473) refers to three studies and states that even anonymized personal data can be subject to reidentification, and it has been a longstanding practice for agencies and journals to acknowledge the value of data privacy adjustments. 
Commenter (11479) refers to four studies and states that a simple anonymization process is inadequate to protect personal data. The commenter states that simple steps like an Internet search or database query search can re-identify research participants. The commenter states that research using 1990 census data found that around 87 percent of Americans can be identified with just three data points: their zip code, gender, and date of birth. The commenter states that one paper found that with just 15 characteristics (like age, gender, and marital status), a machine learning program could reidentify 99.98 percent of Americans from an anonymized database. 
Commenter (12405) states that de-identified does not mean unidentifiable. The commenter states that there are many studies where it is impossible to de-identify data to a level where both the data is usable and participants are properly protected. The commenter states that environmental exposure data often must be specific to a particular house, street or neighborhood. The commenter states that, for example, a 2009 study showed that consuming water from a private well located in an area with historical pesticide use is associated with an increased risk of Parkinson's disease. The commenter states that due to the nature of wells  --  typically serving a relatively limited number of people within a very small radius  --  the detail needed to perform the study renders proper deidentification impossible as all one needs to know is that a certain person lives near a particular well coupled with a demographic detail such as their age, gender, race, etc., and that person's privacy is at great risk.
Commenter (12720) states that one study explained that "[b]y linking demographics to public records such as voter lists, and mining for names hidden in attached documents, we correctly identified 84 to 97 percent of the profiles for which we provided names." The commenter states that another explained that "87% (216 of 248 million) of the population in the United States had reported characteristics that likely made them unique based only on [5-digit ZIP, gender, date of birth]." The commenter states that another explains that "any data that is even minutely useful can never be perfectly anonymous." 
Response: The final rule does not require the release or de-identification of PII or sensitive information. The purpose of the final rule is to codify internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. If dose-response data underlying pivotal science contains PII, the EPA will evaluate whether these data are available in a manner sufficient for independent validation (potentially through restricted access). For pivotal science where there is no access to dose-response data, or access is limited, the Agency may still consider these studies but will give them lesser consideration unless the Administrator grants an exemption under 40 CFR 30.7.
The EPA would like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenters (11360, 11396, 11480, 11576, 11900, 12464, 12546, 12720, 12727) contend that the proposal overlooks the risks to privacy. Commenters (11360, 11396) state that the risks to privacy are well recognized in the research and public health professions.
Commenters (11576, 12720) conclude that the EPA's proposal poses significant threats to personal privacy and express concern that these are threats the EPA has yet to acknowledge and address. Commenter (12464) asserts that the supplemental proposal says virtually nothing about the risk of re-identification from supposedly anonymized data sets and that, in general, the supplemental proposal expresses too much confidence that de-identification can adequately address privacy and confidentiality concerns. 
Commenter (11480) states that the Department of Health and Human Services guidance on De-identification Protected Health Information in Accordance with the HIPAA Privacy Rule acknowledges that, even when properly done, de-identification of data does not eliminate the risk of identification of an individual. 
Commenter (12710) states that the absence of a classification system for statistical and administrative data based on sensitivity of the data limits the government's ability to describe access restrictions and data security and privacy protocols appropriate for each level of sensitivity, in a way similar to the well-known classification and handling schedules for national security information.
Commenter (12546) asserts that it is implausible for these records to be publicized without risk of re-identification of individuals even if the most current techniques of de-identification are applied; ensuring that even the most pivotal and groundbreaking studies underpinning our current regulations will be downgraded or eliminated.
Commenter (11900) states that this level of privacy risk is not ethically acceptable. The commenter states that, although certain data types pose known threats to privacy, privacy is a moving target: as new data types are collected and new analytical methods are developed, novel privacy risks emerge that cannot always be anticipated. The commenter states that mobility traces (high-resolution location information) are one example of an emerging data type in environmental health studies. The commenter states that these data are increasingly being used to infer co-located environmental data as technological and cost barriers have decreased. These data are known to be extremely vulnerable to re-ID. Additionally, the commenter states that their research shows that it is not feasible to distinguish between data types that pose re-identification risk and those that do not. The commenter urges the EPA to treat all data associated with human study participants and their homes as potentially containing PII.
Response: The final rule does not require the de-identification of PII. The purpose of the final rule is to codify internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. If dose-response data underlying pivotal science contains PII, the EPA will evaluate whether these data are available in a manner sufficient for independent validation (potentially through restricted access). For pivotal science where there is no access to dose-response data, or access is limited, the Agency may still consider these studies but will give them lesser consideration unless the Administrator grants an exemption under 40 CFR 30.7.
The EPA would like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Other commenters (10042, 11381, 11899) contend the anonymization process maintains the privacy of the individual.
Commenter (10042) states that claims that transparency is not possible because of potential or actual violations of privacy of human health information are not true; the claims are readily falsifiable. The commenter states that the privacy argument against transparency is specious because human clinical data are made public without any violations of individual privacy through a widely accepted process called "anonymization". The commenter states that the anonymization process maintains the privacy of the individual so that the information can be made public and shared various entities for analyses. The commenter states that the practice of anonymization is used by pharmaceutical companies and contractors such as Privacy Analytics, Novartis, and Otsuka where clinical data that are made public are kept private. The commenter states that some regulatory agencies, such as European Medicines Agency require submitted clinical data to support drug registrations to be anonymized. The commenter states that avoiding transparency because of potential violations of privacy rights regarding health information and data of individuals is a clear misrepresentation. 
Commenter (11381) states that the EPA should adopt by rule performance standards used elsewhere in the government and private sector to protect the privacy of PII. The commenter states that this information already must be disclosed to comply with the IQGs, so there is no marginal burden involved. The commenter states that this will require tradeoffs in some cases, such as circumstances in which data cannot be fully protected from deidentification, but the protection of privacy, a worthy goal, comes at the opportunity cost of forgone reproducibility and validation, and hence, public legitimacy. The commenter states that this is not a new problem, nor is it unique to EPA.
Commenter (11899) states that, while some have claimed the transparency rule would force researchers to reveal personal or confidential information about participants in health studies, in reality, such information is not needed and can easily be redacted; only the health data, and location by city and state, is required for these studies.
Response: The final rule does not require the de-identification of PII. The purpose of the final rule is to codify internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. If dose-response data underlying pivotal science contains PII, the EPA will evaluate whether these data are available in a manner sufficient for independent validation (potentially through restricted access). For pivotal science where there is no access to dose-response data, or access is limited, the Agency may still consider these studies but will give them lesser consideration unless the Administrator grants an exemption under 40 CFR 30.7.
The EPA would like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenter (12692) does not agree with critics who warn that, due to potential hacking, confidential data may not be protected under the proposed rule. The commenter states that, if hacking is the problem, that applies to original investigators no less than to those who audit published studies. 
Response: The EPA would like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
11.2.1.4.3 Participation in Studies
Comment: Commenters (11184, 11377, 11395, 11404, 11479, 11496, 11890, 11900, 12458, 12474, 12540, 12716, 12720, 12727, 12731, 13549) express concern that the proposal will deter individuals from participation in research studies due to the risk of publication of personal health data.
Commenters (11184, 11377, 12435, 12457) express concern that the proposal disadvantages study subjects whose participation and data would be prevented from informing public policy. Commenter (12457) states that the lack of participation by low-income communities and communities of color may impact the long-term accessibility of beneficial treatments or interventions because researchers are unable to understand how these treatments impact low-income communities and communities of color.
Commenter (11377) states that effectively mandating fully-public data leaves potential human subjects of research with an uncomfortable choice: take part in studies that may compromise their privacy and confidentiality, or lose the opportunity to participate in science for the betterment of human health altogether. The commenter states that undermining potential subjects' confidence that their information will be protected skews the already unfavorable ratio of risk to benefit of participation in research. The commenter states that this will likely reduce the population of people willing to participate in research, which will severely hamper the conduct of science.
Commenters (12474, 13549) contend that the proposal will reduce research and data collection needed to protect the health of Tribal communities as individuals are deterred from participation based on the fear their personal information will be released and researchers avoid seeking such information.
Commenter (12540) provides that the SNPRM does not recognize the rigorous process for conducting environmental health research. The commenter notes that when environmental health studies are proposed and conducted, there is close attention paid to privacy of personal information and protected health data. The commenter asserts that without protections for the subjects in these studies, EPA is well aware that there would be little, if any participation, and academics would not be allowed to conduct these high quality research studies because of the stringency of the IRB Protections for Human Subjects.
Commenter (12720) contends that the Supplemental Proposal does not consider the negative effects it would have on recruitment for future epidemiological studies if members of the public had to permit access to sensitive personal and health information as a condition for study participation. The commenter provides that many of the peer-reviewed studies EPA uses to set and revise the national ambient air quality standards (NAAQS) through the CAA analyze the relationship between exposure to polluted air over many years and a range of adverse health effects. The commenter notes that these comprehensive studies have enrolled thousands of American volunteers over periods ranging from several years to decades, in order to understand exactly how pollution harms us. According to the commenter, the Supplemental Proposal would have a chilling effect on the study recruitment process because of the onerous data release requirements. The commenter states that EPA's actual creation of these harmful consequences, and failure to consider and account for these harmful consequences, render the Proposal arbitrary and capricious and an abuse of Agency discretion.
Response: This rule does not require any disclosure of PII. The EPA would also like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenter (12692) does not agree with critics who warn that the rule could deter people from participating in important studies out of fear that they could be identified. The commenter states that, just like epidemiologists, there is no reason why auditors of such studies would not be similarly bound and liable for breaches of confidentiality.
Response: The EPA thanks the commenter. 
11.2.1.5 Intellectual Property
Comment: Commenter (12430) expresses concern that the proposed rule does not address intellectual property rights. The commenter states that, for scientists at academic and research institutions, intellectual property protections for innovative analytical tools, models, and computer code are important to the research process. The commenter states that 40 CFR 30.5(c) of the proposed rule, however, would require the publication of the details of such original models and code. The commenter states that the proposal only allows for restricted access to the information if it contains "CBI, proprietary information or personally identifiable information." The commenter states that the intellectual property rights of academic or governmental researchers do not clearly fall into any of these categories. 
Response: The purpose of the final rule is to codify internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. The EPA would like to emphasize that the final rule does not impose requirements on any entity outside of the EPA and does not direct or require any outside entity or the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenter (12430) states that industry research might still be protected as CBI, while academic or public interest research would not be entitled to rely on intellectual property protections. The commenter states that the absence of publication exceptions or exemptions for protected intellectual property makes it far more likely that industry research, rather than academic or public interest research, would form the basis of regulatory decision-making. The commenter states that this could build an industry preference into EPA's regulatory process and disseminated information.
Response: The EPA does not agree that the approach included in the final rule will lead to systematic bias towards certain types of stakeholder goals because the EPA is not categorically excluding any studies from consideration with promulgating significant regulatory actions or developing influential scientific information. 
 Chapter 12: Administrator's Exemptions
The 2018 notice of proposed rulemaking (NPRM) included a provision at 40 CFR 30.9 allowing the Administrator to grant exemptions from the rule on a case-by-case basis if he or she determines that compliance is impracticable because it is not feasible to conduct independent peer review on all pivotal regulatory science because it is not feasible to ensure that dose response data and models underlying pivotal regulatory science are publicly available in a fashion that is consistent with law, protects privacy and confidentiality, and is sensitive to national and homeland security. 
In the supplemental notice of proposed rulemaking (SNPRM) the EPA proposed deleting the proposed exemption because EPA does not believe that independent peer review of pivotal regulatory science or pivotal science would be infeasible. The EPA specifically requested comment on whether these exemptions are appropriate, and on whether there are other situations in which specific significant regulatory actions, or specific categories of significant regulatory actions should be exempted.
The SNPRM also proposed to modify the scope of the data and models that can be considered when determining whether to grant an exemption. The EPA proposed to use the age of data and models as a factor in the determination that compliance with the rule is impracticable. The EPA also proposed in the supplemental that a study or studies would be eligible for consideration whether they contain confidential business information (CBI), proprietary information, or personally identifiable information (PII), if the underlying data or models were collected, completed or updated before the effective date of this rule. 
The supplemental notice solicited comment on the modified text of 40 Code of Federal Regulations (CFR) 30.9, the applicability of the rule to all data and models, retrospective application of the rule, and whether there are factors other than the age of data and models that should be considered when granting exemptions. 
Several comments were received regarding Administrator exemptions. In this section the public comments are organized according to the following topics under sections 12.1 (Exemptions (40 CFR 30.9)(NPRM Comments)) and 12.2 Administrator Exemptions (SNPRM Comments): 
Section 12.1: Exemptions (40 CFR 30.9)(NPRM Comments)
       Legality of Proposed Exemption Provision in 40 CFR 30.9 of the Rule 
       Case-by-Case Administrator Exemption Authority 
       Criteria for Granting Case-by-Case Exemptions
       Requests for Specific Rule Exemptions
       Define "Reasonable Effort"
       Exemption Process Costs
       Notice-and-Comment Principles Should Inform EPA's Implementation of the Exemption Process
Section 12.2: Administrator Exemptions (SNPRM Comments)
       Granting of Exemptions is Subject to Bias
       Rule Would Preclude Consideration of Best Available Science
       Areas Needing Additional Clarity/Specificity
       Transparency
       Retroactivity
       Support for Exemptions
       Other Comments
The specific comments provided in this Chapter reflect the comment letters that provide arguments supported by detailed rationale, examples, and references/documentation. The Docket ID numbers for these commenters are cited throughout this Chapter. In addition, some general comment letters may also include comments related to comments included and responded to in this Chapter. These comments are summarized in Appendix B. Note that points and issues raised in these general comment letters may also be raised by the more detailed comments summarized in this Chapter. The responses provided in this Chapter adequately address these more general comments.
12.1 Exemptions (40 CFR 30.9) (NPRM Comments)
12.1.1 Legality of Proposed Exemption Provision in 40 CFR 30.9 of the Rule
Comment: Commenters (6125, 9227) contend that it is well established that existence of a waiver or exemption mechanism cannot be used to justify a provision otherwise beyond an agency's legal authority. Dimension Financial Corp. v. Board of Governors of Federal Reserve System, 744 F.2d 1402, 1410 (10th Cir. 1984) ("The possible exception to the initial impact of Regulation Y (Part 225.21(B)(4)) contains requirements with no objective standard and thus unbounded agency discretion. A device to meet objections to the new regulation cannot cure the exercise of powers denied by Congress or not provided for by Congress. Public Utilities Comm. of Calif. v. United States, 355 U.S. 534 (1958); In re Surface Mining Regulation Litigation, 627 F.2d 1346 (D.C. Cir. 1980). Commenter (9227) adds ALLTEL Corp. v. FCC, 838 F.2d 551, 561 (D.C. Cir. 1988) ("The [Federal Communications Commission] FCC cannot save an irrational rule by tacking on a waiver procedure. `The very essence of waiver is the assumed validity of the general rule . . . .')(citing WAIT Radio v. FCC, 418 F.2d 1153, 1158 (D.C. Cir. 1969)); United States Telecom Ass'n v. FCC, 359 F.3d 554, 571 (D.C. Cir. 2004) ("Moreover, even if the FCC had adopted some lawful mechanism for making exemptions from its general national rule, it could not necessarily rely on the existence of that mechanism as the sole justification for not adopting a more narrowly tailored rule. . . . [T]he mere existence of a safety valve does not cure an irrational rule.")
Commenter (9277) provides the following points to support their position:
      First, while the statutory provisions described above require EPA to consider best available science and other relevant information when making regulatory decisions, see, e.g., Safe Drinking Water Act, 42 U.S.C Section 300g-1(b)(3)(A)(i) ("The Administrator shall use the best available, peer reviewed science."), the Administrator has discretion over whether to grant an exception. See Proposed § 30.9 ("The Administrator may grant an exemption to this subpart on a case-by-case basis...")(emphasis added).[128] Where a statute requires that the agency consider certain information in reaching a decision, EPA cannot promulgate a rule that gives the Administrator discretion over whether to allow such consideration.
      Second, the only basis on which the Administrator may grant an exemption under Proposed §30.9 is that it "is not feasible" to "ensure that all dose response data and models underlying pivotal regulatory science is publicly available" as the rule requires. However, the Proposal does not explain how "feasibility" is to be determined in this context -- or even whether the term encompasses practical feasibility, cost-effectiveness, or other considerations. Moreover, there can easily be situations where it is theoretically "feasible" to make underlying data publicly available, but this information is nonetheless not publicly available. For example, a scientist who intends to rely on the same data to publish multiple papers may be disinclined to make that data available to competitors. Yet, because it is technically "feasible" to make the underlying data publicly available, the proposed rule would not even provide the Administrator with authority to grant an exemption authorizing such consideration, thus forcing the Administrator to violate the law.
      Third, even if it were lawful for EPA to ignore relevant science, the exemption provision is arbitrary, as it does not define enough criteria or process steps by which the Administrator may decide to exempt a study. The provision instructs the Administrator to rely on a handful of broad (and highly manipulable) policy considerations in determining whether it would be infeasible to make data and methods publicly available. These factors could be applied broadly to give the Administrator nearly absolute discretion. From the face of the Proposal, it is not even clear that the Administrator would be required to provide a public, written explanation of his decision to grant (or deny) a waiver. This lack of accountability could lead to the arbitrary exclusion of studies the Administrator unilaterally chooses to not exempt.
      Finally, the exemption provision is impractical and likely could not be implemented effectively. According to the Congressional Budget Office, EPA "relies on about 50,000 scientific studies annually to perform its mission," and at times, relies on thousands of studies for one action. Many of the studies that would be affected by this rule are complex and include large datasets that would lead to an extensive decision-making process under the exemption provision. EPA does not include any rationale in the proposal justifying how the Administrator could reasonably decide to exempt studies on a case-by-case basis given the tens of thousands of studies EPA considers each year. This provision could create a large backlog, which would result in important studies being effectively removed from EPA consideration because of the need to finalize a regulation before an exemption for every relevant study is granted. Accordingly, the exemption provision fails to safeguard against the unlawful exclusion of valid science from EPA's regulatory process.
Commenter (6117) states that well-established legal precedent frowns upon case-by case determinations where, as here, a general rule would be appropriate.
Commenter (6373) asserts that the ability of the Administrator to apply the Rule arbitrarily is an inherent legal deficiency and renders the exemption process ripe for exploitation.
Response: The EPA is authorized to promulgate this procedural rule under its authority to promulgate housekeeping regulations governing its internal affairs.
EPA notes that the final rule does not categorically exclude studies. The EPA does not agree with the comments regarding the role of the Administrator in determining whether to grant an exemption under 40 CFR 30.7 in the final rule and finds that the Administrator is the appropriate decision maker in this context. To ensure that the Administrator's decision is appropriately transparent and supported, the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. Finally, as is normal Agency practice, the quality of all pivotal science is evaluated prior to its use and thus before the need for a possible exemption under this final rule. The exemption factors listed in Part 30.7 contained additional science-based criteria.
40 CFR 30.7 lists the circumstances under which an exemption may be warranted. Specifically, the final rule states the Administrator may grant an exemption to this part for a study on a case-by-case basis if he or she determines that greater consideration of studies for which the underlying data are not publicly available in a manner sufficient for independent validation is nevertheless warranted because:
       technological or other barriers render sharing of the dose-response data infeasible; 
       the development of the dose-response data was completed or updated before the effective date of the final rule; 
       making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, confidential business information, or national security;
       a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis; or
       the factors used in determining the consideration to afford to the pivotal science indicate full consideration is justified.
When making a decision to grant an exemption, the Administrator may consider input from EPA staff and public commenters. The EPA shall document the rationale for exemptions granted by the Administrator in the significant regulatory action or influential scientific information. The EPA is confident that the above criteria provide sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
12.1.2 Case-by-Case Administrator Exemption Authority
 12.1.2.1 Opposition/Support for Allowing the Administrator Authority in Determining Case-By Case Exemptions
 12.1.2.1.1 Opposition to Allowing the Administrator Exemption Authority
Comment: Commenters (0562, 0567, 0569, 1006, 1170, 1481, 1945, 2003, 2006, 2023, 2336, 2393, 2429, 2787, 4157, 4420, 4588, 4589, 4595, 4831, 4842, 4878, 4884, 4894, 5011, 5181, 5977, 6046, 6099, 6100, 6103, 6111, 6114, 6116, 6117, 6122, 6131, 6133, 6134, 6135, 6137, 6145, 6157, 6158, 6188, 6191, 6361, 6373, 6378, 6379, 6868, 6894, 6899, 6903, 6909, 6912, 6915, 6940, 8272, 8274, 8281-PH26) disagree with allowing the Administrator to be the sole authority in determining case-by-case exemptions. Commenter (6868) is concerned that delegating the discretionary authority solely to the Administrator would grant excessive authority to one person without accountability. Commenter (6158) urges EPA to reevaluate and alter the means by which exemptions would be acquired. Commenters (4842, 4884) assert that there is no basis for politically appointed administrators to choose which science will be considered and which may not be. Commenter (4420) states that there should be a middle ground for when the EPA determines that the study can be used for regulatory decisions when all supporting data are not available. The commenter recommends, however, that the decision to use a study in absence of an independent review of the original data should not be the responsibility of one individual.
Commenter (6134) expresses concern that the proposal puts the EPA in the position of defining "transparent" and allows the Agency to decide if studies should be included in their rule making process. 
Commenters (0567, 0569, 2429, 4894, 6116, 6158, 6378, 6379, 6876, 6899, 6940) assert that allowing the EPA Administrator the ability to exempt/waive (or not waive) provisions of the rule would reduce/weaken transparency, trust and public confidence in EPA rulemaking. Commenter (6351) notes that it is arbitrary decision-making to give authority to the Administrator to determine the fate of research. 
Commenters (0569, 2006, 2336, 4588, 4842, 4878, 4884, 5011, 6100, 6103, 6122, 6152, 6157, 6188, 6158, 6378, 6446, 6894, 6909, 8274) express concern that exemption discretion would be in the hands of a political appointee with potential conflicts of interest and broad discretion to waive the rule's requirements that could result in barring the use of sound science and "best available science".
Commenter (0569) expresses concern that the proposed language at section 30.9 of the rule would allow the EPA to selectively exclude studies to meet its agenda. The commenter states that while 40 CFR 30.9 gives the EPA Administrator discretion to issue exemptions from the policy on a case-by-case basis, this section is vague on how and by what measures determinations would be made, opening the door for drawing upon studies driven by unknown interests or by political considerations rather than, as EPA's Mission Statement notes, "...the best available scientific information."
Commenter (2006) asserts that the EPA's work is politically controversial and allowing the EPA Administrator to make such decisions creates the opportunity for partisan decisions about which studies are and are not allowed to be considered by the EPA.
Commenter (6157) believes that the EPA's proposal inappropriately injects partisanship into the scientific process by granting the EPA Administrator  -  a political appointee  -  the sole ownership of exemptions to this policy. The commenter states that obfuscating the process of setting rules to protect public health and the environment with additional political bias defies the very nature of science, which is to remain objective and follow a consensus of the most accurate data.
Commenter (1170) states that it is not the EPA's job to determine what constitutes valid science--that should be left to scientists working in the field who are in a position to assess the validity of the results and methods--not to administrators. The commenter states that, while the EPA certainly must have some ways of determining which scientific studies deserve the most weight, its responsibility is to consider the aggregate scientific evidence, with guidance from experts in the field, and from that develop policies that address potential threats to the American public identified by that research.
Commenter (4589) states that the proposed Rule would allow the Agency, acting as a risk manager, too much latitude in deciding in advance which studies to examine and which to ignore. The commenter states that it creates in effect a list of potential study disqualifiers in advance of rigorous review and contextualization of the results. The commenter states that the strictures created by the proposed Rule will not be seen as a set of guidelines (such as the Daubert Criteria), rather, these strictures could be used to disqualify reasonable studies on superficial and technical grounds when in reality the value of a study may often be to provide context. The commenter states that there are hazards inherent in confining one's attention to a selected body of work while ignoring other studies that may show anomalies or unexpected findings. 
Commenter (6122) asserts that there is no statutory authority to suggest that this is a permissible approach to leave such exemption decisions in the hands of a political appointee, and no court of law would ever sanction such an arbitrary, lawless, irrational approach. Commenters (2393, 4884, 6903) provide that allowing the EPA Administrator control of what studies/data that would be exempt from the rule is not supported by the Congressional mandates that have guided the EPA and would force the Administrator into a level of direct control which it has not been given by Congress. Commenters (4884, 6903) state that they doubt the Congressional record underlying environmental laws envisions such a role for the EPA Administrator.
Commenter (5181) asserts that it is unclear why an exemption from compliance with the rule would be needed if the final rule is not excessively burdensome; how that discretionary authority will be exercised; and what, if any, standards would be applied by the Administrator in determining exemptions.
Commenter (6915) argues that allowing the Administrator to make ad-hoc exemptions for specific studies or models does not cure the proposed rule's fatal defect of requiring EPA to consider factors other than those specified by Congress. See Alltel Corp. v. Fed. Commc'n Comm'n, 838 F.2d 551, 561 (D.C. Cir. 1988) (holding that an agency "cannot save an irrational rule by tacking on a waiver procedure" because the "essence of waiver is the assumed validity of the general rule"). 
Commenter (6046) notes that one person could arbitrarily and capriciously decide which studies the EPA ultimately uses to set standards, health values, and other EPA values. According to the commenter, it is not appropriate that any one person have such control over EPA's work and the health of the public. The commenter notes that it is assumed that the Administrator would be more involved in management decisions and having the Administrator review detailed scientific issues is probably not the best use of the Administrator's time. Further, according to commenter, the Administrator is unlikely to have working knowledge about the complexities of many of the scientific datasets used by EPA. For example, the commenter notes that epi studies often conduct complex analyses of their data and determining if the results can be replicated with the confidential data removed would be difficult unless the Administrator is an epidemiologist. The commenter asserts that the Administrator is often not a scientist. The commenter contends that it is unreasonable to expect the Administrator to understand scientific datasets sufficiently well as to determine whether the confidentiality of people's health and other highly personal information should be made public.
Commenters (2787, 6046, 6133) assert that allowing the EPA Administrator to grant exemptions on a case-by-case basis would enable the Administrator to interfere in the rulemaking process in an arbitrary and capricious manner and could result in the rule being applied unevenly. 
Commenters (6137, 6191) argue that the proposed section 30.9 provides the EPA Administrator unfettered discretion to be a case-by-case arbiter of scientific information and does not allow for appropriate oversight and is not subject to notice-and-comment, or any stated standards or principles limiting discretion. 
Commenter (6111) states that any decision to consider the study while allowing the data to remain confidential being left to the whim of the EPA Administrator is inconsistent with the Office of Management and Budget's (OMB's) uniform privacy protections.
Commenter (6145) states that allowing the Administrator to determine  -  separately from the requirements of the proposed rule  -  when he/she can consider science that does not meet the parameters of the proposed rule is unacceptable. The commenter believes that the exemption provisions will result in a loophole that allows the Agency to consider what is called "secret science: selectively.
Commenter (8281-PH26) states that the case-by-case exemption provision raises serious risk that this or future administrations could selectively waive the policy to build a distorted scientific record that is designed to reach a desired result.
Commenter (6188) asserts that allowing the Administrator broad discretion in exempting studies from consideration creates the potential for significant inconsistency in future regulatory decision-making even when the scientific community has reached consensus on the underlying issue. 
Commenter (6135) states that picking and choosing which studies to use as the basis of agency actions is antithetical to both sound scientific analysis and evidence-based rulemaking.
Commenter (0562) asserts that a critical deficit of the rule is that it apparently gives the Administrator the sole authority to determine which studies would be exempt, without any explicit language on the effects of the exemptions on present or future use of those studies for regulatory decisions, or for the checks and balances and science-based criteria needed to ensure those decisions support use of the best available science.
Commenter (2003) expresses that it is problematic that the Administrator would have wide latitude under the proposed rule to exempt studies and datasets from the requirement that they be posted online. According to the commenter, the EPA should be making regulatory decisions based on reviews of all available, relevant, credible science done by its scientists. But if the Administrator can hand-pick the studies that are made available to the scientists, he or she can pre-determine the conclusion of their reviews. Vesting that level of influence over the regulatory process in one individual is likely to make that individual a target for lobbying by those with a vested interest in the outcome and does not seem likely to produce the best outcomes for protecting public health and the environment.
Commenter (6122) states that in any individual rule-making EPA might already decide that there is a sound rationale for excluding a specific study from consideration, while including others. In such a circumstance, the commenter states that the agency would have to provide a reasonable basis for that determination. According to the commenter, under this rule, by contrast, EPA seeks to create an inappropriate presumption that certain studies will not be considered and seeks to leave it up to the Administrator to decide when the agency will make an exception.
Commenter (6868) states that the proposed rule provides no guidance or standards for how exemption discretion should be applied and no public, congressional or administrative oversight process for how this authority is executed. Similarly, commenter (6379) contends that the Proposed Rule does not make clear that the proposed "case-by-case" exemption process would be enough to provide certainty that EPA will utilize the best available CBI. The commenter is concerned that such a process could decrease the consistency with which data are treated and increase the discretion of EPA in making such determinations. In contrast to the stated aims of the Proposed Rule. The commenter is concerned that this could decrease transparency of the rulemaking process by obscuring the basis of and process for taking regulatory actions. Commenter (6188) asserts that the resulting "black box" exemption process would provide no guidance to the scientific community on how to conduct studies that meet the Proposed Rule's requirements (or qualify for an exemption) and can thus serve as meaningful tools for EPA under the new regulatory regime.
Response: EPA notes that the final rule does not categorically exclude studies. The EPA does not agree with the comments regarding the role of the Administrator in determining whether to grant an exemption under 40 CFR 30.7 and finds that the Administrator is the appropriate decision maker in this context. To ensure that the Administrator's decision is appropriately transparent, the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. Finally, as is normal Agency practice, the quality of all pivotal science is evaluated prior to its use and thus before the need for a possible exemption under this final rule. The exemption factors listed in Part 30.7 contained additional science-based criteria.
The EPA is including criteria that may be used by the Administrator when considering whether to grant an exemption in 40 CFR 30.7. Specifically, the rule states that greater consideration under this rule may be warranted because:
       technological or other barriers render sharing of the dose-response data infeasible; 
       the development of the dose-response data was completed or updated before the effective date of the final rule; 
       making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, confidential business information, or national security;
       a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis; or
       the factors used in determining the consideration to afford to the pivotal science indicate full consideration is justified.
When making a decision to grant an exemption, the Administrator may consider input from EPA staff and public commenters. The EPA shall document the rationale for exemptions granted by the Administrator in the significant regulatory action or influential scientific information. The EPA is confident that the above criteria provide sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
12.1.2.1.2 CBI and Non-Industry Sponsored Studies Could be Treated Differently
Comment: Commenters (6114, 6116, 6122, 6131, 6361, 6373, 6868, 6897, 6909, 6912, 6915, 6940) assert that allowing the Administrator to selectively exempt industry studies from the restrictions of this proposal could allow industry to be held to a different standard than academics or regulatory agencies.
Commenter (6373) asserts that the proposed exemption provision makes exceptions for industry-funded or industry-favorable findings while rejecting consideration of studies conducted by established scientific institutions in accordance with scientific best practices. The commenter asserts the EPA's failure to act could undermine the scientific foundation of both past and forthcoming regulations, rendering them insufficiently protective of human health. The commenter asserts that, because the rejection of research grounded in scientific data that contains private or confidential information is itself inherently arbitrary and capricious, there are no additional parameters that could limit the discretion of the Administrator in a manner that would render the Rule rational.
Commenter (6868) envisions a scenario where an Administrator grants data exemptions to industry studies while requiring academic studies to meet stringent data requirements. The commenter believes that capricious application of the proposed data policy and its lack of transparency would undermine the legitimacy of the EPA rule making process and the public's trust in agency actions. 
Commenter (6940) reports that proprietary chemicals and products are protected as CBI. The commenter states that, when safety testing is conducted by private companies using Good Laboratory Practice, there is no requirement for independent validation of the results. Thus, according to the commenter, the case-by-case exemption provision allows the Administrator to make independent regulatory decisions based on data that have not been replicated by the scientific community, and without the input of the Science Advisory Board (SAB). In effect, the commenter asserts this is not the "best available science," nor is it transparent, because the strength of scientific results lies in peer reviewed methods and data that withstands independent replication. The commenter explains that the Administrator is given broad discretion to determine which data will be used in regulatory decisions. According to the commenter, the case-by-case exemption provision may allow EPA to consider significant amounts of information submitted by industry to the Agency, even though the information will be protected as CBI and shielded from the public and independent validation. 
Similarly, commenter (6912) states that studies funded by the chemical industry could be exempted from the proposed rule's data publication requirements based on CBI concerns while studies by independent academic researchers of the same chemicals might not be, even if underlying health data raised privacy concerns. According to the commenter, it would depend on the whim, and the biases, of the Administrator. The commenter states that, given the strong objections the chemical industry expressed to a blanket data publication requirement and EPA's apparent eagerness to allay the chemical industry 's concern, it appears that the exemption provision was specifically designed with the chemical industry in mind. The commenter contends that, providing the Administrator the ability to exempt any study from the proposed rule 's requirements addresses industry concerns that the proposed rule 's data transparency requirements would exclude many chemical industry-funded studies. 
Commenter (6909) asserts that the Administrator would be able to apply this rule to block consideration of studies demonstrating human health impacts from pollutants that would trigger new or additional regulations that industry does not want, but then fail to apply this rule to require transparency in studies of things like the composition of fracking fluid or the confidential information pesticide companies provide in registering pesticides with the EPA.
Commenter (6915) asserts that, without any standardized and objective criteria, the commenter believes that the exemption process could, for example, allow biased determinations by the Administrator that provide an exception for CBI in studies submitted by chemical and pesticide manufacturers, while excluding academic toxicology or epidemiology studies. Commenter (4831) states that "[d]ecisions about exemptions should be based on formal agency guidance and not according to criteria established by a single EPA employee." 
Commenter (6955) states that, as proposed, and as many other commenters have stressed, it appears that the rule will incorporate double standards. For instance, the commenter explains that those who rely on proprietary information and CBI (industry and for-profit contractors) will apparently be exempted from complying with the rule, while academic studies will be barred from regulatory consideration, if their papers do not include public access to the data. The commenter provides that, because studies funded by industry will also be vulnerable to post-hoc multiple comparisons, if their data were to be made public, there will be strong pressure to develop rhetorical arguments to justify their receiving exemptions.
Response: EPA notes that the final rule does not categorically exclude studies. The EPA does not agree with the comments regarding the role of the Administrator in determining whether to grant an exemption under 40 CFR 30.7 and finds that the Administrator is the appropriate decision maker in this context. To ensure that the Administrator's decision is appropriately transparent, the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. Finally, as is normal Agency practice, the quality of all pivotal science is evaluated prior to its use and thus before the need for a possible exemption under this final rule. The exemption factors listed in Part 30.7 contained additional science-based criteria.
40 CFR 30.7 lists the circumstances under which an exemption may be warranted. Specifically, the final rule states the Administrator may grant an exemption to this part for a study on a case-by-case basis if he or she determines that greater consideration of studies for which the underlying data are not publicly available in a manner sufficient for independent validation is nevertheless warranted because:
       technological or other barriers render sharing of the dose-response data infeasible; 
       the development of the dose-response data was completed or updated before the effective date of the final rule; 
       making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, confidential business information, or national security;
       a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis; or
       the factors used in determining the consideration to afford to the pivotal science indicate full consideration is justified.
When making a decision to grant an exemption, the Administrator may consider input from EPA staff and public commenters. The EPA shall document the rationale for exemptions granted by the Administrator in the significant regulatory action or influential scientific information. The EPA is confident that the above criteria provide sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7 and will result in equal consideration of all exemption requests regardless of the entity that produced or sponsored the pivotal science.
Comment: Commenter (1012) asserts that there should be no large-scale exemption of certain types of studies (such as CBI), information, or data. According to the commenter, if a case-by-case exemption is sought, it should be made in a transparent fashion with the request, along with the reasoning behind the Administrator's final determination, made publicly available.
Response: EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions.
 To ensure that the Administrator's decision is appropriately transparent, the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information.
The EPA is including criteria that may be used by the Administrator when considering whether to grant an exemption in 40 CFR 30.7. Specifically, the rule states that greater consideration under this rule may be warranted because:
       technological or other barriers render sharing of the dose-response data infeasible; 
       the development of the dose-response data was completed or updated before the effective date of the final rule; 
       making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, confidential business information, or national security;
       a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis; or
       the factors used in determining the consideration to afford to the pivotal science indicate full consideration is justified.
When making a decision to grant an exemption, the Administrator may consider input from EPA staff and public commenters. The EPA shall document the rationale for exemptions granted by the Administrator in the significant regulatory action or influential scientific information. The EPA is confident that the above criteria provide sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
12.1.2.1.3 Support for Allowing the Administrator Exemption Authority
Comment: Commenters (0844, , 6126, 6143, 6151, 6162, 6179, 6356, 6360, 6891, 6906, 6921) express support for allowing the EPA Administrator authority in determining case-by-case exemptions to the rule. 
Commenter (6126) states that the allowance for exemptions under 40 CFR 30.9 of the proposed rule enables flexibility based on individual study conditions, restrictions, and constraints from Institutional Review Boards (IRBs). 
Commenter (6179) provides that the allowance for the EPA Administrator to exempt studies cited in significant rulemakings containing certain information, including that which is private, sensitive, or confidential, from these requirements is reasonable to improve the evidentiary basis for EPA's regulatory policies and improve regulatory outcomes by targeting resources to where they can achieve the largest benefits. The commenter asserts that, rather than rigidly excluding important studies, the Transparency Proposal provides a flexible, case-by-case exemption process. The commenter states the exemption provision in the proposed rule presumes proactive management of private, sensitive, or confidential information and only requires disclosure of dose response data and models in appropriate circumstances. 
Commenter (6891) recognizes and supports the need for exemptions to address lawful and reasonable restrictions on underlying data and models. The commenter also supports EPA's clear intent to ensure that the exemption process does not effectively cancel out the general policy of transparency. The commenter advises EPA to undertake all reasonable efforts to provide access to underlying data and models while protecting legal and policy interests in privacy, confidentiality, and national and homeland security. To the extent reasonably possible, the commenter urges the EPA to explore all methods to allow for data access without impinging on privacy and confidentiality concerns. 
Commenter (0844) states that, while they agree that the Administrator must have flexibility in such cases, they state that this provision not end up being a loophole to avoid transparency and suggest that it therefore should be used judiciously.
Commenters (6143, 6151, 6360, 6376, 6906, 6918) support the exemption provision in the proposed rule but recommend that the EPA provide clear parameters/criteria to properly constrain the Administrator's use (additional comments provided under section 12.1.5 of this Chapter). 
Commenters (6356, 6918) agree that the Administrator should be allowed to make case-by-case exemptions from this rule but believes that the Administrator must document the basis of such a determination. 
Commenter (6918) states that the unavailability of underlying data and models should not prevent the Administrator from citing and relying on a study for a proposed or final rule. The commenter asserts that it is therefore appropriate that the Proposed Rule allows the Administrator to grant exemptions and allow consideration of studies for which underlying data and models cannot be made publicly available. 83 Fed. Reg. at 18774 (proposed 40 CFR 30.9). The commenter suggests that the unavailability of these materials should be acknowledged in the proposed or final rule's preamble. The commenter also suggests that the Administrator take the unavailability of underlying data and/or models into account in deciding what weight to give to a study and that EPA should evaluate relevant techniques for protecting data that still enable some public access, such as data-masking or anonymizing methods before granting an exemption.
Response: To ensure that the Administrator's decision is appropriately transparent, the EPA has included a provision in 40 CFR 30.7 of the final rule that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information.
EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
40 CFR 30.7 lists the circumstances under which an exemption may be warranted. Specifically, the final rule states the Administrator may grant an exemption to this part for a study on a case-by-case basis if he or she determines that greater consideration of studies for which the underlying data are not publicly available in a manner sufficient for independent validation is nevertheless warranted because: 
 technological or other barriers render sharing of the dose-response data infeasible; 
 the development of the dose-response data was completed or updated before the effective date of the final rule; 
 making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, confidential business information, or national security;
 a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis; or
 the factors used in determining the consideration to afford to the pivotal science indicate full consideration is justified.
When making a decision to grant an exemption, the Administrator may consider input from EPA staff and public commenters. The EPA shall document the rationale for exemptions granted by the Administrator in the significant regulatory action or influential scientific information.
Comment: Commenter (6151) reports that industry coalitions and trade associations often aggregate CBI data and present those data in comments to EPA during rulemaking proceedings. The commenter explains that these data often underpin arguments as to whether a proposed regulation should be adopted, or ground truth the economic impact of a proposed regulation. The commenters report that, in these cases, proprietary market or economic data of individual companies are provided to the trade association and then aggregated for submission to EPA with each individual company's legitimate expectation that its own proprietary data will remain protected from disclosure. The commenter states that the proposal notice provides little information on how EPA will treat these data if they are relied upon as "pivotal regulatory science" in a final regulatory decision. The commenter urges EPA to provide clear parameters as to when exemptions would be considered and what would happen in circumstances in which underlying data could not be provided if an exemption were not granted. 
Response: EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
40 CFR 30.7 lists the circumstances under which an exemption may be warranted. Specifically, the final rule states the Administrator may grant an exemption to this part for a study on a case-by-case basis if he or she determines that greater consideration of studies for which the underlying data are not publicly available in a manner sufficient for independent validation is nevertheless warranted because:
       technological or other barriers render sharing of the dose-response data infeasible; 
       the development of the dose-response data was completed or updated before the effective date of the final rule; 
       making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, confidential business information, or national security;
       a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis; or
       the factors used in determining the consideration to afford to the pivotal science indicate full consideration is justified.
It should also noted that in this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. Furthermore, implementation guidelines are forthcoming.
12.1.2.2 Use of Third-Party Service/Scientists/Appropriate Oversight Body/Other
Comment: Commenters (0562, 1006, 1945, 2006, 2426, 4595, 4878, 4881, 4894, 5011, 6158, 6191, 6940) recommend that case-by-case rule exemption determinations be done through a third-party service/scientists/appropriate oversight body.
Commenters (0562, 1006, 1945, 2426, 4894) support a third-party independent review and agreement among qualified, unbiased experts on the science supporting regulatory decisions and evaluations for case-by-case rule exemption determinations. Commenter (1945) believes that exemption decisions should be made by an independent scientific body based on the value and utility of the data relative to the decision to be made, whether deidentifying the data is possible without compromising the data's value or interpretability. Commenter (2006) believes that case-by-case exemption decisions should be made by a non-partisan group of scientists and other specialists who do not have political or financial motivations for allowing studies that come to one conclusion and not allowing studies that come to another conclusion.
Commenters (2426, 4894) recommend giving governing authority for granting exceptions to the rule to an external entity or entities, and that these external entities should also be given oversight of raw data collection, storage, and access. Commenter (2426) contends that, as with many ethical determinations, organizational independence provides greater trust that decisions were reasonable, objective, and unbiased. The commenter states that the governing entity could keep abreast of upcoming regulations and determine whether the pivotal scientific methods, data, or models used in the regulation's assessment document(s) would be subject to this proposed data transparency rule. Over time, the commenter suggests that the entity could develop a list of pivotal scientific articles, methods, data, or models that could then be available to the public on an open access website. The commenter asserts that documenting pivotal works would help ensure consistent application of the rule and guide researchers in further framing and building their own research agendas. The commenter suggests that, to be successful, the governing entity should be given limited authority to enforce its exclusion determinations. The commenter asserts that balance between data privacy and regulatory transparency would best be respected through both this governing body and the prioritization of regulatory actions that fall under this rule.
Commenter (4595) argues that giving the EPA Administrator sole authority to grant exemptions to the rule does not provide for proper checks and balances with appropriate oversight bodies with relevant scientific advisors that should be playing a role in decision-making. Commenter (1011, 5011, 6126, 6940) states that the process for granting which studies warrant exemption from the transparency standard should require formal consultation with EPA career scientists and/or the SAB and other relevant Agency advisory committees. 
Commenters (4878, 6158, 6191) state that, if the proposed rule takes effect, the EPA should require that exceptions decisions be made by subject matter experts, such as a panel of non-partisan, unaffiliated expert scientists. Commenter (6158) states that subject-matter experts who are most familiar with the particulars of the data and models in question should handle decisions on the feasibility and practicability of data sharing and usage. The commenter (6158) points out that, as with other aspects of the proposed rule, it is worth noting that the National Academy of Sciences recommends against broad discretion to non-subject matter officials. 
Commenter (4881) asserts, rather than providing the Administrator with essentially unguided discretion to decide when science is acceptable and when it is not, the decision should be left on whether enough information is provided in publications to allow for independent validation and analysis to the journals publishing the research and to the peer review process controlled by scientists, rather than politicians.
Response: EPA notes that the final rule does not categorically exclude studies. The EPA does not agree with the comments regarding the role of the Administrator in determining whether to grant an exemption under 40 CFR 30.7 and finds that the Administrator is the appropriate decision maker in this context. To ensure that the Administrator's decision is appropriately transparent, the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. Finally, as is normal Agency practice, the quality of all pivotal science is evaluated prior to its use and thus before the need for a possible exemption under this final rule. The exemption factors listed in Part 30.7 contained additional science-based criteria.
When making a decision to grant an exemption, the Administrator may consider input from EPA staff and public commenters. To ensure that the Administrator's decision is appropriately transparent, in the final rule the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. This documentation would typically be provided as part of the proposed rulemaking given this will be part of the decision of what is the pivotal science for the rule. Regardless, the final rulemaking will provide clear documentationThe EPA is confident that the criteria provide sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
Comment: Commenter (6099) suggests that the EPA Administrator receive congressional approval for any exemptions under 40 CFR 30.9 of the rule. 
Response: The EPA does not agree with the comment regarding the role of the Administrator in determining whether to grant an exemption under 40 CFR 30.7 and finds that the Administrator is the appropriate decision maker in this context. The EPA is confident that the criteria in 40 CFR 30.7 provide sufficient clarity and boundaries for the Administrator to consider when granting an exemption under this rule.
12.1.3 Criteria for Granting Case-by-Case Exemptions
 12.1.3.1 The Proposed Rule Does Not Provide Needed Criteria/Rules or Guidance Bounding Administrator Decisions
Comment: Commenters (0562, 1011, 2023, 2171, 2429, 2908, 3135, 4840, 5021, 5165, 6109, 6110, 6117, 6122, 6125, 6137, 6141, 6143, 6163, 6168, 6185, 6188, 6361, 6376, 6894, 6896, 6897, 6912, 6915, 8272) express concern that the proposed rule provisions allow the Administrator to provide case-by-case exemptions in 40 CFR 30.9 if he or she determines that compliance is impracticable without specifying exemption criteria/rules or guidance bounding the Administrator's decisions. Additionally, several of these commenters express that they are also concerned that the Administrator would also not be required to explain or justify why a study was used or not used in a regulatory decision.
Commenters (0562, 1011, 5185, 6112, 6115, 6143, 6855, 6875, 6906, 6918, 6935) assert that the process for granting which studies warrant exemption from the transparency standard must include clear and objective and well-documented a priori guidelines and regulatory criteria that clarifies the extent to which exemptions can be used. 
Commenters (1006, 1011, 2171, 2908, 3135, 4894, 5021, 5165, 5181, 5419, 6109, 6110, 6117, 6122, 6125, 6153, 6168, 6185, 6361, 6373, 6376, 6874, 6897, 6906, 6912, 6915, 8272) assert that, without expanded specific criteria/guidance or the requirement to describe how or why studies are included or excluded from a regulatory decision, the Administrator would be unbounded in application and free to act in an arbitrary manner. Commenter (6896) expresses that, because the proposed rule does not provide specific criteria for when exemptions would be granted, it will be uncertain to researchers and stakeholders about what types of information EPA is willing to consider for exemption. Commenters (0569, 2429, 4588, 5165, 6125, 6373) express that this could have the effect of interjecting the appearance of politics/conflicts of interest into what should be a fair and unbiased scientific assessment and may run counter to the mission of the EPA. Commenter (4894, 6125) assert that highly relevant and reliable studies for which data cannot be shared could be excluded - at the Administrator's whim. Commenter (1011) states that this could lead to distorting effects on the scientific foundations of regulatory actions. Commenter (6110) asserts that not requiring the Administrator to present the reasoning behind his or her decisions would result in additional uncertainty in the regulatory process. 
According to commenters (6153, 6874, 8272), when the regulatory text of the proposal does not provide specific criteria for exempting studies from the rule's requirements and does not state that all studies meeting these criteria will be exempted, it is impossible for all stakeholders to have full confidence that the Administrator will make case-by-case exemptions appropriately.
According to the commenter (6122), rather than providing some coherent guidance for when the Administrator should consider scientific studies, the proposed rule simply provides that he or she "may" grant an exemption to this rule's requirements whenever compliance is deemed "impracticable." To illustrate their belief of how absurd this is, the commenter provides that, if the Administrator chose to admit a study because he or she deemed it "impracticable" for industry-funded studies to provide underlying scientific data, the Administrator could (at the very same time) refuse to admit a study prepared by universities or other public health officials because protected health information is unavailable. The commenter explains that would be allowed because the Administrator still would be acting in a manner consistent with this proposed rule. The commenter states that this simply demonstrates the unreasonable and entirely non-scientific nature of EPA's proposal, and the arbitrary and capricious nature of the discretion EPA would bestow upon the Administrator.
Commenter (6918) contends that, to ensure the validity and transparency of the exemption process, EPA should formulate objective criteria for demonstrating the inability to provide the underlying data and models without violating important data access limitations. According to the commenter, objective, identifiable criteria would not only ensure that the process works as intended over time, but it would also build public trust in the exemption process and reinforce EPA's commitment to transparency.
Response: To ensure that the Administrator's decision is appropriately transparent, the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. 
40 CFR 30.7 lists the circumstances under which an exemption may be warranted. Specifically, the final rule states the Administrator may grant an exemption to this part for a study on a case-by-case basis if he or she determines that greater consideration of studies for which the underlying data are not publicly available in a manner sufficient for independent validation is nevertheless warranted because:
       technological or other barriers render sharing of the dose-response data infeasible; 
       the development of the dose-response data was completed or updated before the effective date of the final rule; 
       making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, confidential business information, or national security;
       a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis; or
       the factors used in determining the consideration to afford to the pivotal science indicate full consideration is justified.
When making a decision to grant an exemption, the Administrator may consider input from EPA staff and public commenters. The EPA shall document the rationale for exemptions granted by the Administrator in the significant regulatory action or influential scientific information. The EPA is confident that the above criteria provide sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
Comment: Commenter (6141) requests that EPA establish clear and objective rules for exempting the underlying data from public disclosure requirements under limited circumstances and requiring EPA to document the reasons for the exemption in the particular rulemaking. As the Agency points out in the preamble to the Proposed Rule, the commenter states that there are a number of legal and policy reasons for limiting public access to the underlying scientific data and modeling information. The commenter notes that these limitations are necessary to protect "privacy and confidential business information and other compelling interests." The commenter supports the continued recognition of these reasonable limitations in combination with EPA efforts to allow as much transparency as possible for the release of that underlying scientific data. The following are some important reference points that the commenter suggests that EPA consider in developing reasonable limited exemptions from scientific transparency requirements contained in the Proposed Rule.
 The final transparency rules should recognize that there are other ways to assess and confirm the validity of prior epidemiological studies without public access to data and analytic methods. One notable example is the re-analysis of the Harvard Six Cities and American Cancer Society (ACS) studies that the Health Effects Institute (HEI) conducted in support of the 1997 PM2.5 standards. The HEI re-analysis was able to replicate the findings of these studies and, in so doing, also assess the robustness of the studies' findings through sensitivity analyses, instead of releasing the data to the public. Furthermore, over time, additional heath-effects studies (some of which have used publicly available data) have confirmed the basic findings of the Harvard and ACS studies.
 The final transparency rules should attempt to accommodate legitimate concerns for confidential or private information and analytic models. Such accommodation is especially appropriate for those studies that are already completed or well underway. In such cases, the participation of human subjects in those studies is being undertaken in accordance with terms defined by agreement and that the release of the data set cannot be released simply by redacting portions of the data.
 EPA should give weight to mechanisms available for assessing the findings of the studies through EPA's expert panels, such as the SAB, the Clean Air Scientific Advisory Committee ("CASAC"), and Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA) Scientific Advisory Panel. For example, CASAC routinely reviews and evaluates the epidemiologic and toxicological studies that are the basis for dose-response relationships used in risk and exposure assessments for those criteria air pollutants for which EPA must set national ambient air quality standards (NAAQS) under the Clean Air Act (CAA). These evaluations involve a rigorous review by CASAC that goes beyond the typical journal peer review procedures even though the CASAC review does not typically involve the reanalysis of original data using the same methods used in the original studies.
The commenter contends that, if reasonable exemptions are not provided in the final rule, overly prescriptive requirements for scientific transparency could have the unfortunate effect of removing legal, ethical, and peer-reviewed studies of health effects as sources to support the Agency's regulatory efforts to adopt important health-based requirements.
Response: EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
To ensure that the Administrator's decision is appropriately transparent, the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. 
40 CFR 30.7 lists the circumstances under which an exemption may be warranted. Specifically, the final rule states the Administrator may grant an exemption to this part for a study on a case-by-case basis if he or she determines that greater consideration of studies for which the underlying data are not publicly available in a manner sufficient for independent validation is nevertheless warranted because:
 technological or other barriers render sharing of the dose-response data infeasible; 
 the development of the dose-response data was completed or updated before the effective date of the final rule; 
 making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, confidential business information, or national security;
 a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis; or
    the factors used in determining the consideration to afford to the pivotal science indicate full consideration is justified.
When making a decision to grant an exemption, the Administrator may consider input from EPA staff and public commenters. The EPA shall document the rationale for exemptions granted by the Administrator in the significant regulatory action or influential scientific information. The EPA is confident that the above criteria provide sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
12.1.3.2 Process and Guidance Governing Exemptions
Comment: Commenter (1011) requests that the exemption process adhere to the standards for scientific objectivity found in Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies. The commenter adds that any process to exempt certain scientific studies from the proposed transparency requirements should err toward inclusion of the best available science in EPA deliberations.
Commenter (5419) requests clarification and additional detail related to the exemption provision of the proposed rule. The commenter also requests clarification and additional information on the process by which organizations or investigators can seek to resolve a situation where high-quality scientific data/studies might be excluded.
Commenter (6109) suggests that the rule needs to answer the following questions with regards to the exemption provision: How will EPA apply this exemption with equal consideration to both industry and health studies? How will EPA protect against bias in these exemptions? Will there be a mechanism for researchers to protest the Administrator's exemption decision? Likewise, will there be a mechanism for the public to protest the Administrator's exemption decision?
Commenters (6875, 9224) suggest that the rule needs to answer the following question with regards to the exemption provision: [d]oes the exemption cover the agency's project (as a whole) or just a single study and/or data set? 
Commenter (6360) recommends that EPA adopt criteria like those that permit emergency rulemaking and direct final rules under the Administrative Procedure Act (APA). 
Commenter (6868) strongly recommends that EPA develop a guidance document regarding how this authority can be applied and seek public input on this document through the public notice and comment process governed by the APA.
Commenter (6180) recommends that the EPA Administrator only be permitted to grant exemptions in cases where he or she has documented the rationale for the exemption. The commenter states that this would ensure that the EPA Administrator grants exemptions only where appropriate and necessary.
Response: EPA notes that the final rule does not categorically exclude studies. 
To ensure that the Administrator's decision is appropriately transparent, the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. 
40 CFR 30.7 lists the circumstances under which an exemption may be warranted. Specifically, the final rule states the Administrator may grant an exemption to this part for a study on a case-by-case basis if he or she determines that greater consideration of studies for which the underlying data are not publicly available in a manner sufficient for independent validation is nevertheless warranted because:
 technological or other barriers render sharing of the dose-response data infeasible; 
 the development of the dose-response data was completed or updated before the effective date of the final rule; 
 making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, confidential business information, or national security;
 a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis; or
    the factors used in determining the consideration to afford to the pivotal science indicate full consideration is justified.
When making a decision to grant an exemption, the Administrator may consider input from EPA staff and public commenters. The EPA shall document the rationale for exemptions granted by the Administrator in the significant regulatory action or influential scientific information. This documentation will occur as necessary when a significant regulatory action is proposed or when an influential scientific information is released for public comment. Discussion during the proposal or draft stage will give the public the opportunity to review and comment. These decisions will also be discussed when a significant regulatory action or influential scientific information are made final.The EPA is confident that the above criteria provide sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
It should also be noted that in this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
Comment: Commenters (0067, 0567, 5170, 6143, 6875, 6356, 9224) express that justification/documentation of decisions regarding exemptions should be made public. 
Commenter (6143) asserts that granting an exemption without enough justification and documentation is simply an invitation to ignore the need for regulatory transparency -- and would defeat the purpose of the proposal. The commenter recommends that the EPA add regulatory text that allows the public to hold the Administrator accountable for any decision to grant discretionary waivers, including through documentation and judicial review.
Commenter (0567) states that any exemption must be justified in a manner that is understandable to the affected community as well as to knowledgeable non-specialists.
Commenter (5170) recommends that, if an exception is requested, the request and the reasoning for the Administrator's determination should be made publicly available and subject to comment. The commenter reports that the EPA has regularly required states to spell out criteria for exemptions in their rule before, not after, public comment closes and believes that this requirement should apply to EPA's proposed rule.
Commenter (6143) recommends that in all but the most extreme cases, the Administrator's decisions should be made public and subject to judicial review. The commenter adds that, in such extreme cases, the relevant documentation should also be made accessible to appropriate Congressional oversight committees. 
Commenter (6356) recommends that, if the Administrator utilizes "impracticability" at §§30.9(a) and (b) or some other justification criteria for not sharing underlying science or other pivotal information such as dose-response curves or models with the public, that he or she should be required to provide written notice to the public in the respective final agency action with regard to the specific reasons and the legal basis for why the information is not being provided to the public, so that the Administrator can be held accountable for his or her decision.
Response: To ensure that the Administrator's decision is appropriately transparent, the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information.
Comment: Commenters (6112, 6115, 6855) recommend that the EPA Administrator be required to certify EPA's adherence to case-by-case exemption requirements when granting exemptions. 
Response: To ensure that the Administrator's decision is appropriately transparent, the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information.
Comment: Commenter (4534) states that if this provision is included in the final version of the regulation, the Administrator should not be the only individual to approve the exemption. The commenter recommends that such an exemption should be agreed upon by the Administrator, a representative from OMB, and a representative from the Judicial Branch. 
Response: When making a decision to grant an exemption, the Administrator may consider input from EPA staff and public commenters. The EPA shall document the rationale for exemptions granted by the Administrator in the significant regulatory action or influential scientific information.
Comment: Commenter (6126) suggests that the rule allow the expansion of the exemption authority. First, the commenter suggests that Associate Administrators or decision-makers in given regulatory actions could be granted authority to determine exceptions, consistent with respective environmental statutes designations of authority and practice for which policy officials sign certain regulations. Secondly, the commenter suggests that a citizen petition mechanism could be instituted through which individuals in the American public, interest groups, and advisory bodies to EPA can request exceptions.
Response: In 40 CFR 30.7 of the final rule, the Administrator remains the individual who grants exemptions under the rule. When making a decision to grant an exemption, the Administrator may consider input from EPA staff and public commenters. The EPA shall document the rationale for exemptions granted by the Administrator in the significant regulatory action or influential scientific information.
Comment: Commenter (6143) asserts that exemptions and exclusions related to the requirements should be made only by the Administrator on a case-by-case basis, following clear, objective, and well-documented a priori guidelines and criteria that would be specifically established by the Agency for that purpose and that would encourage transparency while accommodating legitimate, protected privacy interests such as CBI. The commenter states that the EPA's proposed provision at 40 C.F.R. section 30.9, which would allow the Administrator to grant certain exemptions, can serve this purpose if properly cabined by clear, objective criteria for what circumstances may justify an exemption.
The commenter states that, in all but the most extreme cases, the Administrator's decisions should be made public and subject to judicial review. The commenter notes that, in such extreme cases the relevant documentation should be made accessible to appropriate Congressional oversight committees.
Response: 40 CFR 30.7 lists the circumstances under which an exemption may be warranted. Specifically, the final rule states the Administrator may grant an exemption to this part for a study on a case-by-case basis if he or she determines that greater consideration of studies for which the underlying data are not publicly available in a manner sufficient for independent validation is nevertheless warranted because: 
 technological or other barriers render sharing of the dose-response data infeasible; 
 the development of the dose-response data was completed or updated before the effective date of the final rule; 
 making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, confidential business information, or national security;
 a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis; or
 the factors used in determining the consideration to afford to the pivotal science indicate full consideration is justified.
When making a decision to grant an exemption, the Administrator may consider input from EPA staff and public commenters. The EPA shall document the rationale for exemptions granted by the Administrator in the significant regulatory action or influential scientific information.
12.1.3.3 Requests for Regulatory Changes
Comment: Commenter (6143) suggests that the EPA add the following new paragraph after section 30.9(b) of the rule: 
      The Administrator shall thoroughly document the justification for granting an exemption according to a specific process and criteria to be developed by EPA. The Administrator's decision to grant an exemption shall be published in the Federal Register and shall be subject to judicial review.
Response: 40 CFR 30.7 of the final rule includes, "The EPA shall document the rationale for exemptions granted by the Administrator in the significant regulatory action or influential scientific information."
Comment: Commenter (0067) provides that 40 CFR 30.9 of the rule does not require EPA to inform the public when the Administrator has granted such an exemption. The commenter recommends that EPA add the following language at the end of 40 CFR 30.9: 
      EPA shall give notice in the Federal Register that the Administrator has granted an exemption with respect to a specified case, unless in that case disclosure of the fact of the granting of the exemption is specifically prohibited by statute or constitutes national security information classified in accordance with Executive Order 13526, as amended.
Response: Title 40 CFR 30.7 of the final rule includes, "The EPA shall document the rationale for exemptions granted by the Administrator in the significant regulatory action or influential scientific information."
12.1.4 Requests for Specific Rule Exemptions
 12.1.4.1 Categorical Exemptions
Comment: Commenter (5170) states that the proposed rule allows the EPA Administrator to grant categorical exemptions to the proposed rule, particularly of CBI. The commenter recommends that, if this rule goes forward, given its premise that transparency is paramount, it should apply to all. The commenter states, that if an exception is requested, the request and reasoning for the Administrator's determination should be made publicly available.
Response: To ensure that the Administrator's decisions are appropriately transparent, the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information.
12.1.4.2 All Regulatory Actions
Comment: Commenter (0671) states that all regulatory actions should be exempted from the proposed rule. The commenter argues that the rule would prevent the EPA from using the best available science. The commenter further argues that the exemption provisions are not workable because it requires researchers to intuit what an Administrator will decide is acceptable and allows the Administrator to determine what qualifies as science (which is outside of the Administrator's purview).
Response: The EPA does not agree with the comment. The EPA is committed to its mission of protecting human health and the environment through sound policy decisions that are informed by robust scientific and technical research. Because of the potential impact of the EPA's significant regulatory actions and influential scientific information on American lives and livelihoods, the American people deserve environmental decisions and policies that are based on the best scientific information. Only through continuous improvement to its procedures, especially those focused on transparency, can the EPA fully demonstrate that commitment. 
The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information. 

To ensure that the Administrator's decision is appropriately transparent, the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. 
40 CFR 30.7 lists the circumstances under which an exemption may be warranted. Specifically, the final rule states the Administrator may grant an exemption to this part for a study on a case-by-case basis if he or she determines that greater consideration of studies for which the underlying data are not publicly available in a manner sufficient for independent validation is nevertheless warranted because:
       technological or other barriers render sharing of the dose-response data infeasible; 
       the development of the dose-response data was completed or updated before the effective date of the final rule; 
       making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, confidential business information, or national security;
       a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis; or
       the factors used in determining the consideration to afford to the pivotal science indicate full consideration is justified.
When making a decision to grant an exemption, the Administrator may consider input from EPA staff and public commenters. The EPA shall document the rationale for exemptions granted by the Administrator in the significant regulatory action or influential scientific information. The EPA is confident that the above criteria provide sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
12.1.5 Define "Reasonable Effort"
Comment: Commenter (4831) asserts that it is critical for the EPA to define what "reasonable effort" would be required to make data publicly available before an exemption is granted. 
Response: The final rule does not clarify "reasonable effort" any further than in the 2018 notice of proposed rulemaking. In 40 CFR 30.5 it states, "The Agency shall make all reasonable efforts to explore methodologies, technologies, and institutional arrangements for making such data and models available before it concludes that doing so in a manner consistent with law and protection of privacy, confidentiality, national security is not possible." The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. "[R]easonable effort" will likely be clarified during the implementation of the rule. Furthermore, implementation guidelines are forthcoming.
12.1.6 Exemption Process Costs
Comment: Commenters (4845, 6196, 6378) state that the EPA proposal duly notes obstacles to study replication and provides that exemptions may be granted on a case-by-case basis where "compliance is impracticable." The commenters believe, however, that an exemption process will add to the considerable cost and effort required to implement the proposed rule and may result in disputes and even litigation over whether exemptions are justified.
Response: The Agency may incur administrative costs to perform analyses and evaluations to support activities such as exemption decisions made by the Administrator, and documenting these or other decisions made pursuant to rule requirements. The Agency finds the incremental costs for these activities would be small relative to current administrative costs for developing significant regulatory decisions or influential scientific information. See III.J of the final rule preamble for further discussion of benefits and costs of the rule.
12.1.7 Notice-and-Comment Principles Should Inform EPA's Implementation of the Exemption Process
Comment: Commenter (6918) recommends that EPA evaluate how to implement its exemption process in a way that allows for more transparency, better public participation, and improved validity of final rules. The commenter provides that notice-and-comment principles are an essential cornerstone of this process. The commenter suggests that, for studies on which EPA relies in formulating a proposed rule, EPA should require that it note in the proposed rule preamble if any such studies have been subject to an exemption and why the exemption was granted. For studies that are submitted to EPA during the public comment period and that are subject to an exemption, the commenter recommends that the EPA note in the final rule preamble which precise studies are exempted and why they meet the exemption criteria. The commenter believes that this process will serve transparency and public participation values and streamline the process for EPA and the public. Furthermore, according to the commenter, it is consistent with the strict requirements of section 307(d) of the CAA, which apply to many EPA rulemakings, that the proposal contain all the factual data on which the proposed rule is based, along with relevant methodologies, policy decisions, and legal interpretations underlying the proposed rule.
Commenter (0567) recommends that exemptions to transparency requirements follow the rule-making process.
Response: To ensure that the Administrator's decision is appropriately transparent, the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information.
12.2. Administrator Exemptions (SNPRM Comments)
12.2.1 Granting of Exemptions is Subject to Bias
Comment: Commenters (10339, 10748, 10749, 11186, 11192, 11332, 11391, 11393, 11395, 11407, 11480, 11482, 11495, 11576, 11894, 11895 11490, 12403, 12412, 12430, 12431, 12434, 12435, 12442, 12474, 12546, 12707, 12716, 12720, 14397, 198105) express concern that providing the Administrator with sole discretion to grant exemptions will introduce personal or political bias to the political process. Commenters (11391, 11480, 12727) note that this issue was also raised by the SAB. Commenter (12727) adds that failing to include science-based criteria for exemption suggests that "EPA is willing to exempt a study from the rule's requirements despite the lack of mechanisms to ensure the study's reliability."
Response: The EPA does not agree with the comments regarding the role of the Administrator in determining whether to grant an exemption under 40 CFR 30.7 in the final rule and finds that the Administrator is the appropriate decision maker in this context. To ensure that the Administrator's decision is appropriately transparent and supported, the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. Finally, as is normal Agency practice, the quality of all pivotal science is evaluated prior to its use and thus before the need for a possible exemption under this final rule. The exemption factors listed in Part 30.7 contained additional science-based criteria. In total, the Agency believes that Part 30.7 is not subject to bias.
12.2.1.1 Rule Gives EPA Administrator Too Much Discretion
Comment: Commenters (10048, 10748, 10749, 11186, 11192, 11330, 11339, 11354, 11391, 11393, 11400, 11404, 11407, 11480, 11482, 11484, 11490, 11495, 11890, 11894, 11895, 12412, 12430, 12443, 12474, 12546, 12727, 298105D, 12715, 198105) oppose giving the EPA Administrator sole, unlimited discretion regarding exemptions. Commenters (10748, 10749, 11186, 11339, 11393, 11393, 11399, 11403, 11404, 11407, 11480, 11482, 11484, 11894, 11895 11490, 12412, 12430, 12431, 12434, 12435, 12442, 12474, 12727, 14397, 12715) express concern about placing the authority to grant exemptions in the hands of a political appointee who likely does not possess advanced scientific expertise. Commenter (11482) adds that the National Academy of Sciences (NAS) recommends against broad discretion to non-subject matter officials. Commenters (11498, 11576, 12727) note that giving the Administrator sole discretion makes it difficult to ensure accountability and fails to provide for proper checks and balances with the appropriate scientific oversight bodies.
Commenter (12715) asserts that allowing the Administrator to make ad-hoc exemptions for specific studies or models does not cure the supplemental proposal's fatal defect of requiring EPA to consider factors other than those specified by Congress. The commenter states that rather, because the supplemental proposal contains no standards requiring the exemptions to be based on the relevance, importance, or scientific validity of the study or model at issue, and public availability is not a criterion that is meaningful for scientific validity and relevance, the Administrator's ability to arbitrarily include certain studies at his or her discretion simply compounds the extent to which the supplemental notice of proposed rulemaking (SNPRM) would allow EPA to deviate from the requirements of the statutes it is charged with implementing.
According to the commenter (12715), the SNPRM narrows the grounds for the exemption to cases where compliance is impractical because: (1) technological barriers make sharing the data or models infeasible; (2) development of the data or model was completed before the date of the rule; or (3) making the data and models publicly available is contrary to law. The commenter notes that the SNPRM still offers no definition or standards to guide the Administrator's determination of what is "practicable" or "feasible" or what constitutes a "technological barrier," and the waiver process is still wholly discretionary. The commenter provides that without any standardized and objective criteria, the exemption process could, for example, allow the Administrator -- a political appointee -- to arbitrarily grant exemptions for CBI in studies submitted by chemical and pesticide manufacturers, but fail to give similar exemptions for confidential patient or medical information in academic toxicology or epidemiology studies. The commenter provides that the National Academies also highlighted this concern, noting that "[d]ecisions about exemptions should be based on formal Agency guidance and not according to criteria established by a single EPA employee." Further, the commenter notes that the proposal does not require the Administrator to spell out the criteria applied in granting an exemption, and, given how severely the supplemental proposal would limit the scientific evidence available for EPA's use, the proposed exemption provisions could become the basis upon which most of the science relied on by EPA in its rulemaking is admitted. The commenter contends that the exceptions could thus largely swallow the rule, resulting in greater arbitrariness in EPA regulatory actions rather than EPA's alleged goal of greater transparency.
Response: The EPA does not agree with the comments regarding the role of the Administrator in determining whether to grant an exemption under 40 CFR 30.7 in the final rule and finds that the Administrator is the appropriate decision maker in this context. To ensure that the Administrator's decision is appropriately transparent and supported, in the final rule the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. This documentation would typically be provided as part of the proposed rulemaking given this will be part of the decision of what is the pivotal science for the rule. Regardless, the final rulemaking will provide clear documentation.. Finally, as is normal Agency practice, the quality of all pivotal science is evaluated prior to its use and thus before the need for a possible exemption under this final rule. The exemption factors listed in Part 30.7 contained additional science-based criteria. 
40 CFR 30.7 lists the circumstances under which an exemption may be warranted. Specifically, the final rule states that greater consideration under this rule of studies for which the underlying data are not publicly available in a manner sufficient for independent validation may nevertheless be warranted because:
 technological or other barriers render sharing of the dose-response data infeasible; 
 the development of the dose-response data was completed or updated before the effective date of the final rule; 
 making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, confidential business information, or national security;
 a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis; or
 the factors used in determining the consideration to afford to the pivotal science indicate full consideration is justified.
To assist the Administrator in determining whether to grant an exemption, the EPA program or region responsible for the significant regulatory action or influential scientific information will prepare a memorandum to the Administrator with an evaluation of the criteria. Furthermore, the Administrator may consider public comments in deciding whether to grant an exemption. The EPA is confident that the above criteria provide sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
Comment: Commenters suggest the following alternatives to giving the EPA Administrator sole discretion to decide exemptions:
 Exemptions should be decided in consultation with EPA's independent scientific advisory committees, scientific advisors, or independent experts or scientific career staff (10748, 11186, 11399, 11482, 11484, 11490, 11890, 12431, 12435).
 Exemptions should be decided at the level of an Office Assistant Administrator or Deputy Assistant Administrator in consultation with Office staff scientists as well as the Office of Research and Development (ORD) (10038).
Response: The EPA finds that the Administrator is the appropriate decision maker in this context. To assist the Administrator in determining whether to grant an exemption, the EPA program or Region responsible for the significant regulatory action or influential scientific information and public commenters can provide input when the Administrator is considering an exemption. To ensure that the Administrator's decision is appropriately transparent, in the final rule the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. This documentation would typically be provided as part of the proposed rulemaking given this will be part of the decision of what is the pivotal science for the rule. Regardless, the final rulemaking will provide clear documentation. The EPA is confident that the above criteria provide sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
12.2.1.2 Lack of Clear and Objective Criteria/Standards
Comment: Commenters (10339, 11182, 11192, 11354, 11358, 11391, 11397, 11399, 11400, 11490, 11892, 11911, 11915, 12399, 12406, 12422, 12430, 12435, 12546, 12702, 12715, 12720, 12727, 198105) express concern that there are no clearly defined standards, criteria, or processes that will govern the granting of exemptions and suggest that EPA rectify this if it is to promulgate this rule. Commenters (11358, 11391, 12399) note that this concern was also raised by the SAB.
Response: 40 CFR 30.7 lists the circumstances under which an exemption may be warranted. Specifically, the final rule states that greater consideration under this rule of studies for which the underlying data are not publicly available in a manner sufficient for independent validation may nevertheless be warranted because:
 technological or other barriers render sharing of the dose-response data infeasible; 
 the development of the dose-response data was completed or updated before the effective date of the final rule; 
 making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, confidential business information, or national security;
 a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis; or
 the factors used in determining the consideration to afford to the pivotal science indicate full consideration is justified.
12.2.1.2.1 Proposed Criteria/Standards
Comment: Commenters suggest the following changes regarding the process and criteria used when granting exemptions:
       A nondiscretionary, mandatory exemption for all studies which have data that cannot legally, ethically, or efficiently be publicly released (11393, 11407, 12412, 12474, 14397).
       A mandatory exemption for all studies begun prior to the date the proposal is finalized (11393, 11407, 11490, 12412, 12474, 14397).
       An overall exemption for individual level data (10972).
       Using the standards originally listed in the version of the supplemental notice that EPA submitted to Office of Information and Regulatory Affairs (OIRA): "soundness, applicability and utility, clarity and completeness, uncertainty and variability, evaluation and review, and the age of the study" (12430).
       Using standards drawn from the principles of the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) system for evaluating quality of evidence, including "study design, sample size, rigor of analytical methods, and other factors" (11490).
       Reinstating the provision to allow exemptions on a case-by-case basis if compliance is deemed impracticable because it is not feasible to conduct independent peer review on all pivotal regulatory science (11358).
       Exempting "studies and data that have been developed under EPA testing programs, where EPA test guidelines or other validated and accepted test methods such as those developed by Office of Economic Cooperation and Development (OECD), American Society of Testing and Materials (ASTM) have been used, and EPA Good Laboratory Practices have been followed and certified" (12402).
       Tasking each program office within EPA to develop criteria to ensure appropriateness and specificity (12406).
       Explicitly consider Health Insurance Portability and Accountability Act (HIPAA) when granting an exemption for health-related information (11892).
       International studies be considered applicable for the technological barrier clause for exemption from the rule (11892).
Response: EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
This rulemaking applies prospectively to significant regulatory actions and influential scientific information. This rulemaking has no retrospective effect on either final significant regulatory actions or influential scientific information. For future significant regulatory actions and influential scientific information, it applies equally to all dose-response data underlying pivotal science.
As detailed in 40 CFR 30.5(b), "The EPA will use the following factors to assess the quality of studies identified in the systematic review: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. The EPA will rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence of a relationship between exposure and effect, the EPA will identify those studies based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both of an EPA significant regulatory action or influential scientific information will be identified as pivotal science." 
The EPA is including criteria that may be used by the Administrator when considering whether to grant an exemption in 40 CFR 30.7. Specifically, the rule states that greater consideration under this rule may be warranted because: 
       technological or other barriers render sharing of the dose-response data infeasible; 
       the development of the dose-response data was completed or updated before the effective date of the final rule; 
       making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, confidential business information, or national security;
       a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis; or
       the factors used in determining the consideration to afford to the pivotal science indicate full consideration is justified.
40 CFR 30.7 of the final rule does not include the exemption on a case-by-case basis if the Agency determined that it was not feasible to conduct independent peer review on pivotal regulatory science, which was removed in the SNPRM.
The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
The HIPAA is not explicitly considered in the final rule with regard to granting an exemption for health-related information.
International studies have not been given a clause in the final rule for exemption due to technological barriers.
12.2.1.2.2 Removal of Infeasibility Criteria for Exemption
Comment: Commenter (11397) supports the SNPRM's proposed removal of 2018 language that would have allowed EPA to grant an exemption on a case-by-case basis if the Agency determined that it was not feasible to conduct independent peer review on pivotal regulatory science. The commenter agrees with EPA's conclusion in this regard and supports the removal of the case-by-case exemption language proposed in 2018.
Response: EPA appreciates this comment. The final rule retains that change to the SNPRM as 40 CFR 30.7 does not include the exemption on a case-by-case basis if the Agency determined that it was not feasible to conduct independent peer review on pivotal regulatory science.
Comment: Commenter (10752) contends that it is outrageous that EPA has removed the provision from the 2018 NPRM that would allow the Administrator to grant exemptions from peer review due to infeasibility. The commenter states that peer review is a laudatory, but inherently expensive process that is not practical for every inquiry. The commenter provides as an example that researchers in the fields of social and environmental sciences may not even have available peer reviewed journals in their field. The commenter suggests that even in fields where a few peer reviewed journals may exist, the journals may not have the ability to examine much of the most important scientific research.
Commenter (12727) states that EPA's claim that there are no circumstances under which peer review would be infeasible is inconsistent with Section VIII of the OMB's Peer Review Bulletin that provides that an agency may defer or waive peer review requirements based on a "compelling rationale." The commenter expects that it is possible that costs, expediency or some other unforeseen circumstance may necessitate that EPA consider a study that has not been peer reviewed. The commenter states that while proposed 40 CFR 30.7 continues to provide that the rule's peer review requirements must be applied in a manner that is "consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review and the Exemptions described therein," EPA's declaration that there will not be circumstances where peer review would be infeasible, combined with EPA's express decision to eliminate the peer review exemption from proposed section 30.9, could be interpreted as overriding the general waiver and deferral authority provided by Section VIII the OMB Peer Review Bulletin. The commenter asserts that eliminating -- or even casting doubt upon -- the Administrator's flexibility to waive or defer peer review requirements when confronted with unforeseen circumstances and a "compelling rationale" is unwarranted and risky. The commenter requests that if EPA moves forward with finalizing its proposed peer review requirements -- which the commenter believes it should not -- EPA must add language to proposed section 30.9 clarifying that the Administrator possesses authority to defer or waive peer review requirements based on a compelling rationale.
Response: Title 40 CFR 30.6 of the final rule states, "The EPA shall conduct independent peer review consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review and the exemptions described therein. The EPA will evaluate whether or not to conduct additional peer review of individual studies identified as pivotal science if the studies have already undergone peer review." In this rule, the EPA is not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review does not typically include reanalysis of the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information) and, thus, peer review is not considered a replacement for the data availability requirements of this rule. Because transparency in pivotal science includes addressing issues associated with assumptions used in analyzing dose-response data, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied and the implications of those assumptions for the results." The final rule also clarifies the criteria the Administrator would consider in determining whether to grant an exemption. The Agency believes this revised language addresses the commenters concerns about ensuring that the Administrator retains the ability to defer or waive peer review.
12.2.1.2.3 Proposed Process
Comment: Commenter (11399) proposes the following process for granting exemptions:
      The rule should provide clear and detailed guidelines and a formal process the agency must follow before deciding to exclude data of any kind from new or old studies. Such guidelines should include a provision that only the portions of data that would trigger violations of privacy or confidential business secrets may be excluded, and that all other data and methodologies should be made transparent and publicly available. Before excluding data, EPA should conduct and make public a systematic review of the scientific literature to determine if there is an alternative study that could be used in place of any study whose data they cannot fully access. If such studies exist and data can be made accessible, EPA should rely on the alternative study or studies in lieu of those whose data is not available. If an alternative study or studies do not exist, then the Administrator should only use the original study if the researchers demonstrate that they explored all scientific methods and tools available to make the data available, but none proved sufficient to protect individual subjects' privacy and/or confidential business secrets. If after such efforts the Administrator deems the original study essential for agency decision making, the agency should publish its systematic review along with a detailed analysis explaining the specific legal reasons why data must be excluded. The Administrator should further detail why the study authors were unable to provide the data in a format that allows reanalysis and replication without compromising privacy or confidential business secrets. It should also include details on how much of the data fall within the exclusion, since only those specific pieces of information should gain exclusion. Details about study methods, model assumptions, and the like should always be transparent. And before the agency uses the study, this agency justification should be subject to public comment. If after public comment the Administrator still determines that the study is essential and some data cannot be released publicly, then the agency shall set up a system for limited release. The agency would then need to set up a committee of researchers to receive all the data subject to nondisclosure agreement. These researchers should include members with a wide range of perspectives on the issue to allow independent analysis, validation, and peer review. Should the original study authors refuse to provide the data under this closed, limited-access process, the study should be excluded from EPA decision making.
Response: The EPA would like to clarify that no studies are categorically excluded from consideration under the final rule. In addition, the EPA included criteria that may be used by the Administrator when considering whether to grant an exemption in 40 CFR 30.7. Specifically, the rule states that greater consideration under this rule may be warranted because:
       technological or other barriers render sharing of the dose-response data infeasible; 
       the development of the dose-response data was completed or updated before the effective date of the final rule; 
       making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, confidential business information, or national security;
       a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis; or
       the factors used in determining the consideration to afford to the pivotal science indicate full consideration is justified.
To assist the Administrator in determining whether to grant an exemption, the EPA program or Region responsible for the significant regulatory action or influential scientific information and public commenters can provide input when the Administrator for his/her consideration. To ensure that the Administrator's decision is appropriately transparent, in the final rule the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. This documentation would typically be provided as part of the proposed rulemaking given this will be part of the decision of what is the pivotal science for the rule. Regardless, the final rulemaking will provide clear documentation. The EPA is confident that the above criteria provide sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
12.2.1.2.4 Standards Should Be Established in Guidelines, Not Rule
Comment: Commenter (11395) writes that "data sharing and other study assessment methodologies should be addressed in guidelines rather than in a fixed regulation, and criteria for how the above considerations are evaluated should be delineated in those guidelines."
Response: The promulgation of these internal Agency procedures in this rulemaking provides a more durable, enforceable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data and highlights the Administration's commitment to maximizing transparency in the EPA's most impactful regulatory actions and influential scientific information. The rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made by this rule provides an important step towards prioritizing transparency in particularly impactful rules and assessments and will inform future statute-specific rulemakings.
12.2.1.3 Standards May Shift Over Time
Comment: Commenter (12720) points out that because the interpretation of "infeasible" is subjective, standards used to decide exemptions may change over time. Commenter (11892) adds that implementation may also vary as EPA leadership changes, which will reduce consistency and accuracy.
Response: The final rule includes criteria that may be used by the Administrator when considering whether to grant an exemption in 40 CFR 30.7. Specifically, the rule states that greater consideration under this rule may be warranted because:
       technological or other barriers render sharing of the dose-response data infeasible; 
       the development of the dose-response data was completed or updated before the effective date of the final rule; 
       making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, confidential business information, or national security;
       a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis; or
       the factors used in determining the consideration to afford to the pivotal science indicate full consideration is justified.
To assist the Administrator in determining whether to grant an exemption, the EPA program or Region responsible for the significant regulatory action or influential scientific information and public commenters can provide input when the Administrator is considering an exemption. To ensure that the Administrator's decision is appropriately transparent, in the final rule the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. This documentation would typically be provided as part of the proposed rulemaking given this will be part of the decision of what is the pivotal science for the rule. Regardless, the final rulemaking will provide clear documentation. The EPA is confident that the above criteria provide sufficient consistency, clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
12.2.1.4 Politicization of Exemptions
Comment: Commenters (10749, 11182, 11186, 11192, 11339, 11395, 11404, 11480, 11484, 11490, 12434, 12442, 12702, 12720, 12727, 13788) express concern about allowing room for political considerations to inappropriately influence science-based decision-making by leaving exemptions up to the discretion of the Administrator rather than basing them on criteria grounded in science.
Response: The EPA does not agree with the comments regarding the role of the Administrator in determining whether to grant an exemption under 40 CFR 30.7 and finds that the Administrator is the appropriate decision maker in this context. To ensure that the Administrator's decision is appropriately transparent and supported, in the final rule the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. This documentation would typically be provided as part of the proposed rulemaking given this will be part of the decision of what is the pivotal science for the rule. Regardless, the final rulemaking will provide clear documentation.
The final rule also includes criteria that may be used by the Administrator when considering whether to grant an exemption in 40 CFR 30.7. Specifically, the rule states that greater consideration under this rule may be warranted because:
       technological or other barriers render sharing of the dose-response data infeasible; 
       the development of the dose-response data was completed or updated before the effective date of the final rule; 
       making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, confidential business information, or national security;
       a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis; or
       the factors used in determining the consideration to afford to the pivotal science indicate full consideration is justified.
To assist the Administrator in determining whether to grant an exemption, the EPA program or Region responsible for the significant regulatory action or influential scientific information and public commenters can provide input when the Administrator is considering an exemption. The EPA will document the rationale for the Administrator's exemption in the significant regulatory action or influential scientific information. The EPA is confident that the above criteria provide sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
12.2.1.5 Industry Bias
Comment: Commenters (10339, 11192, 11339, 11344, 11391, 11403, 12430, 12466, 14397, 12715) express concern that the EPA Administrator could choose to exempt studies sponsored or conducted by regulated entities while excluding public health studies, subjecting the process to lobbying and political consideration rather than solely considering a given study's validity or merit. Commenter (12430) specifically notes, "[t]he absence of publication exceptions or exemptions for protected intellectual property makes it far more likely that industry research, rather than academic or public interest research, would form the basis of regulatory decision-making. This could build an industry preference into EPA's regulatory process and disseminated information."
Response: The EPA does not agree with the comments regarding the role of the Administrator in determining whether to grant an exemption under 40 CFR 30.7 and finds that the Administrator is the appropriate decision maker in this context. To ensure that the Administrator's decision is appropriately transparent, in the final rule the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. This documentation would typically be provided as part of the proposed rulemaking given this will be part of the decision of what is the pivotal science for the rule. Regardless, the final rulemaking will provide clear documentation.
The EPA also disagrees that implementation of the rule would result in a bias toward any stakeholder group. As stated in 40 CFR 30.3, "The provisions of this part apply to science that serves as the basis for informing a final significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science." The rule also ensures the availability of the underlying dose-response data for any subject matter expert, regardless of their affiliation. Further, the final rule dictates that the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or propriety information, available through restricted access in a manner sufficient for independent validation. Restricted access in the context of this rule means that data must be available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. CBI data are not given different or preferential treatment. Finally, because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. The final rule also only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
12.2.2 Rule Would Preclude Consideration of Best Available Science
 12.2.2.1 Provision Fails to Guarantee Consideration of Best Available Science
Comment: Commenters (11393, 11395, 11399, 11482, 11484, 11495, 11890, 11903, 12403, 12430, 12434, 12435, 12457, 12466, 12546, 12707, 12727, 14397, 198105) write that due to the lack of clear criteria/standards, lack of oversight, lack of personal scientific knowledge, personal bias, and other factors, the EPA Administrator may engage in "cherry-picking" and arbitrarily exclude important science or exempt lower quality studies, producing an administrative record that bears little resemblance to the total scientific knowledge relevant to an issue. Commenters (11576, 11921) add that allowing exemptions for older studies fails to ensure that such studies  -  which are less likely to be able to obtain and disclose their underlying data  -  will be included.
Response: The EPA would like to clarify that no studies are categorically excluded from consideration under the final rule. In addition, the EPA included criteria that may be used by the Administrator when considering whether to grant an exemption in 40 CFR 30.7. Specifically, the rule states that greater consideration under this rule may be warranted because:
       technological or other barriers render sharing of the dose-response data infeasible; 
       the development of the dose-response data was completed or updated before the effective date of the final rule; 
       making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, confidential business information, or national security;
       a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis; or
       the factors used in determining the consideration to afford to the pivotal science indicate full consideration is justified.
The EPA maintains that the Administrator is the appropriate decision maker in this context. To assist the Administrator in determining whether to grant an exemption, the EPA program or Region responsible for the significant regulatory action or influential scientific information and public commenters can provide input when the Administrator is considering an exemption. To ensure that the Administrator's decision is appropriately transparent, in the final rule the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. This documentation would typically be provided as part of the proposed rulemaking given this will be part of the decision of what is the pivotal science for the rule. Regardless, the final rulemaking will provide clear documentation. The EPA is confident that the above criteria provide sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
12.2.2.2 Too Difficult to Get Exemptions
Comment: Commenters (10038, 100045, 11361) write that the process of getting an exemption for a given study will take too much time and effort, thus establishing barriers and delays that will prevent EPA from considering all available science. Commenter (12458) adds that "by narrowing the administrator's ability to grant exemptions, the supplemental notice would effectively exclude an even larger swath of public health research from consideration in a broader range of agency activities." 
Response: The EPA disagrees with commenters that the exemption process will result in barriers and delays that would prevent EPA from considering all available science. The EPA would like to clarify that no studies are categorically excluded from consideration under the final rule.
While this rule does require the Agency to evaluate the availability of dose-response data for pivotal science, the incremental burden to the Agency to carry out these requirements is expected to be small given (1) the extensive scientific review EPA already conducts regularly and (2) that the requirement is limited to pivotal science (i.e., a small, though highly important, subset of all the studies EPA reviews).
12.2.2.2.1 Exemption Provisions are Too Limited
Comment: Commenters (11388, 12727) write that the provision allowing the Administrator to grant exemptions on a case-by-case basis should be added back into the final rule because it is impossible to foresee every circumstance where an exemption would be warranted. 
Response: Title 40 CFR 30.7 in the final rule allows the Administrator to grant exemptions on a case-by-case basis if he or she determines that greater consideration is warranted because:
       technological or other barriers render sharing of the dose-response data infeasible; 
       the development of the dose-response data was completed or updated before the effective date of the final rule; 
       making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, confidential business information, or national security;
       a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis; or
       the factors used in determining the consideration to afford to the pivotal science indicate full consideration is justified.
12.2.2.2.2 Non-Technological Barriers
Comment: Commenter (11393, 11407, 11484, 11490, 12412, 12474, 12720, 12727, 14397) points out that many factors could prevent studies from being able to comply with the disclosure requirements, including legal, institutional, and moral reasons. Commenters argue that these constraints should also be considered when granting exemptions.
Response: As stated in the final rule, the Administrator may grant an exemption on a case-by case basis if he or she determines that greater consideration is warranted because:
       technological or other barriers render sharing of the dose-response data infeasible; 
       the development of the dose-response data was completed or updated before the effective date of the final rule; 
       making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, confidential business information, or national security;
       a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis; or
       the factors used in determining the consideration to afford to the pivotal science indicate full consideration is justified. 
12.2.2.2.3 Intellectual Property Should Be Exempted
Comment: Commenter (12430) writes, "intellectual property is absent from the list of potential exemptions to the data publication requirements in proposed section 30.9. This lack of protection for intellectual property would thwart innovation and/or prevent the consideration of newer tools and models in EPA's regulatory decision-making." 
Response: EPA notes that the final rule does not require EPA or any other entity to publish data. In any event, if the EPA chooses to make dose-response data publicly available, the final rule requires EPA to do so in accordance with the law. Furthermore, the Administrator may grant an exemption on a case-by-case basis if he or she determines that greater consideration is warranted because, among other factors, technological or other barriers render it infeasible to make the dose-response data publicly available.
12.2.2.3 Legal Mandates
Comment: Commenters (12435, 12466, 12702, 12707, 12727) write that EPA is required by numerous environmental and public health statutes to consider "the best available science" when making regulatory decisions; take into account certain specific factors (such as relevance, importance, or scientific validity) when evaluating a given study; and incorporate input from an independent scientific review committee when considering air quality standards. Commenters write that giving the Administrator discretion to decide exemptions rather than requiring compliance with EPA's congressional mandates violates those obligations.
Response: This internal procedural rule does not interpret or apply provisions of the environmental statutes that EPA administers. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
12.2.3 Areas Needing Additional Clarity/Specificity
Comment: Commenters identified the following exemption related terms/requirements that they contend are ambiguous in the supplemental notice and need clarification:
 The terms "practicable/impracticable" (12720, 12715)
 The terms "feasible/infeasible" (11381, 198105, 12715)
 What constitutes a "technological barrier" (12430, 12727, 12715)
 Who is authorized to decide whether compliance is impracticable (12720)
 How, when, or by whom requests for exemptions or claims of technological barriers would be submitted to EPA (11399, 12720)
 Whether the presence of barriers would be independently confirmed or what efforts might be required to surmount such barriers (12720)
 Whether non-technical (e.g., legal) barriers would qualify for exemptions (12720)
 Whether a waiver would apply to a study publication, the underlying data and models, or both (12720)
 At what point in the process of reviewing pivotal regulatory science would an exemption be granted (11496, 12720)
 How long would an exemption apply (12720)
 Whether an exemption would only apply to the application of specific data or models in specific regulatory context(s) or not (12720)
 Whether any proposed exemption could be challenged, reviewed, or appealed (11399, 12720)
 How various factors will be weighted when determining exemptions (198105)
Response: The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
12.2.4 Transparency
 12.2.4.1 The Process of Granting Exemptions is Not Transparent
Comment: Commenters (10048, 10339, 10749, 11381, 11395, 11399, 11404, 11482, 11495, 11496, 12403, 12434, 12435, 12463) express concern that giving the EPA Administrator sole discretion to grant exemptions without clear criteria, oversight, documentation, or public input will reduce transparency in rulemaking by obscuring which studies are considered or emitted and why.
Commenter (11192) provides that EPA is recommending multiple options for Agency or Administrator discretion without providing the public any details regarding decision-making criteria, much less opportunities for oversight. The commenter contends that for a rule purported to increase Agency decision-making transparency, the EPA has shrouded its proposed decision-making criteria and processes in mystery and thereby opened the door to manipulation and politicization. The commenter strongly opposes these Agency discretions without opportunity for additional input, response, or course-correction.
Response: The EPA maintains that the Administrator is the appropriate decision maker in this context. To ensure that the Administrator's exemption decisions are appropriately transparent, in the final rule the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. This documentation would typically be provided as part of the proposed rulemaking given this will be part of the decision of what is the pivotal science for the rule. Regardless, the final rulemaking will provide clear documentation. To assist the Administrator in determining whether to grant an exemption, the EPA program or Region responsible for the significant regulatory action or influential scientific information and public commenters can provide input when the Administrator is considering an exemption. The EPA will document the rationale for the Administrator's exemption in the significant regulatory action or influential scientific information. The EPA is confident that the above criteria provide sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
12.2.4.2 Granting Exemptions Reduces Transparency in Rulemaking
Comment: Commenter (9362) expresses concern about granting exemptions for studies which rely on private third-party information even if the data are not made public. Commenter argues that such data should not be used, as doing so reduces transparency.
Response: The Administrator may grant an exemption on a case-by-case basis based on criteria provided in 40 CFR 30.7, including if he or she determines that greater consideration is warranted because making the dose-response data publicly available would conflict with laws and regulations governing privacy, confidentiality, CBI, or national security.
EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science is unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. These additional considerations are outlined in 40 CFR 30.5(d).
12.2.4.3 Oversight and Documentation
Comment: Commenters (11192, 11332, 11404, 11495, 12422, 12727, 12715) point out that the rule fails to provide mechanisms for oversight (such as panels, petitions, required record-keeping, etc.) and that the Administrator is not required to report the criteria applied in granting exemptions. Commenter (11404) adds that the current requirement to provide a short summary of the decision-making process is insufficient.
Commenters (11397, 11500, 11892, 12422, 12727) write that when an exemption is granted or rejected, EPA should be required to provide a notice to the public identifying the data or studies as well as the specific rationale and legal basis for the decision. Commenter (11892) specifies that this should include "a summary and citations of information used in decision making; entities or persons consulted if outside the agency; sources of the information, unless prohibited for national or homeland security reasons, and reasons for how/why the decision was made."
Response: To ensure that the Administrator's decision is appropriately transparent, in the final rule the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. This documentation would typically be provided as part of the proposed rulemaking given this will be part of the decision of what is the pivotal science for the rule. Regardless, the final rulemaking will provide clear documentation. 
To assist the Administrator in determining whether to grant an exemption, the EPA program or Region responsible for the significant regulatory action or influential scientific information and public commenters can provide input when the Administrator is considering an exemption. The EPA will document the rationale for the Administrator's exemption in the significant regulatory action or influential scientific information. The EPA is confident that the above criteria provide sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
12.2.4.4 Public Input
Comment: Commenters (10048, 11404, 11482) write that allowing the Administrator to decide exemptions hampers public involvement in decision-making. Commenter (10048) adds that the process of granting exemptions should be subject to public review and comment.
Response: To ensure that the Administrator's decision is appropriately transparent, in the final rule the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. This documentation would typically be provided as part of the proposed rulemaking given this will be part of the decision of what is the pivotal science for the rule. Regardless, the final rulemaking will provide clear documentation. 
To assist the Administrator in determining whether to grant an exemption, the EPA program or Region responsible for the significant regulatory action or influential scientific information and public commenters can provide input when the Administrator is considering an exemption. The EPA will document the rationale for the Administrator's exemption in the significant regulatory action or influential scientific information. The EPA is confident that the above criteria provide sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
12.2.5 Retroactivity
 12.2.5.1 Support for Retroactive Exemptions
Comment: Commenters (11919, 11490, 11491, 11499, 11919, 12402, 12431) support granting exemptions on the basis of the age of a study if the rule is promulgated and write that this exemption should be mandatory. Commenters (11490, 11491, 12727) oppose all retroactive application of the rule.
Commenter (12402) notes that it is clear that older research and data development that may ultimately be considered pivotal to the development of significant regulatory decisions and influential scientific information may not have been conducted and recorded with the intention or provisions to provide public access to underlying data. Therefore, the commenter supports the need for the Administrator to maintain authority to exempt certain pivotal older studies or data that otherwise meet EPA program standards for acceptable data quality from transparency requirements.
Response: This rulemaking applies prospectively to significant regulatory actions and influential scientific information. This rulemaking has no retrospective effect on either final significant regulatory actions or influential scientific information. For future significant regulatory actions and influential scientific information, it applies equally to all dose-response data underlying pivotal science.
The Administrator may grant an exemption for a study on a case-by-case basis based on criteria provided in 40 CFR 30.7, including if he or she determines that compliance is impracticable because the development of the dose-response data was completed or updated before the effective date of the final rule.
12.2.5.2 Age of Studies/Data Should Not Be Considered When Granting Exemptions
Comment: Commenter (11381) writes that making data and models eligible for exemption because they were produced prior to the effective date of the final rule is unnecessary and undermines transparency. Commenter (12471) also opposes consideration of age on the basis that "the rule would have very little impact for several years since most rules rely on studies completed and peer reviewed several years ago (if not decades earlier)." Commenter (12699) notes that allowing exemptions for older studies is unnecessary because studies can already be exempted on the basis of genuine privacy, confidentiality, and CBI concerns, and the provision would encourage the use of old data while discouraging the use of new data. Commenter (12399) adds that exempting all research conducted before the implementation of the rule would essentially make the rest of the rule meaningless.
Response: This rulemaking applies prospectively to significant regulatory actions and influential scientific information. This rulemaking has no retrospective effect on either final significant regulatory actions or influential scientific information. For future significant regulatory actions and influential scientific information, it applies equally to all dose-response data underlying pivotal science .
The Administrator may grant an exemption for a study on a case-by-case basis based on criteria provided in 40 CFR 30.7, including if he or she determines that compliance is impracticable because the development of the dose-response data was completed or updated before the effective date of the final rule.
Comment: Commenter (11381) writes that the proposed exemption for studies completed or updated prior to the rule being promulgated violates the Information Quality Guidelines (IQGs), which do not exempt information disseminated by EPA prior to when they came into force.
Response: This rulemaking applies prospectively to significant regulatory actions and influential scientific information. This rulemaking has no retrospective effect on either final significant regulatory actions or influential scientific information. For future significant regulatory actions and influential scientific information, it applies equally to all dose-response data underlying pivotal science.
12.2.5.3 Exemptions Should Only Be Available for Older Studies
Comment: Commenter (11922) writes that exemptions should only be available for studies generated prior to the promulgation of the rule to encourage compliance by current researchers.
Response: The EPA would like to clarify that this rule exclusively governs the EPA's internal process for determining the consideration to afford pivotal science with respect to certain actions. If researchers want to increase the likelihood that their studies receive greater consideration by the EPA, they may take steps to ensure that the underlying dose-response data are available to the greatest extent possible. But any such response to this final rule would be purely voluntary. 
As stated in the rule, the Administrator may grant an exemption on a case-by case basis if he or she determines that greater consideration is warranted because:
       technological or other barriers render sharing of the dose-response data infeasible; 
       the development of the dose-response data was completed or updated before the effective date of the final rule; 
       making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, confidential business information, or national security;
       a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis; or
       the factors used in determining the consideration to afford to the pivotal science indicate full consideration is justified.
This rulemaking applies prospectively to significant regulatory actions and influential scientific information. This rulemaking has no retrospective effect on either final significant regulatory actions or influential scientific information. For significant regulatory actions and influential scientific information, it applies equally to all dose-response data underlying pivotal science.
12.2.5.4 Exemptions Based on the Age of a Study
Comment: Commenters (11388, 11386, 11395, 11484, 11490, 11491, 12702, 12710, 12727) provide that any information that originated prior to the finalization of the proposed rulemaking should be excluded from the requirements. Specifics include:
Commenter (11388) supports "grandfathering" completed work, models, and data that have already been used by the Agency and supports implementing this rule in stages to prevent further delay of EPA's work. The commenter recommends ensuring that during this review process, studies that have historically been used by the Agency to make regulatory decisions are not rejected from use because of this rulemaking.
Commenter (11386) provides that the rule should only apply to data and models that are generated after the effective date. According to the commenter, even if the transparency of data and ability to be reanalyzed did indicate quality of science, then this information was available to EPA when considering past rules. The commenter states that nothing about the science used to make previous rules will be changed as a result of this new rule. Thus, the commenter notes, to suddenly treat previous studies in a way different from how they were previously treated by the EPA would be arbitrary and capricious, as nothing about the studies themselves had changed. Unless new information came to light, the commenter states, then it would be quite reasonable to change how older studies are considered, but to do so due to factors that have nothing to do with the science itself would be arbitrary.
Commenter (11395) notes that the SNPRM asks whether age of data and models should be considered in determining the feasibility of making underlying data and models publicly available and whether additional factors should also be considered in this determination. The commenter supports the consideration of age of data, as well as privacy laws, technology barriers, the nature of confidentiality agreements, and other factors that may preclude data accessibility, in any feasibility determination. The commenter states that an older study should not be excluded or devalued solely because raw data are no longer available or because accessibility to those data are restricted by use agreements in place at the time the study was conducted. The commenter states that, in addition, any other factor that precludes or limits data accessibility, including, as listed in section 30.9 of the SNPRM, technological barriers and conflicts with "laws governing privacy, confidentiality, CBI, or national and homeland security," should be considered. The commenter states that the criteria should be specified in guidelines, and data availability should be considered as only one factor in weighing the quality of studies.
Commenter (11484) writes that the age of a study "has no direct bearing on the feasibility of publicly sharing the underlying data. Although it can be credibly argued that it is much harder in 2020 to access the data underlying a study published in 1970, the same argument cannot be credibly made for a study published in 2019."
Commenter (12702) states that EPA confirms that it intends to apply the requirements of the rule to only future rulemakings in the supplemental proposal. This, according to the commenter, would help avoid regulatory uncertainty or potential disruptions for entities that have complied with past rulemakings. Additionally, the commenter encourages that requirements only be applied to future data and models to avoid undermining existing science on which decades of rulemakings rely and thus support consistency in ongoing rulemakings.
Commenter (12710) states that most studies prior to the digital age would not meet the proposed requirements, leading to EPA dismissing most science conducted in the 20th century. The commenter notes that the supplemental proposal would apply the rule to data and models evaluated at the time a significant regulatory action or influential scientific information is developed, regardless of when the data and models were generated," and recommends that the rule only be applied to studies generated after the effective date of the supplemental proposal.
Commenter (12727) asserts the new language in the SNPRM confirms it applies to pre-existing data and studies unless the Administrator exercises their discretion to exempt them due to the inclusion of the exemption criterion of the age of the data. The commenter states EPA must expressly not apply the rule to data completed or updated prior to the rulemaking as technological barriers and costs would prevent a significant number of older studies from being reviewed and brought into compliance.
Response: This rulemaking applies prospectively to significant regulatory actions and influential scientific information. This rulemaking has no retrospective effect on either final significant regulatory actions or influential scientific information. For significant regulatory actions and influential scientific information, it applies equally to all dose-response data underlying pivotal science.
Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science is unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. These additional considerations are outlined in 40 CFR 30.5(d).
The Administrator may grant an exemption for a study on a case-by-case basis based on criteria provided in 40 CFR 30.7, including if he or she determines that greater consideration is warranted because the development of the data was completed or updated before the effective date of the final rule.
Comment: Commenter (14397) notes that all studies begun prior to the date the SNPRM is finalized should be given a nondiscretionary, mandatory exemption from the rule.
Commenter (11900) suggests that, rather than pressure researchers into sharing their data in ways that their informed consent could not have anticipated, potentially exposing them to legal liability, the EPA should exempt all research initiated prior to the effective date of the final rule from the requirements of the rule. The commenter states that, to ensure the exemption is applied without bias, it should be applied on a blanket basis to all studies initiated prior to the passage of the rule -- and not on a case-by-case basis by the administrator. The commenter states that the exemption should continue in perpetuity, and exempted research should carry equal weight to research that complies with the requirements of the final rule.
Response: This rulemaking applies prospectively to significant regulatory actions and influential scientific information. This rulemaking has no retrospective effect on either final significant regulatory actions or influential scientific information. For future significant regulatory actions and influential scientific information, it applies equally to all dose-response data underlying pivotal science. In addition, the Administrator may grant an exemption for a study on a case-by-case basis based on criteria provided in 40 CFR 30.7, including if he or she determines that greater consideration is warranted because the development of the data was completed or updated before the effecitive date of the final rule.
This rule does not require any researcher or other outside entity to provide dose-response data to the EPA. If researchers want their studies to receive greater consideration by the EPA, they may take steps to ensure that the dose-response data are available to the greatest extent possible. But any such response to this final rule would be purely voluntary. It is not required by the rule.
12.2.6 Support for Exemptions
 12.2.6.1 General Support
Comment: Commenters (11499, 11500) express general support for the version of section 30.9 put forward in the supplemental notice. Commenter (11397) writes that exemptions should be granted only after exhausting all other options for enabling public access to data.
Response: 40 CFR 30.7 in the final rule is similar to the version in the SNPRM except it includes the criteria the Administrator should consider when determining whether to grant an exemption and notes that the rationale for an exemption shall be documented in the significant regulatory action and influential scientific information. This documentation would typically be provided as part of the proposed rulemaking given this will be part of the decision of what is the pivotal science for the rule. Regardless, the final rulemaking will provide clear documentation.
An exemption will only be granted for cases where all options for enabling public access to data have been exhausted. 
Comment: Commenter (12402) encourages the EPA to pursue and provide public access to underlying pivotal data whenever possible, even in cases where the data were developed prior to the effective date of this rule. The commenter states that it is clear that older research and data development that may ultimately be considered pivotal to the development of significant regulatory decisions and influential scientific information may not have been conducted and recorded with the intention or provisions to provide public access to underlying data. For this reason, the commenter supports the need for the Administrator to maintain authority to exempt certain pivotal older studies or data that otherwise meet the EPA program standards for acceptable data quality from transparency requirements. 
Response: This rulemaking applies prospectively to significant regulatory actions and influential scientific information. This rulemaking has no retrospective effect on either final significant regulatory actions or influential scientific information. For future significant regulatory actions and influential scientific information, it applies equally to all dose-response data underlying pivotal science. 
The Administrator may grant an exemption for a study on a case-by-case basis based on criteria provided in 40 CFR 30.7, including if he or she determines that greater consideration is warranted because the development of the dose-response data was completed or updated before the effective date of the final rule.
12.2.6.2 Scope
Comment: Commenters (11358, 11499) support that EPA has modified the scope for exemptions to include instances where data may not be available due to technological barriers. 
Commenters (10866, 11397) support the deletion of the provision concerning the granting of exemptions based on impracticability of peer review.
Response: The Administrator may grant an exemption for a study on a case-by-case basis based on criteria provided in 40 CFR 30.7, including if he or she determines that greater consideration is warranted because technological or other barriers render sharing of the dose-response data infeasible. 
Title 40 CFR 30.7 of the final rule does not include the exemption on a case-by-case basis if the Agency determined that it was not feasible to conduct independent peer review on pivotal regulatory science, which was removed in the SNPRM.
12.2.7 Other Comments
 12.2.7.1 Exemptions Provision Undermines Proposed Rule
Comment: Commenter (12715) writes, "given how severely the supplemental proposal would limit the scientific evidence available for EPA's use, the proposed exemption provisions could become the basis upon which most of the science relied on by EPA in its rulemaking is admitted. The exceptions could thus largely swallow the rule, resulting in greater arbitrariness in EPA regulatory actions rather than EPA's alleged goal of greater transparency." 
Response: The EPA would like to clarify that no studies are categorically excluded from consideration under the final rule. Further, as discussed in the preamble to the final rule, although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science is unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. These additional considerations are outlined in 40 CFR 30.5(d).
12.2.7.2 Rule Will Reduce Trust in EPA's Regulatory Decisions
Comment: Commenters (11399, 11482, 12431) write that if this rule is promulgated, allowing the EPA Administrator sole discretion could give the public the impression that decisions are arbitrary or biased, which may result in a loss of trust in EPA's regulatory decisions. 
Response: The EPA maintains that the Administrator is the appropriate decision maker in this context. 
To ensure that the Administrator's decision is appropriately transparent, in the final rule the EPA has included a provision in 40 CFR 30.7 that requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. This documentation would typically be provided as part of the proposed rulemaking given this will be part of the decision of what is the pivotal science for the rule. Regardless, the final rulemaking will provide clear documentation. 
To assist the Administrator in determining whether to grant an exemption, the EPA program or Region responsible for the significant regulatory action or influential scientific information and public commenters can provide input when the Administrator is considering an exemption. The EPA will document the rationale for the Administrator's exemption in the significant regulatory action or influential scientific information. The EPA is confident that the above criteria provide sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
12.2.7.3 Exemptions Provision Demonstrates Arbitrariness of Larger Rule
Comment: Commenters (11403, 11892, 12727) write that allowing the Administrator to grant exemptions to some studies demonstrates EPA's awareness that public availability of data is not necessarily indicative of quality or validity. Commenter (11892) notes, "It is unclear why an exemption from compliance with the rule would be needed if the final rule is not excessively burdensome." Commenter (12399) also writes that because laws prevail over Agency rules, it is not necessary to state that the rule would not apply in the case of a conflict, and that doing so "appears to do nothing more than give the Administrator an excuse to waive the rule for rather arbitrary reasons."
Response: As more fully discussed in the preamble of the final rule, the Agency is retaining the Administrator's exemption provision because there are conditions under which compliance with the requirements in 40 CFR part 30 might be impracticable. For example, the underlying dose-response data for some studies, particularly older studies, may not be readily publicly available because of the technological barriers to data sharing (e.g., differences in data storage devices or data retention practices) that existed when they were developed. As a result, the EPA is finalizing the Administrator's exemption provision as proposed in the 2020 SNPRM, with additional conditions. The EPA is confident that the above criteria provide sufficient clarity and boundaries for the Administrator to consider when granting an exemption under 40 CFR 30.7.
12.2.7.4 Waiver Cannot Save an Unlawful Rule
Commenter: Commenters (12715, 198105) point to Alltel Corp. v. Federal Communications Commission, in which the D.C. Circuit found that an agency "cannot save an irrational rule by tacking on a waiver procedure" because the "essence of waiver is the assumed validity of the general rule." Commenter (198105) writes that because the proposed rule is not authorized by any of the statutory authorities cited in the supplemental notice and it "actively hinders EPA's ability to effectuate the statutes that it is charged with implementing," the rule is unlawful, and thus cannot be rescued by the creation of a waiver provision.
Commenter (12731) states that the EPA's proposal to allow the Administrator to grant exemptions from the rule's disclosure requirements demonstrates the EPA's awareness that a study can be valid and worthy of consideration even if underlying data or models are not publicly available. The commenter states that, where a statute requires that the Agency consider certain information in reaching a decision, the EPA cannot promulgate a rule that prohibits the Agency from considering such information, that automatically downgrades the value placed on such information regardless of its scientific merit, or that gives the Administrator discretion to decide whether the Agency will consider such information. The commenter states that none of the SNPRM's changes to proposed section 30.9 do anything to resolve this fundamental conflict with the relevant environmental and public health statutes. The commenter states that the mere fact that the Administrator might in the future use his or her exemption authority to allow the Agency to consider a particular study does not resolve the rule's unlawfulness.
Response: EPA notes that this final rule does not categorially exclude any studies. As detailed in the preamble to the final rule, the EPA is authorized to promulgate this procedural rule under its authority to promulgate housekeeping regulations governing its internal affairs. This final rule does not interpret or apply the provisions of any environmental statutes; such efforts will occur in the subsequent rulemakings under the relevant statutes. If implementing this procedural rule would result in conflicts with existing environmental statutes, the procedural rule will yield to the EPA statutes and regulations.
 Chapter 13: Peer Review
The NPRM (40 CFR 30.7) includes the following requirement:
      EPA shall conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the Office of Management and Budget (OMB) Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein. 
The EPA solicited comment in the notice of proposed rulemaking (NPRM) on whether the exemptions in Chapter 9 of the OMB's Final Information Quality Bulletin for Peer Review are appropriate, and on whether there are other situations in which specific significant regulatory actions, or specific categories of significant regulatory actions should be exempted.
Under the supplemental notice of proposed rulemaking (SNPRM), the EPA clarifies that neither the proposed nor the alternative 40 CFR 30.5 changes would require that EPA, a member of the public or other entity must independently validate a study before it can be considered to be pivotal regulatory science or pivotal science. The EPA also clarifies that independent validation is not required under proposed 40 CFR 30.7 which describes the role of independent peer review.
Related, the 2018 NPRM also included a provision at 40 CFR 30.9 allowing the Administrator to grant exemptions from the rule on a case-by-case basis if he or she determines that compliance is impracticable because it is not feasible to conduct independent peer review on all pivotal regulatory science. The SNPRM proposes to delete that proposed exemption language. 
Several comments were received regarding the proposal's reference to and consistency with the OMB Bulletin and issues related to "independent peer review." These comments are organized according to the following topics under sections 13.1 NPRM Peer Review-Related Comments and 13.2 SNPRM Independent Peer Review:
Section 13.1: NPRM Peer Review-Related Comments
       Office of Management and Budget (OMB) Final Information Quality Bulletin for Peer Review
       Independent Peer Review Requirements (40 CFR 30.7)
       Comparison of the Approach in the Proposed Rule to Existing Scientific Review Processes
Section 13.2 SNPRM Independent Peer Review
       Existing Peer Review Processes and Protocols are Sufficient
       Concerns Related to Transparency in EPA's Peer Reviews
       Supplemental Proposal Violates the Principles of the OMB Final Information Quality Bulletin for Peer Review
       Resource Burdens of Peer Review Requirements
       Considerations of Bias and Independence of Peer Reviewers
The specific comments provided in this Chapter reflect the comment letters that provide arguments supported by detailed rationale, examples, and references/documentation. The Docket ID numbers for these commenters are cited throughout this Chapter. In addition, some general comment letters may also include comments related to comments included and responded to in this Chapter. These comments are summarized in Appendix B. Note that points and issues raised in these general comment letters may also be raised by the more detailed comments summarized in this Chapter. The responses provided in this Chapter adequately address these more general comments.
13.1 NPRM Peer Review-Related Comments
 13.1.1 Office of Management and Budget (OMB) Final Information Quality Bulletin for Peer Review
 13.1.1.1 Statutory Authority to Codify the OMB Bulletin 
Comment: Commenters (6133, 9227) argue that the EPA does not have the legal authority to codify the requirements of the OMB Bulletin.
Commenter (6133) states that the EPA proposes to bind itself to conduct a particular form of peer review based on an OMB Bulletin that has never been the subject of notice and comment rulemaking and is not authorized by any federal law. According to the commenter, treating the content of the OMB Bulletin as a binding regulation would violate the federal statutes the EPA implements because those statutes do not codify the Bulletin. In addition, the commenter asserts that the proposal identifies no statutory authority for EPA to conduct independent peer review for any "pivotal regulatory science" consistent with the Bulletin, nor did the proposal identify the statutory authority for the EPA to bind itself to conduct peer review only in this fashion. Further, the commenter says the proposal identifies no Congressional intent for EPA to conduct peer review consistent with the OMB Bulletin, although Congress has known about the OMB Bulletin since 2005. The commenter contends that the EPA has made up proposed § 30.7 and the putative authority for the Proposal out of whole cloth.
Commenter (9227) asserts that there is no statutory authority for EPA to "conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein." The federal statutes that EPA lists as putative authority for the Proposal provide no authority for proposed § 30.7. See 83 Fed. Reg. at 18,769/2 (citing Clean Air Act (CAA) sections 103, 301(a), 42 U.S.C. 7403, 7601(a); Clean Water Act (CWA) sections 104, 501, 33 U.S.C. 1254, 1361; Safe Drinking Water Act (SDWA) sections 1442, 1450(a)(1), 42 U.S.C. 300j - 1, 300j -  9(a)(1); Resource Conservation and Recovery Act (RCRA) sections 2002(a)(1), 7009, 42 U.S.C. 6912(a)(1), 6979; Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA) (as delegated to the Administrator via Executive Order 12580) sections 115, 311, 42 U.S.C. 9616, 9660; Emergency Planning and Community Right-To-Know Act (EPCRA) section 328, 42 U.S.C. 11048; Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA) sections 25(a)(1), 136r(a), 7 U.S.C. 136r(a), 136w; and Toxic Substances Control Act (TSCA) as amended, section 10, 15 U.S.C. 2609, and the Administrative Procedures Act (APA)). According to the commenter, the claimed authorities that EPA lists do not mention peer review or even allude to the concept. Id. The commenter provides that neither the Proposal nor accompanying docket materials identify any provision of any federal statute that authorizes EPA to promulgate proposed § 30.7 and that their own research revealed no provision of any federal statute that authorizes EPA to promulgate proposed § 30.7. The commenter asserts that Congress writes federal statutes. 
Response: The OMB Bulletin provides that "agencies are granted broad discretion to weigh the benefits and costs of using a particular peer review mechanism for a specific information product. The selection of an appropriate peer review mechanism for scientific information is left to the agency's discretion". EPA is subject to the OMB Bulletin and its provisions have been incorporated into the Agency's peer review handbook. EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will lead to consistent implementation of those procedures across the Agency.
As stated in the final rule, EPA is authorized to promulgate this procedural rule under its authority to promulgate housekeeping regulations governing its internal affairs. The final rule does not interpret or apply the provisions of any environmental statutes. If implementing this procedural rule would result in conflicts with existing EPA environmental statutes, and their implementing regulations, the procedural rule will yield to the existing statutes and regulations.
Comment: Commenter (6449) notes that the OMB Bulletin implements the Information Quality Act (IQA) (Pub L. No. 106-554, § 515, Dec. 21, 2000) and that the rule should reference the IQA along with the OMB Bulletin.
Response: EPA references the guidelines because they provide interpretations of key terms. EPA relied on these guidelines, as well as other federal guidance documents that require greater data transparency, when describing the consistency of this rule's focus on transparency, the need to characterize the sensitivity of an agency's conclusions to analytic assumptions.
13.1.1.2 Existing OMB Bulletin Sufficient
Comment: Commenters (6915, 6925) assert that there is an existing and well-established framework in place governing the use of scientific information to ensure that the information informing regulatory actions is the best-available, peer-reviewed science. These commenters point to the OMB Bulletin as one of the policy/procedure documents that comprises that existing framework.
Commenter (6895) asserts that the OMB Bulletin states that pivotal research must include information so that agency reviewers can "ensure that scientific uncertainties are clearly identified and characterized." The commenter remarks that this is the task of peer reviews, and there is no need to make anything else available if scientific uncertainties can be quantified.
Response: The EPA would like to clarify that the rule requirements are not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary" (Ref. 8). Peer review does not typically include reanalysis of the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information) and, thus, peer review is not considered a replacement for the data availability requirements of this rule. 
Comment: Commenter (9227) asserts that the EPA has not provided any information to suggest that EPA is not already following the OMB Bulletin. According to the commenter, the EPA's lack of any supporting rationale or analysis frustrates the public's ability to provide meaningful comment on the 40 CFR 30.7 provision that "EPA shall conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 Fed. Reg. 2664) and the exemptions described therein," and is itself a sign that this requirement is fundamentally arbitrary.
Response: The reference to OMB Final Information Quality Bulleting for Peer Review in the final rule is to ensure that EPA will conduct any peer reviews required under the rule in accordance with this existing policy and procedures.
13.1.1.3 Consistency of Proposal with the OMB Bulletin 
 13.1.1.3.1 OMB Bulletin Does Not Require Re-review of Adequately Peer-Reviewed Studies 
Comment: Commenters (6125, 9227) state that the EPA must clarify that studies that have already been adequately peer-reviewed by third parties do not need to be re-reviewed by the EPA, as provided for in the OMB Bulletin. Commenter (9227) also states that the EPA must clarify that prior peer reviews are not subject to the requirement in §30.7 that reviewers consider the strengths and weaknesses of EPA's justifications.
Commenter (9227) notes that proposed §30.7 expressly incorporates the OMB Bulletin, which provides that "agencies need not have further peer review conducted on information that has already been subjected to adequate peer review." The commenter also notes that the language in proposed §30.7 instructs the EPA to "ask peer reviewers to articulate the strengths and weaknesses of EPA's justifications for the assumptions applied and the implications of those assumption for the results." The commenter remarks that peer reviews conducted prior to the EPA's reliance on a study could not have involved a review of the strengths and weaknesses of EPA's justifications, and this language implies that the EPA is required to re-peer review all influential scientific information.
Commenters (4589, 6137, 9927) state that requiring a re-review of previously peer-reviewed studies would burden the EPA with unnecessary, duplicative and significant costs and would prevent the EPA from fulfilling its statutory mission of protecting public health and the environment by bringing many EPA rulemakings to a standstill. 
Commenters (6125, 9227) suggest that in order to prevent stalling rulemakings, the EPA must clarify that the rule will not supersede EPA's existing authority under the OMB Bulletin to not conduct further peer review where information has already been subject to adequate peer review. Commenter (6137) states that the OMB Bulletin recognizes, "[p]ublication in a refereed scientific journal may mean that adequate peer review has been performed." Commenter (6125) specifically requests that the EPA revise the proposed rule language to clarify what is required and what is advised.
Response: The EPA would like to clarify that the rule requirements are not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review does not typically include reanalysis of the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information) and, thus, peer review is not considered a replacement for the data availability requirements of this rule. The EPA is, therefore, finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate peer review, consistent with the OMB's Bulletin for Peer Review and the EPA's Peer Review Handbook, of individual studies identified as pivotal science if the studies have already undergone journal peer review. If the Agency conducts peer review on pivotal science, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied in analyzing dose-response data and the implications of those assumptions for the results.
To address the public comment regarding "unnecessary duplicative and significant costs", the preamble of the final rule specifies, "given the costs of the current robust process for identifying and reviewing scientific studies and documentation that are existing Agency practice, as well as that the determination of dose-response data availability is limited to pivotal science underlying significant regulatory actions and influential scientific information, the EPA anticipates that the incremental costs of this rule will be small....relative to current administrative costs for developing significant regulatory actions or influential scientific information."
13.1.1.3.2 The Proposal Will Lead to Undue Delays, Contrary to the OMB Bulletin
Comment: Commenter (6137) asserts that the provisions in the proposed rule requiring independent peer review on all pivotal regulatory science used to justify regulatory decisions is inconsistent with provisions in the OMB Bulletin that exclude time-sensitive health and safety disseminations, which encompasses most of the studies the EPA intends to review, and direct agencies to ensure peer review does not unduly delay the release of urgent findings. The commenter states that in contrast to the OMB Bulletin, the proposal appears to require another layer of peer review, which would lead to unnecessary delays. The commenter notes that there can be hundreds to thousands of studies supporting a standard, such as the national ambient air quality standards (NAAQS), and it is difficult to see how the EPA could subject all of these to an additional peer review process, which can take months for each study to be reviewed, and still complete scientific assessments in a timely manner. The commenter asserts that the EPA must clearly outline how it will ensure the additional peer review process will not lead to undue delay.
Response: EPA disagrees that the rule adds another layer of peer review that could result in unnecessary delays. EPA clarified in 40 CFR 30.6 that "the EPA will evaluate whether or not to conduct additional peer review of individual studies identified as pivotal science if the studies have already undergone peer review." This should ensure that no unnecessary peer reviews are conducted that could cause undue delay.
13.1.1.3.3 Independent Peer Reviewer OMB Bulletin Guidelines 
Comment: Commenter (6120) urges the EPA to follow the OMB Bulletin guidelines for peer review to ensure peer reviewers are independent of industry and include academic scientists. The commenter provides that the OMB Bulletin states that "For scientific assessments relevant to specific regulations, a reviewer's financial ties to both regulated entities (e.g., businesses) and the agency should be examined" and "goes beyond financial investments and business relationships and includes work as an expert witness, consulting arrangements, honoraria and sources of grants and contracts." The commenter further provides that the OMB Bulletin also clearly states that scientists who receive government research grants are still able to offer "independent scientific advice to the agency on other projects."
Commenter (1972) requests that EPA clarify the use of peer review in proposed 40 CFR 30.7. The commenter provides that this section states that "EPA shall conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein." The commenter further provides that the OMB Bulletin states that, for more important work, "independent" peer review should involve "[r]reviewers" that "are generally not employed by the agency or office producing the document. As the National Academy of Sciences (NAS) has stated, `external experts often can be more open, frank, and challenging to the status quo than internal reviewers, who may feel constrained by organizational concerns.'" Citing Nuclear Regulatory Commission (NRC), Peer Review in Environmental Technology Development Programs: The Department of Energy's Office of Science and Technology, Nat'l Academy Press (1998) at 3. According to the commenter, the EPA's own Guidelines similarly note that "[f]or those work products that are intended to support the most important decisions or that have special importance in their own right, external peer review is the procedure of choice." The commenter asserts that, given the credibility bestowed by both the courts and the public upon the studies that are "peer reviewed," the EPA should also clarify the conditions/parameters under which it will use truly independent external peer reviewers. 
Response: The final rule language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) clarifies that the Agency will evaluate whether or not to initiate peer review, consistent with the OMB's Bulletin for Peer Review and the EPA's Peer Review Handbook, of individual studies identified as pivotal science if the studies have already undergone journal peer review. If the Agency conducts peer review on pivotal science, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied in analyzing dose-response data and the implications of those assumptions for the results.
Adequate peer review is peer review that is consistent with the requirements of the OMB Peer Review Bulletin and EPA's Peer Review Handbook. The Agency will continue conducting peer reviews according to the peer review policies and practices that were in place before the rule goes into effect. The EPA Peer Review Handbook, 4[th] Edition, describes how peer reviewers are selected by the Agency.
13.1.1.3.4 OMB Bulletin Exemption Criteria
Comment: Commenter (9227) states that the EPA must clarify that the OMB Bulletin waiver provisions will remain in effect. The commenter notes that proposed §30.9(b) provides that the Administrator may grant an exemption from the peer review requirements if he or she determines that "[it] is not feasible to conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions for reasons outlined in OMB Final Information Quality for Peer Review (70 FR 2664), Section IX." The commenter notes that only two of seven exemptions in Section IX of the OMB Bulletin relate to feasibility, and suggests that the EPA amend its Proposal to clarify that all of the exemptions in the OMB Bulletin remain in effect regardless of whether they pertain to feasibility. The commenter recommends that the EPA clarify what additional effect, if any, is intended by the exemption provision in the proposed §30.9.
Commenter (9227) requests that the EPA amend the proposed rule to confirm that the "Deferral and Waiver" provision of Section VIII of the OMB Bulletin remains in effect for EPA. The commenter contends that this provision allows an agency to waive or defer the peer review requirements of Sections II and III of the OMB Bulletin where warranted by a compelling rationale to ensure flexibility in unusual and compelling situations that are not otherwise covered by the exemptions in the OMB Bulletin. The commenter contends that without clarifications in proposed §30.9(b), it is possible that it could be read to encompass the entirety of the Administrator's ability to grant exemptions, supplanting Section VIII of the OMB Bulletin.
Commenter (6375) states that the exemptions described in the rule and under OMB's Information Quality Bulletin for Peer Review to ensure compliance with privacy and national security laws and regulations are appropriate. The commenter recommends that the EPA handle these exemptions in a similar manner to the case-by-case exceptions for studies: EPA should provide a robust and transparent explanation in the Federal Register early in the rulemaking process to explain why the provisions of this rule do not apply to the rulemaking in question (e.g. which laws would be in conflict). The commenter believes the exemption for "promptness" under OMB's Information Quality Bulletin for Peer Review is appropriate for an exemption except during the implementation after the effective date of this regulation. The commenter states that this rule will initiate a significant change in the way EPA manages pivotal regulatory science and some time will be needed to set up new processes and systems to ensure internal compliance. The commenter suggests that the EPA identify in any rule finalized during the implementation window which dose-response studies used as pivotal regulatory science have not been made publicly available and release a schedule for making those studies available. The commenter states that once those processes and systems are set up, EPA should be evaluating new dose-response studies well in advance of any rulemaking activities such that exemptions for "promptness" should not be needed.
Response: 40 CFR 30.6 in the final rule clarifies that "the EPA shall conduct independent peer review consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review and the exemptions described therein." 
In the 2020 SNPRM, the EPA did not incorporate the OMB Bulletin for Peer Review Section IX exemptions to peer review, including routine statistical information and financial information. Rather than modify the proposed 40 CFR 30.2 definition of "influential scientific information," the EPA is modifying 40 CFR 30.3 in the final rule to clarify the Agency's intent. As a result, the requirements in 40 CFR 30.3 apply to influential scientific information, unless the influential scientific information is exempted from peer review requirements as described in Section IX of the OMB Bulletin for Peer Review. Consistent with this approach, the EPA is finalizing the definition of "influential scientific information" as proposed in the 2020 SNPRM. The Agency will continue to follow the provisions of the OMB Bulletin for Peer Review.
The basis for all rulemaking is provided for public review and comment in the public docket when the proposed rule is issued or if subsequently added to the docket through a separate opportunity for public comment. The EPA expects the basis for its decisions in proposed significant regulatory actions or external review drafts of influential scientific information, which will allow the public, if they so choose, to provide comment to the EPA.
13.1.1.3.5 EPA Does Not Explain Whether the OMB Bulletin is or is not Consistent with the Proposal
Comment: Commenter (6133) says the regulatory text of the proposal requiring EPA to conduct peer review on pivotal regulatory science consistent with the OMB Bulletin is too vague for the reader to understand what the EPA intends and whether the EPA's regulatory science would be consistent with the OMB Bulletin. Similarly, commenter (9227) asserts that there is some vagueness as to whether the Proposal maintains, expands, or narrows the OMB Bulletin requirements. Commenter (6133) states that it appears that neither the memorandum pointed to in the proposal on the issuance of the OMB Bulletin nor the OMB Bulletin itself supports the EPA's proposal regarding peer review. The commenter notes that the OMB Bulletin does not address standardized test methods or good laboratory practices, which the EPA proposes to use. The commenter also points out that the proposal does not identify any peer-reviewed studies without publicly available data that reached incorrect conclusions, nor does it explain how the current peer review process EPA uses for disseminating information conflicts with the OMB Bulletin. Additionally, the commenter notes that EPA does not present any strategies for dealing with past studies. The commenter asserts that before the EPA adopts any final rule with this requirement, it must propose a new rule with clear regulatory text, justification for its independent peer review requirement, and supporting legal, factual, scientific and technical information.
Response: The EPA disagrees with the commenter and believes that the language at 40 CFR 30.6 in the final rule is clear. The language clarifies that "the EPA shall conduct independent peer review consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review and the exemptions described therein. The EPA will evaluate whether or not to conduct additional peer review of individual studies identified as pivotal science if the studies have already undergone peer review. Because transparency in pivotal science includes addressing issues associated with assumptions used in analyzing dose-response data, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied and the implications of those assumptions for the results."
Adequate peer review is peer review that is consistent with the requirements of the OMB Peer Review Bulletin and EPA's Peer Review Handbook. The Agency will continue conducting peer reviews according to the peer review policies and practices that were in place before the rule goes into effect. 
13.1.1.4 OMB Bulletin Implementation Costs and Benefits Must Be Considered to be Judicially Enforceable 
Comment: Commenter (9227) asserts that the proposal would apparently make the requirements of the OMB Bulletin judicially enforceable, and the EPA did not analyze the costs and benefits associated with this action. The commenter notes that the OMB Bulletin requirements are not currently judicially enforceable, and the EPA may become subject to numerous legal challenges to its regulations based on compliance with the OMB Bulletin requirements. The commenter points out that there would be a financial cost related to the litigation and a cost to public health and the environment due to delays in the implementation of regulations. The commenter asserts that the EPA must evaluate the anticipated costs and benefits from these proposed requirements and explain why they are needed.
Response: This rule exclusively pertains to the internal practices of the EPA and does not regulate the conduct or determine the rights or obligations of any entity outside the federal government. Therefore, as detailed in the preamble to the final rule, the incremental burden to the Agency to carry out the requirements of this rule is expected to be small given (1) the extensive scientific review the EPA already conducts regularly and (2) that the requirement is limited to pivotal science (i.e., typically a small, though highly important, subset of the studies the EPA would review).
13.1.1.5 EPA Peer Review Handbook
Comment: Commenter (9227) provides that the EPA's Peer Review Handbook incorporates the provisions of OMB's Peer Review Bulletin. In the Handbook, the commenter states that the EPA confirms that it "conducts peer review of its products in accordance with the guidance in the OMB Peer Review Bulletin." However, the commenter provides that the EPA Peer Review Handbook adds details and specific procedures that are not present in the OMB Peer Review Bulletin. According to the commenter, surprisingly, EPA's proposed peer review regulations do not even mention EPA's Peer Review Handbook, let alone explain how the new proposed regulations would impact EPA's compliance with the Handbook. For example, the commenter states that the EPA's Handbook specifies "exemption criteria" in section 3.3. The commenter requests that the EPA clarify whether anything in the proposed peer review regulation would supplant instructions in the Peer Review Handbook, and if so, provide a reasoned explanation for the change. Likewise, the commenter requests that the EPA explain the role of the Peer Review Handbook going forward in administering peer review requirements.
Response: Adequate peer review is peer review that is consistent with the requirements of the OMB Peer Review Bulletin and EPA's Peer Review Handbook. The Agency will continue conducting peer reviews according to the peer review policies and practices that were in place before the rule goes into effect. If implementing this procedural rule would result in conflicts with the guidance found in the Agency's existing Peer Review Handbook, the conflicting requirements found in the Handbook will yield to this final rule.
13.1.2 Independent Peer Review Requirements (40 CFR 30.7)
 13.1.2.1 EPA Shall Conduct Independent Peer Review on All Pivotal Regulatory Science Used to Justify Regulatory Decisions
 13.1.2.1.1 Support the Proposed Independent Peer Review Requirements
Comment: Commenters (0045, 4541, 6150, 6180, 6891, 8281-PH85) support the proposed independent review requirements.
Commenter (0045) states that the public good will be served by the proven discipline of peer review and reproducibility on all sources. Commenter (4541) supports the Proposed Rule and states that it can ensure EPA is protecting public health and the environment in a manner that the public can trust and understand. Commenter (6150) states that independence is important to the peer review process and suggests that the peer review process can be improved. Commenters (6180, 8281-PH85) support the use of independent expert peer reviews as an additional level of review for pivotal regulatory science. 
Commenters (6180, 8281-PH85) state that they have been involved in independent peer reviews of various permitting or related activities which have concluded that the technical basis for EPA's permit limits or water quality targets were scientifically indefensible. The commenters assert that, had no peer reviews occurred, these permits or water quality targets would have imposed hundreds of millions of dollars of wastewater treatment costs to rate payers with no anticipated benefit. 
In support of their position that there is a need for independent peer review, commenter (6180) describes a case study involving a Nutrient Criteria document. The commenter provides that this case study highlights the necessity and utility of an independent peer review of EPA regulatory science and raises concerns about EPA's current peer review procedures and charge questions. The commenter notes that it is disconcerting that the findings of an EPA peer review panel of "national experts through the EPA's Nutrient Scientific Technical Exchange Partnership and Support (N-STEPS) program" were entirely contradicted by an independent peer review panel only a few years later. The following is a summary of a case study described by the commenter:
      New Hampshire Department of Environmental Services (DES), with support from EPA Region 1, published the Draft 2009 Numeric Nutrient Criteria for the Great Bay Estuary document. In early 2010, the Draft 2009 Nutrient Criteria underwent review by an EPA peer review panel which validated the scientific methods and decision-making used by DES and praised the use of multiple lines of evidence to develop the criteria as a way to enhance confidence in the results. The Great Bay Municipal Coalition (GBMC) formed in 2010 to contest the 2009 Nutrient Criteria. DES and the GBMC agreed to organize a 2013 independent peer review to review the scientific basis of the 2009 Nutrient Criteria. The peer reviewers concluded that the Draft 2009 Nutrient Criteria were not scientifically defensible. As a result of the 2013 Peer Review conclusions, DES agreed to rescind the 2009 Nutrient Criteria in April 2014.
As further support for their position that there is a need for independent peer review, commenter (6180) states that, historically, more often than not, when the scientific validity of EPA's scientific rationales, methodologies, and derivations has been called into question, even by experts on the specific issue, EPA has refused commenter's requests to conduct independent peer reviews. The commenter describes three examples which are summarized below.
      In 2013, EPA issued a draft permit and fact sheet to the City of Taunton claiming that the City's nitrogen discharge caused or contributed to excessive algal growth and dissolved oxygen (DO) violations. EPA imposed a 3 mg/L (limits of technology) total nitrogen (TN) limit, based on the presumption that nitrogen was a primary factor causing low DO. EPA used a "reference station" approach, without assessing location and site differences that would cause a completely different DO regime and nutrient response to exist at that location. Multiple water quality experts informed EPA that the methodology used to determine the nitrogen limitation was not scientifically defensible. EPA acknowledged that the simplified approach had never been determined to be scientifically defensible but still refused to conduct a peer review of the methodology. Taunton challenged the scientific defensibility of the TN permit limit in multiple venues but the EPA's Environmental Appeals Board (EAB) and the First Circuit Court of Appeals upheld the permit limit derivation and imposition. Both the EAB and the First Circuit also asserted that no cause and effect relationship is required to impose a water quality-based effluent limit. The City requested that the Massachusetts Department of Environmental Protection (MassDEP) reopen the permit based on the updated post-2006 information demonstrating that 1) the assumptions underlying EPA's TN limit derivation were erroneous, and 2) extensive water quality improvements have been documented throughout Narragansett Bay, including Mount Hope Bay (MHB), which would significantly alter, if not completely eliminate, Taunton's TN permit limit. The City's requests have been denied.
      EPA has approved water quality standards for Minnesota that include biochemical oxygen demand (BOD) and DO flux as nutrient impairment indicators. Regarding BOD, Standard Methods, the developer of the test used to measure BOD, published a memorandum explaining that BOD and the BOD test could not be used as a nutrient impairment indicator. Moreover, the response to a July 31, 2014 Freedom of Information Act (FOIA) request to EPA seeking the technical basis for the assertion that DO flux alone can cause an aquatic life impairment indicated that EPA had no supporting documentation. EPA Region 5 conducted an anonymous external review (i.e., reviewers' names are redacted) of the water quality standards in which several reviewers raised serious concerns with the water quality standards and their derivations. This constitutes an inappropriate use of a peer review as 1) it lacks transparency and may incorporate bias as the reviewers remain anonymous, and 2) the concerns of the peer reviewers were not adequately addressed in revisions to the water quality standards. While these issues have been repeatedly raised with both EPA and the Minnesota Pollution Control Agency (MPCA), MPCA has continued applying these BOD and DO flux water quality standards throughout the state. The cascading effect of inappropriately listing waters as nutrient impaired can result in hundreds of millions of dollars wasted on unnecessary Publicly Owned Treatment Works (POTW) upgrades which do not have any measurable environmental benefit. 
      In 2008, EPA Region 3 proposed a group of total maximum daily loads (TMDLs) that regulated total phosphorus. The TMDLs were based on reports prepared by EPA contractor Tetra Tech, Inc. in 2007 (the "Endpoint Report"). The proposed TP endpoint was then used to derive waste load allocations in the various TMDLs that would impose multi-million-dollar costs on the regulated communities. The Endpoint Report was based on a "weight of evidence" evaluation which included novel statistical evaluations. The Endpoint Report was challenged as not scientifically defensible. EPA agreed to the Science Advisory Board (SAB) review and prepared a draft guidance manual (August 17, 2009). The SAB reviewed the draft manual and provided extensive comments on revisions necessary for the guidance to be scientifically defensible (April 27, 2010). Subsequently, EPA revised the draft guidance to address the SAB comments (November 2010). EPA asked Tetra Tech, Inc. to review and revise its 2007 Endpoint Report with consideration for USEPA's new guidance. This follow-up analysis (July 18, 2012) concluded that the original evaluation was supported by the revised evaluation. It should be obvious that the original author of the Endpoint Report should not be the arbiter of a follow-up evaluation to address significant scientific deficiencies of the original report. Even if Tetra Tech, Inc. were allowed to author the revised report, an independent scientific evaluation should have been conducted to assess the scientific defensibility of the revised evaluation. In June 2017, the Southeastern Pennsylvania Nutrient Coalition submitted a request for a peer review to EPA, however, this request was also denied.
Response: EPA appreciates the comments supporting the inclusion of independent peer review in the final rule. As provided in section 30.7, the EPA "shall conduct independent peer review consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review and the exemptions described therein. The EPA will evaluate whether or not to conduct additional peer review of individual studies identified as pivotal science if the studies have already undergone peer review." In this rule, the EPA is not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin state that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary."
The Agency is not responding to comments not directly relevant to the proposed rule. Any concerns about specific projects should be directed to the appropriate EPA programmatic or regional office 
13.1.2.1.2 Do Not Support the Proposed Independent Peer Review Requirements
 Independent Review as Proposed by EPA Would Cause a Delay in EPA Actions
Comment: Commenters (6119, 6133,6137, 6866) express concern regarding EPA's proposed independent peer review requirements because it would be time-consuming and would delay EPA from meeting its responsibilities under statutory deadlines and its mission to protect the public health and environment. Commenter (6137) states that the standard time taken to review scientific manuscripts in the fields of medicine, public health, and natural sciences is, on average, 12-14 weeks. Commenter (6133) states that EPA already misses an unacceptably high number of congressional deadlines in the statutes it administers, and the Proposal to apply peer review to "all pivotal regulatory science" will only exacerbate that endemic problem and the unlawfulness that it represents. Commenter (6119) states that this proposed rule has the potential to create serious technical gaps in the regulatory process and further burden a process that is already years behind in issuing new or updated science-based regulations. Commenter (6866) adds that slowing down rulemaking in ways that may be detrimental to protecting environmental and public health may invite lawsuits that would further stymie regulatory processes.
Response: EPA disagrees that the rule could result in unnecessary delays. EPA clarified in 40 CFR 30.6 that "the EPA will evaluate whether or not to conduct additional peer review of individual studies identified as pivotal science if the studies have already undergone peer review." This should ensure that no unnecessary peer reviews are conducted that could cause undue delay.
Independent Review as Proposed by EPA is Duplicative and/or Unnecessary
Comment: Commenters (2425, 5165, 6119, 6125, 6137, 6142, 6866, 6915) suggest that independent peer review by EPA is duplicative and unnecessary.
Commenter (2425) states that scientific studies used by EPA to make regulatory changes are already rigorously examined prior to being published in peer-reviewed scientific journals. Commenter (6915) states that EPA fails to acknowledge the rigor of existing processes in statutes, policies and federal procedures.
Commenter (6119) states that the process that proposed 40 CFR 30.7 describes is already well established. The commenter states that, for example in December 2009, EPA completed their review of the current research on particle pollution by engaging a panel of expert scientists, and the Clean Air Scientific Advisory Committee, to help EPA assess the evidence. The commenter adds that scientific studies are peer reviewed before being published. According to the commenter, rigorous review practices are already in place for dose response data models used for developing reference values for toxic substances and environmental contaminants, and that federal regulations already set out review requirements for research involving human subjects. 
Commenter (6137) states that, while it is certainly a best practice to consider only science that has been independently peer reviewed when making regulatory decisions, that does not necessitate independent peer review by EPA. The commenter notes that most scientific bodies  -  including Nature, Science, the Bipartisan Policy Center, and Proceedings of the National Academies of Sciences  -  employ some of the most robust peer review practices and that they already apply to the types of studies for which the Proposed Rule will require EPA review. The commenter asserts that the additional review by EPA that is mandated under the proposed rule is duplicative, wholly unnecessary, and inconsistent with the provision in the OMB Bulletin that states that "[p]ublication in a refereed scientific journal may mean that adequate peer review has been performed." 
Commenter (6866) believes that, since most EPA decisions involve evaluating sizable scientific literature on a topic  -  comparing a range of studies  -  routinely re-peer reviewing each study seems particularly duplicative. 
Commenter (6125) states that it is puzzling that section 30.7 is included in the proposed rule, given that external peer review generally already occurs in the cases of significant regulatory actions. The commenter notes that it would appear that the authors of this proposed rule are either not aware of or have never bothered to read the Agency's Peer Review Handbook. The commenter adds that, if they have read it, they chose not to cite it. The commenter provides that the Handbook lays out very clearly the levels of peer review expected to be implemented, depending upon the nature and potential impact of a particular scientific product, including risk assessments. According to the commenter, there is no evidence given in the proposed rule to support that EPA is not following these policies and that this rulemaking is necessary. 
Response: Adequate peer review is peer review that is consistent with the requirements of the OMB Peer Review Bulletin and EPA's Peer Review Handbook. The EPA is finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will conduct independent peer review consistent with the OMB Bulletin for Peer Review and the exemptions described therein. The EPA will evaluate whether or not to conduct additional peer review of individual studies identified as pivotal science if the studies have already undergone journal peer review. In this rule, the EPA is not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble of the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review does not typically include reanalysis of the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information) and, thus, peer review is not considered a replacement for the data availability requirements of this rule. The Agency will continue conducting peer reviews according to the peer review policies and practices that were in place before the rule goes into effect. If implementing this procedural rule would result in conflicts with the guidance found in the Agency's existing Peer Review Handbook, the conflicting requirements found in the Handbook will yield to this final rule.
To address the public comment regarding "unnecessary duplicative and significant costs", the preamble of the final rule specifies, "given the costs of the current robust process for identifying and reviewing scientific studies and documentation that are existing Agency practice, as well as that the determination of dose-response data availability is limited to pivotal science underlying significant regulatory actions and influential scientific information, the EPA anticipates that the incremental costs of this rule will be small....relative to current administrative costs for developing significant regulatory actions or influential scientific information." 
The final rule does not interpret or apply the provisions of any environmental statutes. If implementing this procedural rule would result in conflicts with existing EPA environmental statutes, and their implementing regulations, the procedural rule will yield to the existing statutes and regulations.
Independent Review Would Result in Increased Costs
Comment: Commenters (6119, 6137, 6142, 6866, 6868, 6915) express concern that the proposed independent EPA review requirement would be costly. Commenter (6142) states that this rule proposes adding this costly step to the process for developing EPA policy without setting aside resources for its execution. Commenter (6866) asserts that EPA would be better off having its staff assess study quality as it reviews the scientific literature, as recommended in the Science for Policy Project report, rather than putting in place a blanket requirement for an additional outside peer review of all studies. Commenter (6868) states that the proposed rule seeks to create a supplemental--and likely expensive-- peer review process conducted by agency staff.
Commenter (6119) states that, in reviewing this proposal, it appears EPA will either have to spend millions of dollars to format data to make it publicly accessible for review or limit the total number of studies used to make regulatory decisions because of the costs associated with evaluating them. According to the commenter, for those interested in providing EPA with data for regulatory purposes, new administrative requirements will need to be accounted for and potentially make certain types of studies by EPA impractical. The commenter states that some researchers may choose to avoid certain types of research if it appears likely that administrative hurdles will limit their available funding sources or the application of their findings. 
Commenter (6137) asserts that the proposal would lead to undue cost if EPA were to conduct its own independent peer review of the underlying data for studies that already have undergone a rigorous scientific review process, which is the case for those studies published in scientific journals or independently evaluated by a scientific body. The commenter provides, for example, that the NAAQS Integrated Science Assessment and Risk and Exposure Assessments regularly include the review of thousands of scientific studies. According to the commenter, it would be enormously inefficient and costly (if even possible) for EPA to rereview all of these studies.
Commenter (6915) asserts that any requirement for EPA to conduct additional review would entail additional significant costs, contrary to the proposal's assertion. Id. at 18,772. The practical outcome of the proposal is that EPA may end up relying on a much smaller number of studies and/or on a less robust subset of relevant available studies, thus undermining the regulatory decision-making process. 
Response: The EPA is finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate peer review, consistent with the OMB Bulletin for Peer Review and the EPA's Peer Review Handbook, of individual studies identified as pivotal science if the studies have already undergone journal peer review. In this rule, the EPA is not changing the pre-existing requirements of the OMB's Final Information Quality Bulletin for Peer Review. The preamble of the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review does not typically include reanalysis of the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information) and, thus, peer review is not considered a replacement for the data availability requirements of this rule. If the Agency conducts peer review on pivotal science, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied in analyzing dose-response data and the implications of those assumptions for the results.
This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. The final rule does not require the EPA to collect, store, or publicly disseminate dose-response data and models underlying pivotal science. 
To address the public comment regarding costs, the preamble of the final rule specifies, "given the costs of the current robust process for identifying and reviewing scientific studies and documentation that are existing Agency practice, as well as that the determination of dose-response data availability is limited to pivotal science underlying significant regulatory actions and influential scientific information, the EPA anticipates that the incremental costs of this rule will be small....relative to current administrative costs for developing significant regulatory actions or influential scientific information."
The EPA disagrees that the practical outcome of this rule will be that EPA may end up relying on a much smaller number of studies and/or on a less robust subset of relevant available studies or that the rule will undermine the regulatory decision-making process. The final rule includes sufficient flexibility in terms of the allowance for restricted access, considerations in 40 CFR 30.5, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data to minimize such potential unintended consequences.
Independent Review as Proposed by EPA Has No Rationale/Legal Basis
Lack of Rationale Provided for the Proposed Rule
Comment: Commenters (6137, 6142, 6866, 6868, 6915, 9227) express concern that there is no rationale/basis provided for requiring the independent peer review. Commenter (6142) states that these requirements are likely superfluous in the vast majority of cases since all studies that appear in standard scholarly journals are already peer reviewed by several independent experts prior to acceptance and publication. Commenter (6866) states that this requirement is unnecessary since there is no indication that EPA decision-making has been marred by the use of questionable studies. Commenter (6868) states that, without articulating any deficiencies in the existing scientific peer review process, the proposed rule seeks to create a supplemental peer review process conducted by agency staff. Commenter (6915) states that EPA fails to explain how its proposal would provide any added value. 
Commenter (6137) states that the requirement in proposed §30.7 that EPA independently peer-review all pivotal regulatory science used to make regulatory decisions is arbitrary. The commenter adds that, as part of the Proposed Rule, EPA injects a new requirement without justification. The commenter contends that a review of the support upon which EPA relies demonstrates that this peer review obligation is contrary to current policy. The commenter states that it is also unnecessary, costly, and unrealistic. According to the commenter, it seems the only things this new peer review process would accomplish is the exclusion of reliable science, increased cost and time for effective reviews, and delay of regulatory decisions that impact public health.
Commenter (9227) states that proposed §30.7 generally appears to be designed to enshrine OMB's existing peer review requirements for "influential scientific information." The commenter asserts that the preamble to the proposed rulemaking lacks any explanation whatsoever for why EPA is proposing this new peer review requirement or what its impact might be. The commenter adds that EPA has additionally not provided any information to suggest that EPA is not already following OMB's Peer Review Bulletin. The commenter contends that EPA's lack of any supporting rationale or analysis frustrates the public's ability to provide meaningful comment on this provision, and is itself a sign that this requirement is fundamentally arbitrary. 
Response: EPA disagrees that the rule adds another layer of peer review that could result in unnecessary delays. EPA clarified in 40 CFR 30.6 that "the EPA will evaluate whether or not to conduct additional peer review of individual studies identified as pivotal science if the studies have already undergone peer review." This should ensure that no unnecessary peer reviews are conducted that could cause undue delay. 
The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. Adequate peer review is peer review that is consistent with the requirements of the OMB Peer Review Bulletin and EPA's Peer Review Handbook. The Agency will continue conducting peer reviews according to the peer review policies and practices that were in place before the rule goes into effect. The EPA Peer Review Handbook, 4[th] Edition, describes how peer reviewers are selected by the Agency. As discussed in the final rule preamble, the EPA is not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." If implementing this procedural rule would result in conflicts with the guidance found in the Agency's existing Peer Review Handbook, the conflicting requirements found in the Handbook will yield to this final rule.
Further, EPA is subject to the OMB Bulletin and its provisions have been incorporated into the Agency's peer review handbook (Ref. 44). EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will lead to consistent implementation of those procedures across the Agency.
Reliance on Published Studies Without Conducting Independent Analysis/Review of Underlying Data
Comment: Commenter (6119) states that the courts have spoken clearly on the degree to which independent analyses are necessary. The commenter provides that, in American Trucking Associations, Inc. v. EPA, 283 F.3d 355, 372 (D.C. Cir. 2002), the court said, "we agree with EPA that requiring agencies to obtain and publicize the data underlying all studies on which they rely "would be impractical and unnecessary." The commenter adds that the court then quoted what it called a persuasive EPA argument: "`If EPA and other governmental agencies could not rely on published studies without conducting an independent analysis of the enormous volume of raw data underlying them, then much plainly relevant scientific information would become unavailable to EPA for use in setting standards to protect public health and the environment.... [S]uch data are often the property of scientific investigators and are often not readily available because of ... proprietary interests ... or because of [confidentiality] arrangements [with study participants].'"
Commenter (6125) expresses concern that proposed §30.7 may be trying to revive the argument thoroughly rejected and debunked by the D.C. Circuit in Coalition for Responsible Regulation v. EPA, 684 F. 3d 102 (D.C. Cir. 2012) (finding that substantial evidence supported EPA's finding that emissions of enumerated greenhouse gases endanger public health and welfare and that vehicular emissions contribute to that endangerment). The commenter provides that the court stated that "EPA simply did here what it and other decision-makers often must do to make a science-based judgment: it sought out and reviewed existing scientific evidence to determine whether a particular finding was warranted. It makes no difference that much of the scientific evidence in large part consisted of `syntheses' of individual studies and research. Even individual studies and research papers often synthesize past work in an area and then build upon it; this is how science works. EPA is not required to re-prove the existence of the atom every time it approaches a scientific question." 684 F. 3d at 120. According to the commenter, the proposed §30.7 language shows signs of being designed to `re-prove the existence of the atom' by requiring re-peer review. The commenter states that there is no need for doing so and the need for doing so is not provided in the notice, and that the proposed provision appears both substantively and procedurally defective as a result.
Response: This rule does not categorically exclude any study because the underlying dose-response data are not available, either publicly or via restricted access, for independent validation. If a study is identified as pivotal science, EPA will still consider the study but may give it lesser consideration if the dose-response data are not available for independent validation. Further, the final rule does not require EPA to obtain and publish data underlying the studies it uses to support significant regulatory actions and influential scientific information. The EPA is finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate peer review, consistent with the OMB Bulletin for Peer Review and the EPA Peer Review Handbook, of individual studies identified as pivotal science if the studies have already undergone journal peer review. 

It should also be noted that in this final rule, rather than considering all studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact as "pivotal science," the EPA is balancing transparency and feasibility by focusing on those studies that inform the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect. Thus, "pivotal science" includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). These are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. If peer review under section 30.7 is warranted, it would be conducted on "pivotal science" and not on every dose-response study considered by the Agency in developing a significant regulatory action or influential scientific information.
Lack of Statutory Authority for Proposed §30.7
Comment: Commenter (6133) states that there is no statutory authority for EPA to "conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein." The commenter provides that the federal statutes that EPA lists as putative authority for the Proposal provide no authority for proposed § 30.7. See 83 Fed. Reg. at 18,769/2 (citing CAA sections 103, 301(a), 42 U.S.C. 7403, 7601(a); CWA sections 104, 501, 33 U.S.C. 1254, 1361; SDWA sections 1442, 1450(a)(1), 42 U.S.C. 300j - 1, 300j -  9(a)(1); RCRA sections 2002(a)(1), 7009, 42 U.S.C. 6912(a)(1), 6979; CERCLA (as delegated to the Administrator via Executive Order 12580) sections 115, 311, 42 U.S.C. 9616, 9660; EPCRA section 328, 42 U.S.C. 11048; FIFRA sections 25(a)(1), 136r(a), 7 U.S.C. 136r(a), 136w; and TSCA, as amended, section 10, 15 U.S.C. 2609, and the APA. The commenter states that the claimed authorities that EPA lists do not mention peer review or even allude to the concept. The commenter adds that neither the Proposal nor accompanying docket materials identify any provision of any federal statute that authorizes EPA to promulgate proposed §30.7. According to the commenter, their own research revealed no provision of any federal statute that authorizes EPA to promulgate proposed §30.7.
Commenter (6133) asserts that the Proposal is unlawful, arbitrary and capricious, and an abuse of EPA discretion. The commenter provides that under longstanding federal case law, when Congress authorizes an approach in one section of a statute using specific language but does not do the same in other sections of a statute, the courts presume that Congress acted purposefully and did not mean to address or authorize that approach in those other statutory sections. See, e.g., Dean v. United States, 556 U.S. 568 (2009) ("It is generally presumed that Congress acts intentionally when including particular language in one section of a statute but not in another." (citing Russello v. United States, 464 U.S. 16, 23 (1983)). According to the commenter, not only are there no implied grants of authority to an agency in the other statutory sections, but the congressional decisions to authorize the approach elsewhere in the statute give even greater force to the conclusion that the agency has not been given authority where Congress did not use the same or similar authorizing language.
Commenter (6133) states that the EPA approach proposed in §30.7 is even more unlawful than would be the case, independently, under this case law. The commenter notes that the proposed language of §30.7 says "EPA shall conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein." According to the commenter, the EPA proposes to bind itself ("shall") to conduct a particular form of peer review on "all pivotal regulatory science" based on an unenforceable, non-binding OMB Bulletin that has never been the subject of notice and comment rulemaking and that itself is not authorized by any federal law. The commenter states that treating the content of the unenforceable bulletin as a binding regulation would itself violate the federal statutes that EPA implements, because those statutes do not codify the bulletin, and EPA would be unlawfully codifying a mere policy preference. 
Commenter (6133) contends that the Proposal identifies no statutory authority for EPA to bind itself, and future administrations, to conduct peer review only in this fashion, unless and until future notice-and-comment rulemaking is undertaken. The commenter adds that the Proposal identifies no suggestion in statutory language or legislative history that Congress intended EPA to conduct binding peer review consistent with this OMB Bulletin, notwithstanding that Congress has known about this Bulletin since 2005. The commenter expresses that they believe EPA has simply made up proposed §30.7 -- with its link to the OMB Bulletin, and the putative authority for the Proposal -- out of whole cloth.
Response: The OMB Bulletin provides that "agencies are granted broad discretion to weigh the benefits and costs of using a particular peer review mechanism for a specific information product. The selection of an appropriate peer review mechanism for scientific information is left to the agency's discretion." EPA is subject to the OMB Bulletin and its provisions have been incorporated into the Agency's peer review handbook. EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will lead to consistent implementation of those procedures across the Agency.
As stated in the final rule, EPA is authorized to promulgate this procedural rule under its authority to promulgate housekeeping regulations governing its internal affairs. The final rule does not interpret or apply the provisions of any environmental statutes. If implementing this procedural rule would result in conflicts with existing EPA environmental statutes, and their implementing regulations, the procedural rule will yield to the existing statutes and regulations.
Existing Federal Environmental Statutory Requirements for Peer Review
Comment: Commenter (6915) states that the EPA fails to acknowledge the rigor of existing processes in statutes, policies and federal procedures, or to explain how its proposal would provide any added value and minimize costs. 
Commenter (6133) states that the Proposal's peer review provision in §30.7 is contrary to existing requirements for peer review. The commenter states that, when Congress writes federal statutes, Congress knows how to create legal authority for peer review, who shall conduct that peer review, what role, if any, that EPA or other parties will play, and how that peer review may be conducted. The commenter asserts that none of the provisions in the law authorize EPA's Proposal in §30.7. The commenter provides that the statutes require that EPA use peer-reviewed science, regardless of whether it would meet EPA's definition of the term in §30.7. The commenter notes that, in cases where the law requires EPA to conduct the review, the statutes often spell out specifically how that should happen. The commenter provides the following excerpts from differing federal environmental statutes that establish specific peer review requirements.
CAA
      §7511b. Federal ozone measures.
      (g) Ozone design value study. The Administrator shall conduct a study of whether the methodology in use by the Environmental Protection Agency as of November 15, 1990, for establishing a design value for ozone provides a reasonable indicator of the ozone air quality of ozone nonattainment areas. The Administrator shall obtain input from States, local subdivisions thereof, and others. The study shall be completed and a report submitted to Congress not later than 3 years after November 15, 1990. The results of the study shall be subject to peer and public review before submitting it to Congress.
      §7412. Hazardous air pollutants.
      (p) Mickey Leland National Urban Air Toxics Research Center.
      [...] 
      (3) Scientific Advisory Panel. The Board of Directors shall be advised by a Scientific Advisory Panel, the 13 members of which shall be appointed by the Board, and to include eminent members of the scientific and medical communities. The Panel membership may include scientists with relevant experience from the National Institute of Environmental Health Sciences, the Center for Disease Control, the Environmental Protection Agency, the National Cancer Institute, and others, and the Panel shall conduct peer review and evaluate research results. The Panel shall assist the Board in developing the research agenda, reviewing proposals and applications, and advise on the awarding of research grants.
CWA
      §1321. Oil and hazardous substance liability.
      (a) Definitions. (27) the term "best available science" means science that-- (A) maximizes the quality, objectivity, and integrity of information, including statistical information; (B) uses peer-reviewed and publicly available data; and (C) clearly documents and communicates risks and uncertainties in the scientific basis for such projects.
SDWA
      §300g-1. National drinking water regulations.
      (b) Standards.
      [...]
      (3) Risk assessment, management, and communication.
      (A) Use of science in decision-making In carrying out this section, and, to the degree that an Agency action is based on science, the Administrator shall use--
      (i) the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices; and(ii) data collected by accepted methods or best available methods (if the reliability of the method and the nature of the decision justifies use of the data).
      (B) Public information.
      In carrying out this section, the Administrator shall ensure that the presentation of information on public health effects is comprehensive, informative, and understandable. The Administrator shall, in a document made available to the public in support of a regulation promulgated under this section, specify, to the extent practicable--
      (i) each population addressed by any estimate of public health effects;
      (ii) the expected risk or central estimate of risk for the specific populations;
      (iii) each appropriate upper-bound or lower-bound estimate of risk;
      (iv) each significant uncertainty identified in the process of the assessment of public health effects and studies that would assist in resolving the uncertainty; and
      (v) peer-reviewed studies known to the Administrator that support, are directly relevant to, or fail to support any estimate of public health effects and the methodology used to reconcile inconsistencies in the scientific data.
      [...]
      (12) Certain contaminants.
      (B) Sulfate.
      (i) Additional study. Prior to promulgating a national primary drinking water regulation for sulfate, the Administrator and the Director of the Centers for Disease Control and Prevention shall jointly conduct an additional study to establish a reliable dose-response relationship for the adverse human health effects that may result from exposure to sulfate in drinking water, including the health effects that may be experienced by groups within the general population (including infants and travelers) that are potentially at greater risk of adverse health effects as the result of such exposure. The study shall be conducted in consultation with interested States, shall be based on the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices, and shall be completed not later than 30 months after August 6, 1996.
      §300j-2. Grants for State programs.
      (d) New York City watershed protection program.
      (1) In general. The Administrator is authorized to provide financial assistance to the State of New York for demonstration projects implemented as part of the watershed program for the protection and enhancement of the quality of source waters of the New York City water supply system, including projects that demonstrate, assess, or provide for comprehensive monitoring and surveillance and projects necessary to comply with the criteria for avoiding filtration contained in 40 C.F.R. 141.71. Demonstration projects which shall be eligible for financial assistance shall be certified to the Administrator by the State of New York as satisfying the purposes of this subsection. In certifying projects to the Administrator, the State of New York shall give priority to monitoring projects that have undergone peer review.
RCRA
      §6939a. Exposure information and health assessments.
      (b) Health assessments.
      [...]
      (2) Whenever in the judgment of the Administrator, or the State (in the case of a State with an authorized program), a landfill or a surface impoundment poses a substantial potential risk to human health, due to the existence of releases of hazardous constituents, the magnitude of contamination with hazardous constituents which may be the result of a release, or the magnitude of the population exposed to such release or contamination, the Administrator or the State (with the concurrence of the Administrator) may request the Administrator of the Agency for Toxic Substances and Disease Registry to conduct a health assessment in connection with such facility and take other appropriate action with respect to such risks as authorized by section 9604(b) and (i) of this title. If funds are provided in connection with such request the Administrator of such Agency shall conduct such health assessment.
      [...]
      (e) Periodic reports. The Administrator of such Agency shall issue periodic reports which include the results of all the assessments carried out under this section. Such assessments or other activities shall be reported after appropriate peer review.
CERCLA
      §9604. Response authorities.
      (i) Agency for Toxic Substances and Disease Registry; establishment, functions, etc.
      [...]
      Any toxicological profile or revision thereof shall reflect the Administrator of ATSDR's assessment of all relevant toxicological testing which has been peer reviewed. The profiles required to be prepared under this paragraph for those hazardous substances listed under subparagraph (A) of paragraph (2) shall be completed, at a rate of no fewer than 25 per year, within 4 years after October 17, 1986. A profile required on a substance listed pursuant to subparagraph (B) of paragraph (2) shall be completed within 3 years after addition to the list. The profiles prepared under this paragraph shall be of those substances highest on the list of priorities under paragraph (2) for which profiles have not previously been prepared. Profiles required under this paragraph shall be revised and republished as necessary, but no less often than once every 3 years. Such profiles shall be provided to the States and made available to other interested parties.
      [...]
      (7)(A) Whenever in the judgment of the Administrator of ATSDR it is appropriate on the basis of the results of a health assessment, the Administrator of ATSDR shall conduct a pilot study of health effects for selected groups of exposed individuals in order to determine the desirability of conducting full scale epidemiological or other health studies of the entire exposed population. (B) Whenever in the judgment of the Administrator of ATSDR it is appropriate on the basis of the results of such pilot study or other study or health assessment, the Administrator of ATSDR shall conduct such full scale epidemiological or other health studies as may be necessary to determine the health effects on the population exposed to hazardous substances from a release or threatened release. If a significant excess of disease in a population is identified, the letter of transmittal of such study shall include an assessment of other risk factors, other than a release, that may, in the judgment of the peer review group, be associated with such disease, if such risk factors were not taken into account in the design or conduct of the study.
      [...]
      (13) All studies and results of research conducted under this subsection (other than health assessments) shall be reported or adopted only after appropriate peer review. Such peer review shall be completed, to the maximum extent practicable, within a period of 60 days. In the case of research conducted under the National Toxicology Program, such peer review may be conducted by the Board of Scientific Counselors. In the case of other research, such peer review shall be conducted by panels consisting of no less than three nor more than seven members, who shall be disinterested scientific experts selected for such purpose by the Administrator of ATSDR or the Administrator of EPA, as appropriate, on the basis of their reputation for scientific objectivity and the lack of institutional ties with any person involved in the conduct of the study or research under review. Support services for such panels shall be provided by the Agency for Toxic Substances and Disease Registry, or by the Environmental Protection Agency, as appropriate.
FIFRA
      §136w. Authority of Administrator.
      (e) Peer review. The Administrator shall, by written procedures, provide for peer review with respect to the design, protocols, and conduct of major scientific studies conducted under this subchapter by the Environmental Protection Agency or by any other Federal agency, any State or political subdivision thereof, or any institution or individual under grant, contract, or cooperative agreement from or with the Environmental Protection Agency. In such procedures, the Administrator shall also provide for peer review, using the advisory panel established under subsection (d) of this section or appropriate experts appointed by the Administrator from a current list of nominees maintained by such panel, with respect to the results of any such scientific studies relied upon by the Administrator with respect to actions the Administrator may take relating to the change in classification, suspension, or cancellation of a pesticide. Whenever the Administrator determines that circumstances do not permit the peer review of the results of any such scientific study prior to the Administrator's exercising authority under section 136d(c) of this title to immediately suspend the registration of any pesticide to prevent an imminent hazard, the Administrator shall promptly thereafter provide for the conduct of peer review as provided in this sentence. The evaluations and relevant documentation constituting the peer review that relate to the proposed scientific studies and the results of the completed scientific studies shall be included in the submission for comment forwarded by the Administrator to the advisory panel as provided in subsection (d). As used in this subsection, the term "peer review" shall mean an independent evaluation by scientific experts, either within or outside the Environmental Protection Agency, in the appropriate disciplines.
      §136w-8. Pesticide registration service fees.
      (a) Definition of costs. In this section, the term "costs", when used with respect to review and decision-making pertaining to an application for which registration service fees are paid under this section, means-- (1) costs to the extent that -- (A) officers and employees provide direct support for the review and decision-making for covered pesticide applications, associated tolerances, and corresponding risk and benefits information and analyses; (B) persons and organizations under contract with the Administrator engage in the review of the applications, and corresponding risk and benefits information and assessments; and (C) advisory committees and other accredited persons or organizations, on the request of the Administrator, engage in the peer review of risk or benefits information associated with covered pesticide applications; (2) costs of management of information, and the acquisition, maintenance, and repair of computer and telecommunication resources (including software), used to support review of pesticide applications, associated tolerances, and corresponding risk and benefits information and analyses; and (3) costs of collecting registration service fees under subsections (b) and (c) and reporting, auditing, and accounting under this section.
TSCA
      §2625. Administration.
      (h) Scientific standards. In carrying out sections 2603, 2604, and 2605 of this title, to the extent that the Administrator makes a decision based on science, the Administrator shall use scientific information, technical procedures, measures, methods, protocols, methodologies, or models, employed in a manner consistent with the best available science, and shall consider as applicable-- (1) the extent to which the scientific information, technical procedures, measures, methods, protocols, methodologies, or models employed to generate the information are reasonable for and consistent with the intended use of the information; (2) the extent to which the information is relevant for the Administrator's use in making a decision about a chemical substance or mixture; (3) the degree of clarity and completeness with which the data, assumptions, methods, quality assurance, and analyses employed to generate the information are documented; (4) the extent to which the variability and uncertainty in the information, or in the procedures, measures, methods, protocols, methodologies, or models, are evaluated and characterized; and (5) the extent of independent verification or peer review of the information or of the procedures, measures, methods, protocols, methodologies, or models.
      §2617. Preemption.
      (b) New statutes, criminal penalties, or administrative actions creating prohibitions or other restrictions. (1) In general Except as provided in subsections (c), (d), (e), (f), and (g), beginning on the date on which the Administrator defines the scope of a risk evaluation for a chemical substance under section 2605(b)(4)(D) of this title and ending on the date on which the deadline established pursuant to section 2605(b)(4)(G) of this title for completion of the risk evaluation expires, or on the date on which the Administrator publishes the risk evaluation under section 2605(b)(4)(C) of this title, whichever is earlier, no State or political subdivision of a State may establish a statute, criminal penalty, or administrative action prohibiting or otherwise restricting the manufacture, processing, distribution in commerce, or use of such chemical substance that is a high-priority substance designated under section 2605(b)(1)(B)(i) of this title.
      [...]
      (f) Waivers. 
      [...]
      (2) Required exemptions. Upon application of a State or political subdivision of a State, the Administrator shall exempt from subsection (b) a statute or administrative action of a State or political subdivision of a State that relates to the effects of exposure to a chemical substance under the conditions of use if the Administrator determines that-- (A)(i) compliance with the proposed requirement of the State or political subdivision of the State would not unduly burden interstate commerce in the manufacture, processing, distribution in commerce, or use of a chemical substance; (ii) compliance with the proposed requirement of the State or political subdivision of the State would not cause a violation of any applicable Federal law, rule, or order; and (iii) the State or political subdivision of the State has a concern about the chemical substance or use of the chemical substance based in peer-reviewed science; or (B) no later than the date that is 18 months after the date on which the Administrator has initiated the prioritization process for a chemical substance under the rule promulgated pursuant to section 2605(b)(1)(A) of this title, or the date on which the Administrator publishes the scope of the risk evaluation for a chemical substance under section 2605(b)(4)(D) of this title, whichever is sooner, the State or political subdivision of the State has enacted a statute or proposed or finalized an administrative action intended to prohibit or otherwise restrict the manufacture, processing, distribution in commerce, or use of the chemical substance.
      §2605. Prioritization, risk evaluation, and regulation of chemical substances and mixtures. 
      (b) Risk evaluations. 
      [...]
      (E) Metals and metal compounds. In identifying priorities for risk evaluation and conducting risk evaluations of metals and metal compounds, the Administrator shall use the Framework for Metals Risk Assessment of the Office of the Science Advisor, Risk Assessment Forum, and dated March 2007, or a successor document that addresses metals risk assessment and is peer reviewed by the Science Advisory Board.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. Because this is purely a procedural rule, the EPA is not relying on any substantive environmental statutes as authority. If implementing this procedural rule would result in conflicts with existing environmental statutes and their implementing regulations, the procedural rule will yield to the existing statutes and regulations. This rulemaking is not intended to modify the Agency's interpretations of "best available science." The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. Further, the EPA plans to promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
To address the public comment regarding "added value and minimize cost", the preamble of the final rule specifies, "given the costs of the current robust process for identifying and reviewing scientific studies and documentation that are existing Agency practice, as well as that the determination of dose-response data availability is limited to pivotal science underlying significant regulatory actions and influential scientific information, the EPA anticipates that the incremental costs of this rule will be small relative to current administrative costs for developing significant regulatory actions or influential scientific information."
Unlawfully Vague
Comment: Commenters (4838, 6133) suggest that the Proposed §30.7 language is unlawfully vague. The commenters assert that EPA did not provide an explanation or accompanying regulatory text about what §30.7 means. Commenter (4838) states that, for example, it is unclear whether 40 CFR 30.7 requires EPA to conduct its own peer review of all pivotal regulatory science and, if so, whether EPA has the capacity or capability to perform those reviews. Commenter (6133) expresses concern that the Proposal provides no answers to the following questions:
          How will that peer review be conducted? 
          By whom? 
          Who will select the peer reviewers? 
          How many will there be? 
          Who will assure their independence and expertise? 
          Will peer reviewers be subject to federal conflict of interest rules and policies? 
          Will peer reviewers be anonymous? 
          Where will the funds come from to conduct EPA peer reviews for "all pivotal regulatory science"? 
          Has EPA estimated how many instances of "pivotal regulatory science" it anticipates conducting peer review for in one year? In prior years? 
          Will the peer review be conducted openly and publicly? 
          Will it be conducted in accordance with the Federal Advisory Committee Act? 
          What will the duration of any peer review be? 
          What purpose will that peer review serve? 
          How will it affect future regulatory decisions? Or will it? 
          Will there be an administrative docket? 
          Will any product of the peer review be included in the administrative dockets for rulemaking? 
          Will peer reviewer comments be part of the certified record for judicial review? 
          Will the agency seek deference from future reviewing courts for the views expressed by peer reviewers? 
          Does EPA not believe that peer review conducted by professional journals and societies is valid? Or sufficient? 
          On what basis does EPA think professional peer review is invalid or insufficient, considering there is not one iota of evidence or support for that belief in the Proposal or the accompanying docket? 
          What is the basic justification for proposed § 30.7?
Response: The EPA is finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate peer review, consistent with the OMB Bulletin for Peer Review and the EPA's Peer Review Handbook, of individual studies identified as pivotal science if the studies have already undergone journal peer review. 
The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. Adequate peer review is peer review that is consistent with the requirements of the OMB Peer Review Bulletin and EPA's Peer Review Handbook. The Agency will continue conducting peer reviews according to the peer review policies and practices that were in place before the rule goes into effect. The EPA Peer Review Handbook, 4[th] Edition, describes how peer reviewers are selected by the Agency. As discussed in the final rule preamble, the EPA is not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." If implementing this procedural rule would result in conflicts with the guidance found in the Agency's existing Peer Review Handbook, the conflicting requirements found in the Handbook will yield to this final rule.
Censoring of Peer Review Studies/Research and Data
Comment: Commenter (2169) states that the pragmatic impact of rule §30.7 is straight forward: According to the commenter, the EPA proposes to set itself up as the ultimate arbiter of science and, effectively, censor all future peer review studies conducted by the specialists in a given area. The commenter asserts that the reality is that science, not subject to the suffocating and chilling overhang of governmental censorship, is far more likely to develop a robust protocol for enhancing and assessing reproducibility than is a government agency subject to the fickle influence of private industry and its paid representatives who are in the business of producing doubt with respect to any proposal that implies increased cost to, or potentially diminished market share for, the industry affected by a particular EPA proposal.
Commenter (6111) states that it is contrary to good scientific study and practice and the advancement of knowledge for EPA to arrogate to itself the determination of what constitutes useable research and data, and to grant sweeping discretion to the Administrator -- who may not even be a scientist -- to make those determinations. Commenter states that the proposed rule was announced by EPA without any meaningful consultation with the broad research community despite the fact that it addresses a complex and contentious issue that is not yet ripe for regulatory action. 
Response: The EPA is committed to its mission of protecting human health and the environment through sound policy decisions that are informed by robust scientific and technical research. Because of the potential impact of the EPA's significant regulatory actions and influential scientific information on American lives and livelihoods, the American people deserve environmental decisions and policies that are based on the best scientific information. Only through continuous improvement to its procedures, especially those focused on transparency, can the EPA fully demonstrate that commitment.
The final rule does not categorically exclude studies based on the availability of the underlying dose-response data. The EPA would also like to clarify that the final rule does not require the EPA to collect, store, or publicly disseminate data and models underlying pivotal science. The EPA would also like to clarify that the Agency is not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary."Peer review does not typically include reanalysis of the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information) and, thus, peer review is not considered a replacement for the data availability requirements of this rule. 
The EPA is finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate peer review, consistent with the OMB Bulletin for Peer Review and the EPA's Peer Review Handbook, of individual studies identified as pivotal science if the studies have already undergone journal peer review. The EPA Peer Review Handbook, 4th Edition, describes the process for peer reviews conducted by a Federal Advisory Committee and the National Academy of Sciences. If the Agency conducts peer review on pivotal science, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied in analyzing dose-response data and the implications of those assumptions for the results.
Comment: Commenter (2475) expresses concern that the Administrator could claim there is no budget to support independent peer review, making it infeasible, and justify an exemption.
Response: The EPA is not finalizing the criterion for an Administrator's exemption related to the infeasibility of conducting independent peer review on all pivotal regulatory science used to justify regulatory decisions for reasons outlined in OMB Final Information Quality Bulletin for Peer Review. The EPA is also clarifying that the Agency is not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review does not typically include reanalysis of the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information) and, thus, peer review is not considered a replacement for the data availability requirements of this rule. 
The EPA is finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate peer review, consistent with the OMB Bulletin for Peer Review and the EPA's Peer Review Handbook, of individual studies identified as pivotal science if the studies have already undergone journal peer review. The EPA Peer Review Handbook, 4th Edition, describes the process for peer reviews conducted by a Federal Advisory Committee and the National Academy of Sciences. If the Agency conducts peer review on pivotal science, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied in analyzing dose-response data and the implications of those assumptions for the results. The considerations in 40 CFR 30.5, the Administrator's exemption in 40 CFR 30.7, and the EPA's incremental approach to maximizing transparency by first focusing on dose-response data are intended to minimize potential unintended consequences, such as the potential skew toward newer, domestic, and/or industry-funded studies suggested by the commenters. In addition, the EPA is maintaining language from the 2020 SNPRM in 40 CFR 30.3 stating that statutory requirements and corresponding implementing regulations will control in the event of any conflict, and clarifying in this preamble that the requirements in this final rule set the overarching structure and principles for transparency in significant regulatory actions and influential scientific information. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. The EPA believes the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making.
13.1.2.1.3 Process of Independent Peer Review
 Recommendations for a Process to Conduct Independent Peer Review
Comment: Commenter (4420) provides the following detailed recommendations and rationale for a process to conduct the independent peer review.
      A Call for the Standardization of the Review of an Epidemiology Studies  -  Establishment of an Agency wide Epidemiology Study Peer Review Council and their product Record of Epidemiological Study Review (ROESR).
      Each epidemiology study occurring in the open literature or otherwise needs to have a supporting formal Record of Epidemiological Study Review (ROESR) that clearly delineates the justification for a decision to include or not include the study in the regulatory decisions for the chemical or environmental situation. The production of the ROESR will consist of the separate and independent reports of several Sub-Committees that will be integrated by the Epidemiological Study Peer Review Council and signed by the Council Chairperson and each member of the Council. It is important to have independent sub-discipline committees review the study so that the biases of other sub-disciplines are minimized. The committee members should be drawn from the relevant staff throughout the Agency as well as other government agencies as needed.
      The roles of the five suggested Sub-Committees are as follows:
      Endpoint Evaluation Sub-Committee: Depending upon the lesion/ environmental condition claimed by the epidemiological study, the endpoint evaluation committee would be experts in cancer, behavioral response, or other appropriate discipline for interpretation of the significance of the lesion. This committee need not actually see the supporting data since they will be judging the endpoint itself relative to the plausibility that the lesion is actually in excess of the norms and that all confounds are addressed as well as consistency reported for the subjects in the cohort. Commentary on any known chemicals or environmental factors that also promote the occurrence of the lesion/condition will also be provided. This evaluation would limit its conclusions with regard to how well the character of the endpoint was assessed for and whether or not the increase in the lesion reported is plausibly related to treatment or within normal variation. This Sub-Committee may require additional input from the study authors regarding how the endpoint was characterized in all subjects.
      Statistical Evaluation Sub-Committee: The sole role of the statistical evaluation sub-committee is limited to determining if the statistical methods were/were not appropriate and adequately conducted to support the conclusion of the report. A committee of statisticians with expertise in epidemiological studies would need to be established for this evaluation. This Sub-Committee would be very critical in determining if the original data are needed to do an independent analysis.
      Exposure Assessment Sub-Committee: The role of the exposure assessment committee is limited to determining that the methodology and reliability used to determine the exposure of the cohort is adequate and appropriate. The committee would provide commentary on whether or not exposure was supported by analytical chemical data or by oral history. Also, if actual persons exposed provided the oral history by direct conversation, survey or by telephone. The committee will determine if second-hand exposure information by a relative (or friend or coworker) provided the exposure information is reliable. The exposure assessment is always a problematic area and may also be critical in a decision to provide supporting data.
      Analytical Chemistry Assessment Sub-Committee: The role of the analytical chemistry committee is limited to determining if the analytical techniques/ methodology were appropriate and adequate for the study. In some cases where no analytical chemistry data were generated, the committee will comment on the need to have included such data and/or if quantitative analysis of exposure data was even possible. Since the analytical chemistry data such as sample preparation, method of detection, and lower limit of detection are generic, there should be no issue for the study authors to provide this information. However, this Sub-Committee may still determine that analytical data from the subjects in the cohort be provided. 
      Animal Toxicity Assessor or Sub-Committee: An actual Sub-Committee may not be necessary for this evaluation since other committees should already have characterized the toxicity profile based on animal studies. For example, for a pesticide, this assessor would normally be the person most responsible for reviewing the guideline studies or other required studies. If not then a committee that reviews the animal study data may need to be convened. This assessment will also include in addition to the standard set of required studies, reports from the open literature suggesting the association of the lesion with the chemical in animals as well as structure activity relationships (SAR). However, any publication supporting the association or mechanism of action will need its own Record of Review (ROR) signed by two qualified toxicologists. If the lesion occurs in animals or has SAR correlates, such information can support the epidemiological findings. If not, it does not automatically dismiss the epidemiological findings since the induction of the lesion may be unique to humans. There should be no need for this Sub-Committee to require additional data from the epidemiological report.
      Council Chairperson and resulting product: ROESR: Once the five Sub-Committees finish their respective reports, they meet together with the Council members to discuss their findings and conclusions. Each Sub-Committee will assert if additional or original data supporting the study are required or not. The Council Chairperson, with the approval of the Council, makes the final determination in deciding if the epidemiological study provides quality data that can or should not be incorporated into the regulatory decisions. The Council Chairperson with the Council's approval can overrule a request for additional data made by any of the Sub-Committees. However, clear justification for overruling a Sub-Committee's decisions must be provided. For example, the Council may determine that the association between the chemical or environmental condition is otherwise strong enough to support regulatory action to protect the public health. The Council Chairperson (or secretary of the Council) will prepare the ROESR product that contains the decision with supporting justification. The ROESR product would include the report of the Council with the Chairperson and Council members signatures. Separate attachments for the signed Sub-Committee reports and the original epidemiological report (i.e. a publication) will also be attached. The Council Chairperson should not be in the same program that has responsibility for regulating the chemical or environmental condition. That is, if the chemical is a pesticide, the Chairperson cannot be from the Office of Pesticide Programs (OPP). Representatives of OPP may still be on the Council if an epidemiological study with a pesticide is being reviewed.
      The purpose of this procedure is to assure that responsible persons for each of the critical sub-disciplines independently reviewed the study in terms of their expertise. The resulting ROESR that either determines that the study can be used in the absence of additional original data or the need for critical additional data is determined will be clearly indicated. The Council Chairperson and Council and the sub-discipline committees own their decisions and are responsible for defending them. If industry or any concerned public interest groups object to the conclusions, they can address the conclusions presented in the ROESR and/or the sub-disciplines. Therefore, the decisions in the resulting ROESR should be transparent. The current system renders too much power to individuals that are sometimes not clearly identified who can profit by making career projects for themselves based on the face value of the publication.
      Preparing a ROESR for such studies with the several sub-disciplines all independently contributing to the Agency's determination of the conclusions in the report should greatly help to minimize controversies. The ROESR is not a final conclusion (no documents in EPA should be considered final). It is very likely that a Science Advisory Panel will ultimately review the ROESR. Also, as new information is generated, the ROESR can be updated and the recommendations for use (or otherwise) can be adjusted as appropriate.
Response: EPA appreciates this recommendation. However, EPA does not agree that it needs to establish this type of process. The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. All Agency peer review follows the recommended procedures and approaches consistent with the requirements of the OMB Peer Review Bulletin and EPA's Peer Review Handbook, 4[th] Edition. The Agency will continue conducting peer reviews according to the peer review policies and practices that were in place before the rule goes into effect.
Comment: Commenter (4594) recommends that EPA partner with scientific organizations and professional communities to administer and manage these reviews. The commenter states that such outsourcing and partnerships will help to ensure that EPA gains access to independent and highly qualified experts; and will promote greater public confidence in the independence of these peer-reviews. The commenter notes that this kind of process for managing peer-review will also allow EPA to more cost-effectively, nimbly, and rapidly conduct reviews as it will not require EPA to substantially increase staffing for convening reviews. According to the commenter, such a process will also provide EPA with greater capacity to conduct reviews on timescales that do not needlessly delay regulatory and rulemaking schedules. 
Response: Thank you for the recommendation. However, EPA does not agree that it should partner with scientific organizations and professional communities to administer and manage the reviews under this rule. The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. This rulemaking is not intended to modify the Agency's interpretations of "best available science." The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. All Agency peer review follows the recommended procedures and approaches consistent with the requirements of the OMB Peer Review Bulletin and EPA's Peer Review Handbook, 4th Edition. The Agency will continue conducting peer reviews according to the peer review policies and practices that were in place before the rule goes into effect. Further, the EPA plans to promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
Comment: Commenter (6180) supports formal peer reviews, but also alternatively supports rigorous, detailed, and transparent development of pivotal regulatory science which takes into account public comments and input from a wide range of experts and stakeholders. As an example, the commenter refers to EPA's 1991 publication of the Technical Support Document (TSD) for Water Quality-based Toxics Control. The commenter provides that, while no formal peer review took place, the TSD underwent a rigorous and detailed multi-year review by "a diverse group of people from EPA Regions, states, environmental groups, trade associations, academia, private industry, and municipalities." 
Response: Thank you for the recommendation. All Agency technical work products go through rigorous internal review and, when appropriate, peer review. The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. In the final rule, the EPA shall conduct independent peer review consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review and the exemptions described therein. All Agency peer review follows the recommended procedures and approaches found in the EPA's Peer Review Handbook [4[th] Edition]." The Agency will continue conducting peer reviews according to the peer review policies and practices that were in place before the rule goes into effect. Further, the EPA plans to promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. 
Comment: Commenters (6376, 6891) suggest that there should and can be a balance between transparency and the ability to draw from peer-reviewed scientific studies from outside sources. 
Commenter (6891) advises EPA to recognize, as it has in the past, that a variety of quality assurance and quality control measures can result in properly validated data that is appropriate for use in justifying regulatory decisions, not just that which has been independently peer reviewed. The commenter recommends that EPA adopt a broader view of what is necessary to ensure validity, rather than require independent peer review in all instances. 
Commenter (6376) states that, while EPA needs to evaluate and judge the merits of different data sets, it is impractical to expect the Agency to supplant the role of peer-review. The commenter notes that the Agency is explicit in their intention to rely on internal review of the scientific studies used to make substantial regulatory decisions. The commenter expresses that, theoretically, this practice makes sense for the EPA, but it could be problematic in application. The commenter provides that there is an ongoing downsizing process across the federal government and the EPA, in particular, is currently executing its mission with fewer staff members and is planning on decreasing those numbers even more in the coming years. According to the commenter, practically speaking, this means substantive internal review of the scientific studies used to inform "significant regulatory decisions" could be unrealistic based on the Agency's workforce. The commenter asserts that the ability to utilize peer-reviewed studies needs to be maintained as EPA moves towards creating more transparency. 
Response: All Agency technical work products go through rigorous internal review and, when appropriate, peer review. The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. In the final rule, the EPA shall conduct independent peer review consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review and the exemptions described therein. All Agency peer review follows the recommended procedures and approaches found in the EPA's Peer Review Handbook [4[th] Edition]." The Agency will continue conducting peer reviews according to the peer review policies and practices that were in place before the rule goes into effect. Further, the EPA plans to promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
The EPA is finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate peer review, consistent with the OMB Bulletin for Peer Review and the EPA's Peer Review Handbook, of individual studies identified as pivotal science if the studies have already undergone journal peer review. Although not exhaustive or exclusive, the EPA has identified several factors that balance some of the important technical considerations the EPA will consider in addition to data availability and that are particularly relevant to the stage of the analysis where dose-response data are used. The EPA will thus consider the following factors:
 the quality of the study relative to other studies for which the dose-response data are available; 
 the extent to which there are other studies for which the dose-response data are available;
 the sensitivity of the conclusions in the significant regulatory action or influential scientific information based on the use of the study;
 the extent to which the study is fit for the purpose or intended use relative to other studies for which the dose-response data are available;
 the use of exposures or doses in a range and duration that is relevant for the intended use and that minimizes the need for extrapolations;
 the extent to which the study is supported by other scientific evidence;
 the extent to which the study accounts for unique scientific considerations;
 the extent to which the study minimizes the use of defaults and assumptions, uses appropriate and strong statistical methods, and includes a robust representation of uncertainty and confidence intervals; and
 the study's consideration of a broad range of parametric dose-response or concentration-response models, a robust set of potential confounding variables, nonparametric models that incorporate fewer assumptions, various threshold models across the dose or exposure range, and models that investigate factors that might account for spatial heterogeneity. 
Clarification of the Independent Peer Review Process is Needed
Comment: Commenters (4594, 5165, 5419, 6125, 6142, 6185, 6376, 6868, 6915, 6935) express concern that the proposed language of §30.7 is accompanied by inadequate explanation, leaving commenters to speculate about how it would apply, or what problems its implementation might present. Commenter (6142) states that there is no value in this second peer review without guidelines ensuring the review is rigorous and unbiased in nature and conducted by subject matter experts. Commenter (6935) suggests that the regulatory language in the proposed language of §30.7 should be expanded to address these issues. Commenter (6376) believes the proposal would benefit from a clearly outlined process for utilizing peer-reviewed science to ensure regulatory decisions can be based on the best available science. The commenters request clarification and additional detail related to the following questions:
 Who will conduct and manage the peer-review process? (4594, 6142)
 Will these reviews be managed by the Office of Research and Development or by the various regulatory offices within EPA? (4594)
 Does EPA have appropriate staffing, expertise, and resources to manage these peer-reviews? (4594)
 Would EPA need to hire new experts, how many, in what fields and how much would this cost? (5165)
 What are the selection criteria for peer reviewers? (5419)
 What are the decision criteria? (5419)
 How the proposed rule compares to existing processes through the SAB and third‐party contractor‐managed peer reviews? (5419)
 What scientific standards will be applied? (6142)
 How transparent will the evaluation will be to the study's authors, the scientific community, and the public? (6142)
 Would EPA's staff scientists conduct these validations and reviews, or would EPA contract this out to third parties? (6185)
 How would EPA determine that third parties have the necessary qualifications and resources to perform these validations and reviews? (6185)
 Would peer reviewers be anonymous, or will their names and reviews be made public? (6185)
 How will an EPA agency "independent peer review" process be funded? (6868)
 How will EPA vet reviewers to identify persons who are purportedly more competent than those already used in past or current peer review processes? (6915) 
 What is the relationship between EPA's existing science advisors and the new requirement for independent peer review? (6935) 
 If the new independent peer review process will be supported by new external advisors, how will such advisors be identified and selected, and will such a process will be used for each regulatory action on an ad hoc basis? (6935) 
Commenter (6915) notes that despite the existing peer review process, EPA apparently proposes to require that EPA itself conduct an additional "independent" review. See 83 Fed. Reg. at 18,774. Yet the commenter states that the proposal nowhere discusses how EPA would vet reviewers to identify persons who are purportedly more competent than those already used in past or current peer review processes, or the level of EPA staffing and associated costs that would be needed for additional review -- only stating that EPA will implement the proposed rule in a manner "that minimizes costs." Id. at 18,774. 
Commenter (6185) suggests that EPA needs to describe how the proposed text in §30.7 would affect current practice. The commenter asserts that, without this information, commenters are left to guess at the scope and potential impact of EPA's proposal as it applies to "independent review."
Response: In this rule, the EPA is not changing the pre-existing requirements of the OMB's Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review does not typically include reanalysis of the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information) and, thus, peer review is not considered a replacement for the data availability requirements of this rule. 
The EPA is finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate peer review, consistent with the OMB Bulletin for Peer Review and the EPA's Peer Review Handbook, of individual studies identified as pivotal science if the studies have already undergone journal peer review. The EPA Peer Review Handbook, 4th Edition, describes the process for peer reviews conducted by a Federal Advisory Committee and the National Academy of Sciences. All Agency peer review follows the recommended procedures and approaches found in the EPA's Peer Review Handbook. This Peer Review Handbook is used as guidance by EPA staff and managers to ensure that the Agency's Peer Review Policy is implemented objectively and that the integrity of our peer review activities can be demonstrated transparently to the American public. If the Agency conducts peer review on pivotal science, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied in analyzing dose-response data and the implications of those assumptions for the results.
It should also be noted that in this final rule, rather than considering all studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact as "pivotal science," the EPA is balancing transparency and feasibility by focusing on those studies that inform the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect. Thus, "pivotal science" includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). These are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. 
13.1.2.1.4 Other Comments
Comment: Commenter (0045) recommends that deliberations of the EPA include enough time for peer review of all sources to proceed before consideration and action. The commenter states that the ability of all constituents to participate in EPA activity comes with a commitment for an open and respectful process which will maintain the quality of all information used to make decisions at the EPA.
Response: This rulemaking is not intended to modify the Agency's interpretations of "best available science." The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. All Agency peer review follows the recommended procedures and approaches consistent with the requirements of the OMB Peer Review Bulletin and EPA's Peer Review Handbook, 4th Edition. The Agency will continue conducting peer reviews according to the peer review policies and practices that were in place before the rule goes into effect. The EPA is finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate peer review, consistent with the OMB Bulletin for Peer Review and the EPA's Peer Review Handbook, of individual studies identified as pivotal science if the studies have already undergone journal peer review. Further, the EPA plans to promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
Comment: Commenter (6143) suggests that EPA replace the proposed text: "EPA shall conduct independent peer review . . ." with: "EPA shall conduct and thoroughly document independent peer review . . ."
Response: EPA appreciates this recommendation. The EPA maintains that all Agency peer review follows the recommended procedures and approaches consistent with the requirements of the OMB Peer Review Bulletin and EPA's Peer Review Handbook, 4th Edition. The Agency will continue conducting peer reviews according to the peer review policies and practices that were in place before the rule goes into effect.
Comment: Commenter (6362) states that the language of 40 CFR 30.7 appears to be missing one or more words in the header to the section: "What role does independent peer review in this section?" The commenter expresses that they believe the missing word is likely "have," but suggests EPA should clarify and correct this section in the final rule.
Response: There was a typographical error in the header of Part 30.7 in the April 2018 notice of proposed rulemaking. The mistake was corrected in the supplemental notice of proposed rulemaking and in the final rule to read, "§30.6 What role does independent peer review have in this part?"
Comment: Commenter (6376) states that the ability to utilize peer-reviewed studies needs to be maintained as EPA moves towards creating more transparency. While EPA needs to evaluate and judge the merits of different data sets, the commenter suggests that it is impractical to expect the Agency to supplant the role of peer-review. The commenter adds that there should and can be a balance between transparency and the ability to draw from peer-reviewed scientific studies from outside sources.
Response: The EPA is finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate peer review, consistent with the OMB Bulletin for Peer Review and the EPA's Peer Review Handbook, of individual studies identified as pivotal science if the studies have already undergone journal peer review. All Agency peer review follows the recommended procedures and approaches found in the EPA's Peer Review Handbook.
Comment: Commenter (6362) notes that EPA uses the word "justify" frequently throughout the various sections of proposed regulatory text when referencing the use of regulatory science to make its decisions. The commenter provides, for example, §30.7 states: "EPA shall conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions." The commenter suggests that there are more precise words that EPA should use to link "pivotal regulatory science" with "regulatory decisions," such as "underpin" or constitute the "foundation" of the "scientific basis" of its regulatory decisions. 
Response: Part 30.6 in the final rule states, "EPA shall conduct independent peer review consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review and the exemptions described therein. The EPA will evaluate whether or not to conduct additional peer review of individual studies identified as pivotal science if the studies have already undergone peer review." The word "justify" is not used in this provision.
13.1.2.2 EPA's Review Shall Be Consistent with the Requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the Exemptions Described Therein.
 13.1.2.2.1 Time-Sensitive
Comment: Commenter (6137) suggests that, to be consistent with the OMB Bulletin, the independent peer review guidelines "do[] not cover time-sensitive health and safety disseminations, for example, a dissemination based primarily on data from a recent clinical trial that was adequately peer reviewed before the trial began. For this purpose, `health' includes public health, or plant or animal infectious diseases". The commenter provides that this encompasses most of the studies that EPA intends to review, including but not limited to, the Integrated Risk Information System (IRIS) assessments, TSCA risk evaluations, and National Emissions Standards for Hazardous Air Pollutants risk assessments. According to the commenter, in each case, EPA utilizes scientific data to make safety determinations for chemical pollutants and impacts on human health and the environment, and thus, timely review is tantamount to protecting public health. The commenter contends that the proposal to add another layer of review is antithetical to the time sensitive nature of these reviews.
Commenter (6137) states that the Bulletin makes clear that agencies should "ensure peer review does not unduly delay the release of urgent findings." The commenter asserts that, if EPA wants to independently peer review all pivotal science, then it must clearly outline how it will ensure the process will not lead to undue delay. The commenter notes that the Agency has failed to do so. The commenter provides, for example, there is nothing in the Proposed Rule outlining how EPA will conduct an independent review of the studies underlying the NAAQS standard, which will require review of hundreds if not thousands of science documents, within the NAAQS review cycle. The commenter adds that the standard review period in the independent peer review process (i.e., from submission of a manuscript to final review) is time intensive. The commenter states that, if it can take months to review one manuscript, and that it is difficult to envisage how EPA could subject every piece of pivotal regulatory science to the same standard of review and still complete scientific assessments in a timely manner.
Response: The EPA finds that comments received from the 2018 proposed rule and the 2020 SNPRM have merit, in part. However, in this rule, the EPA is not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review does not typically include reanalysis of the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information) and, thus, peer review is not considered a replacement for the data availability requirements of this rule.
Part 30.6 in the final rule states, "the EPA shall conduct independent peer review consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review and the exemptions described therein. The EPA will evaluate whether or not to conduct additional peer review of individual studies identified as pivotal science if the studies have already undergone peer review." The Agency will follow the requirements by the OMB Final Information Quality Bulletin for Peer Review ensuring that peer reviews do not unduly delay the release of urgent findings.
13.1.2.2.2 Studies that have Already Been Peer Reviewed
Comment: Commenter (2337) expresses concern regarding the Proposed Rule's requirement that EPA conduct independent peer review of scientific evidence on which it relies. The commenter states that the Proposed Rule applies this peer review requirement uniformly and does not exempt studies that have already been independently peer reviewed, as many health and scientific studies are. The commenter notes that EPA has not explained why EPA must conduct additional peer review of data that has already been reviewed. The commenter adds that this is of particular concern as many environmental statutes require EPA to periodically review, and where appropriate revise, regulatory standards. The commenter notes that, if EPA were now to add independent peer review to its periodic review of standards, it would be even less able to meet this mandated timeframe.
Comment: Commenter (5165) asks why should scientific literature that has already undergone peer review and been vetted by EPA's science advisory panels be subjected to an additional layer of government peer review? Commenter states that these key questions should have been considered, and the answers made public, prior to the rule's proposal.
Comment: Commenter (9227) asserts that EPA must clarify that studies that have already been adequately peer reviewed by third parties need not be re-reviewed by EPA. The commenter notes that, because the proposed language of §30.7 expressly incorporates the OMB Peer Review Bulletin "and the exemptions described therein," it appears that EPA intends to incorporate the OMB Peer Review Bulletin provision providing that "agencies need not have further peer review conducted on information that has already been subjected to adequate peer review." According to the commenter, however, there is some ambiguity due to the proposed language in §30.7 instructing that EPA must "ask peer reviewers to articulate the strengths and weaknesses of EPA's justifications for the assumptions applied and the implications of those assumption for the results." The commenter provides that peer review conducted prior to EPA's reliance on a study would not have involved review of the strengths and weaknesses of EPA's justifications. According to the commenter, if EPA were required to re-peer review all influential scientific information, this rulemaking would burden EPA with needless and significant costs that likely would bring many EPA rulemakings to a standstill, preventing EPA from fulfilling its statutory mission of protecting public health and the environment. The commenter asserts that, to prevent this from happening, EPA must clarify that the proposed rule will not supplant EPA's existing authority under the OMB Peer Review Bulletin not to conduct further peer review where information has already been subject to adequate peer review -- and that such prior peer review is not subject to the requirement in proposed §30.7 that reviewers consider the strengths and weaknesses of EPA's justifications.
Response: This rulemaking is not intended to modify the Agency's interpretations of "best available science." The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. All Agency peer review follows the recommended procedures and approaches consistent with the requirements of the OMB Peer Review Bulletin and EPA's Peer Review Handbook, 4th Edition. The Agency will continue conducting peer reviews according to the peer review policies and practices that were in place before the rule goes into effect.
The EPA finds that comments received from the 2018 proposed rule and the 2020 SNPRM have merit, in part. However, in this rule, the EPA is not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review does not typically include reanalysis of the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information) and, thus, peer review is not considered a replacement for the data availability requirements of this rule.
The EPA is finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate peer review, consistent with the EPA's Peer Review Handbook, of individual studies identified as pivotal science if the studies have already undergone journal peer review. Implementation of this rule may increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information. 
13.1.2.2.3 Prevention of Conflict of Interest in Peer Reviewers
Comment: Commenters (0044, 0844, 2241, 4541, 6180, 6877, 6891, 6927) express concern about how to prevent conflicts of interest in peer reviewers from impacting decision-making.
Commenter (6891) supports EPA's recent policy decision to have external scientific advisory boards be represented by a diverse group of stakeholders, including those representatives from regulated industry. 
Commenter (6927) recommends EPA ensure any third-party peer reviewers are truly independent and external from the agency staff, former staff or organizations that receiving EPA funding.
Commenter (6877) states that the proposed rule's statement, "EPA shall conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions," risks the introduction of potential bias through the selection of the peer reviewers. The commenter notes that the reviewers should be individuals who have the relevant expertise and are impartial with full disclosure of any conflicts of interest. The commenter asserts that, having had industry, government, or other funding for work on the topic at hand should not by itself be disqualifying to serving in peer review of scientific research.
Commenter (0044) asserts that scientists from industries directly impacted by rulemaking on chemicals must not be involved in the peer review; they could serve as non-voting advisors to a review panel; they have an opportunity to submit their opinions under FIFRA applications. The commenter notes that, otherwise, the potential for conflict of interest is significant. The commenter states that the goal of rulemaking for chemical exposures is to protect the public health; it is not for putting industry concerns first.
Commenters (0844) states that when scientists who receive EPA grants also serve on one of the agency's advisory boards, a conflict is created. According to the commenter, these conflicts were eliminated when Administrator Pruitt issued a new directive in October 2017, that scientists who receive an EPA grant, or reap a substantial benefit from a grant, could not also serve on an advisory committee. The commenter notes that the scientists would be given the option of either ending their grants or to continue to advise EPA, but not both.
Commenter (4541) asserts that EPA needs to conduct its own peer review and remove from its advisory committees and peer review process individuals with conflicts of interest. The commenter states that it is always a concern when reviewers and their institutions are conflicted by their dependence on agency/government funding and yet expected to provide impartial, objective reviews for science that is funded by that same agency/government.
Commenter (6180) states that the proposed rule must require and ensure that peer reviews be comprised of independent experts and not only EPA staff or otherwise biased individuals. The commenter supports the OMB Final Bulletin which notes the importance of the independent nature of peer reviewers and that government employees are subject to federal ethics requirements (70 FR 2664).
Response: EPA is subject to the OMB Bulletin and its provisions have been incorporated into the Agency's peer review handbook (Ref. 44). EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will lead to consistent implementation of those procedures across the Agency. In the final rule, the EPA "shall conduct independent peer review consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review and the exemptions described therein." All Agency peer review follows the recommended procedures and approaches found in the EPA's Peer Review Handbook. This Peer Review Handbook is used as guidance by EPA staff and managers to ensure that the Agency's Peer Review Policy is implemented objectively and that the integrity of our peer review activities can be demonstrated transparently to the American public.
Comment: Commenter (1972) requests that EPA clarify the use of peer review in proposed 40 CFR 30.7. The commenter provides that the OMB Bulletin states that, for more important work, "independent" peer review should involve "[r]reviewers" that "are generally not employed by the agency or office producing the document. As NAS has stated, `external experts often can be more open, frank, and challenging to the status quo than internal reviewers, who may feel constrained by organizational concerns.'" The commenter adds that EPA's own guidelines similarly note that "[f]or those work products that are intended to support the most important decisions or that have special importance in their own right, external peer review is the procedure of choice." The commenter expresses that, given the credibility bestowed by both the courts and the public upon the studies that are "peer reviewed," EPA should clarify the conditions under which it will use truly independent external peer reviewers. The commenter recommends that this include some parameters on how external peer reviewers are selected, such as taking care to ensure that reviewers are not those working under, or applying for, EPA grants or otherwise have long-standing relationships with EPA, such as former EPA officials now serving in academia. The commenter suggests that independent external peer review be used for all "significant regulatory actions" as that term is defined under Executive Order 2866. The commenter contends that avoiding internal reviews by other EPA officials for significant regulatory actions will help restore public trust in the quality of information relied upon by EPA.
Response: This rulemaking is not intended to modify the Agency's interpretations of "best available science." The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. All Agency peer review follows the recommended procedures and approaches consistent with the requirements of the OMB Peer Review Bulletin and EPA's Peer Review Handbook, 4th Edition. The Agency will continue conducting peer reviews according to the peer review policies and practices that were in place before the rule goes into effect.
The EPA finds that comments received from the 2018 proposed rule and the 2020 SNPRM have merit, in part. However, in this rule, the EPA is not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review does not typically include reanalysis of the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information) and, thus, peer review is not considered a replacement for the data availability requirements of this rule.
Comment: Commenter (0555) states that, when engaging experts in peer review, EPA should consider the recommendations of recent interdisciplinary efforts regarding scientific advisory panels. The commenter adds that such advisors can provide a necessary and valuable source of information and peer review for agency science, but care should be taken in both the composition of the panels and the charges they are given. According to the commenter, an important 2012 Keystone Center report offers a series of recommendations on "the composition of committees that are empaneled to review the science behind a regulatory decision." The commenter notes that the report acknowledges the importance of choosing panelists that "have the knowledge, training, and experience needed to address the charge to the panel," and it encourages agencies "to recognize that all potential panelists will have conscious and unconscious biases." The commenter adds that it recommended, "because biases exist, an agency should strive to engage a wide range of perspectives of qualified scientific experts." The commenter notes that both the Keystone group and a group convened by the Bipartisan Policy Center (BPC) in 2009 recommend that scientific peer reviewers restrict their advice to matters of science, and not be asked to recommend regulatory policies. The commenter states that EPA's Clean Air Scientific Advisory Committee (CASAC), for example, is tasked with advising on policy choices, which creates incentives to present policy views as scientific recommendations.,  The commenter recommends, when drafting charge questions for individual peer reviewers and scientific advisory committees, EPA be careful to solicit their scientific expertise without encouraging them to blur the lines between scientific expertise and policy judgment. According to the commenter, as both the BPC and Keystone reports emphasized, the questions posed to experts "should be clearly articulated, and `explicitly differentiate, to the extent possible, between questions that involve scientific judgments and questions that involve judgments about economics, ethics, and other matters of policy.'"
Response: This rulemaking is not intended to modify the Agency's interpretations of "best available science." The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. All Agency peer review follows the recommended procedures and approaches consistent with the requirements of the OMB Peer Review Bulletin and EPA's Peer Review Handbook, 4th Edition. The Agency will continue conducting peer reviews according to the peer review policies and practices that were in place before the rule goes into effect. However, as detailed in OMB's Final Information Quality Bulletin for Peer Review, "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary". Peer review does not typically include reanalysis of the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information) and, thus, peer review is not considered a replacement for the data availability requirements of this rule.
In the final rule, the EPA "shall conduct independent peer review consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review and the exemptions described therein." All Agency peer review follows the recommended procedures and approaches found in the EPA's Peer Review Handbook. This Peer Review Handbook is used as guidance by EPA staff and managers to ensure that the Agency's Peer Review Policy is implemented objectively and that the integrity of our peer review activities can be demonstrated transparently to the American public.
Comment: Commenter (1390) asserts that scientists within government agencies and outside of government agencies should be involved in reviewing scientific information. The commenter states that they should be in continual dialogue about strengths and limitations of available scientific research. According to the commenter, government agencies should not be permitted to submit, approve, or implement policies until it is clear that agencies have exhausted all possible methods to review and dialogue about all scientific information relevant to a policy. The commenter adds that it will be impossible for all public policies to be completely aligned with all relevant scientific information, however, government agencies should not be permitted to submit, approve, or implement policies that are incongruent with relevant scientific information until they have transparently acknowledged the scientific information that pertains to the policy and transparently justified the reasons for which a policy is incongruent with specific scientific information. 
Response: All Agency technical work products go through rigorous internal review and, when appropriate, peer review. The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. In the final rule, the EPA shall conduct independent peer review consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review and the exemptions described therein. All Agency peer review follows the recommended procedures and approaches found in the EPA's Peer Review Handbook [4[th] Edition]." The Agency will continue conducting peer reviews according to the peer review policies and practices that were in place before the rule goes into effect. Further, the EPA plans to promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
Comment: Commenter (2241) recommends that the rule indicate how transparency will be used in the appointment of peer reviewers. The commenter expresses concern that the rule does not describe how this will be implemented. The commenter adds that, in particular, 70 FR 2664 clearly states that reviewers shall be selected for expertise, balancing of perspectives, independence, and freedom from conflicts of interest (which are to be disclosed).
Response: This rulemaking is not intended to modify the Agency's interpretations of "best available science." The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. In the final rule, the EPA "shall conduct independent peer review consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review and the exemptions described therein." All Agency peer review follows the recommended procedures and approaches found in the EPA's Peer Review Handbook. All Agency peer review follows the recommended procedures and approaches consistent with the requirements of the OMB Peer Review Bulletin and EPA's Peer Review Handbook, 4th Edition. The Agency will continue conducting peer reviews according to the peer review policies and practices that were in place before the rule goes into effect.
The EPA finds that comments received from the 2018 proposed rule and the 2020 SNPRM have merit, in part. However, in this rule, the EPA is not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary."
Comment: Commenter (4595) expresses concern that the proposal does not articulate whether the "peer reviewers" would be the same peer reviewers or scientific advisory committee that conducted the original scientific assessment. The commenter states that this introduces an unnecessary phase of peer review that risks introducing regulatory inertia and the ability of the agency to utilize the best available science in its decision making.
Response: The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. In the final rule, the EPA "shall conduct independent peer review consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review and the exemptions described therein." All Agency peer review follows the recommended procedures and approaches found in the EPA's Peer Review Handbook. All Agency peer review follows the recommended procedures and approaches consistent with the requirements of the OMB Peer Review Bulletin and EPA's Peer Review Handbook, 4th Edition.
The EPA finds that comments received from the 2018 proposed rule and the 2020 SNPRM have merit, in part. However, in this rule, the EPA is not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." The EPA maintains that all Agency peer review follows the recommended procedures and approaches consistent with the requirements of the OMB Peer Review Bulletin and EPA's Peer Review Handbook, 4th Edition. The Agency will continue conducting peer reviews according to the peer review policies and practices that were in place before the rule goes into effect.
13.1.2.2.4 Other Comments
Comment: Commenter (6375) states that the exemptions described under OMB's Information Quality Bulletin for Peer Review to ensure compliance with privacy and national security laws and regulations are appropriate. The commenter recommends that EPA handle these exemptions in a similar manner to the case-by-case exceptions for studies: The commenter further recommends that EPA provide a robust and transparent explanation in the Federal Register early in the rulemaking process to explain why the provisions of this rule do not apply to the rulemaking in question (e.g. which laws would be in conflict). 
Response: All Agency peer review follows the recommended procedures and approaches found in the EPA's Peer Review Handbook. This Peer Review Handbook is used as guidance by EPA staff and managers to ensure that the Agency's Peer Review Policy is implemented objectively and that the integrity of our peer review activities can be demonstrated transparently to the American public. Part 30.6 in the final rule states, "the EPA shall conduct independent peer review consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review and the exemptions described therein. The EPA will evaluate whether or not to conduct additional peer review of individual studies identified as pivotal science if the studies have already undergone peer review. As part of promulgating a significant regulatory action or drafting influential scientific information the Agency will document its decisions and reasoning." 
Comment: Commenter (9227) suggests that, if EPA moves ahead with these proposed provisions, EPA revise the proposed regulatory language to clarify that all of the exemptions set forth in section IX of the OMB Peer Review Bulletin remain in effect regardless of whether they pertain to feasibility. The commenter further suggests that EPA clarify what, if any, additional effect is intended by the exemption provision in proposed 40 CFR 30.9. The commenter provides that proposed 40 CFR 30.9(b) provides that the Administrator may grant an exemption from the peer review requirements if he or she determines that "[it] is not feasible to conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions for reasons outlined in OMB Final Information Quality for Peer Review (70 FR 2664), Section IX." The commenter notes, however, only two of the seven enumerated exemptions in Section IX of the OMB Peer Review Bulletin pertain to feasibility -- Exemption 1 governing "national security, foreign affairs, or negotiations involving international trade or treaties" and Exemption 3 governing time-sensitive health or safety disseminations.
Response: The requirements in 40 CFR 30.3 apply to influential scientific information, unless the influential scientific information is exempted from peer review requirements as described in Section IX of the OMB Bulletin for Peer Review (Ref. 8). Consistent with this approach, the EPA is finalizing the definition of "influential scientific information" as proposed in the 2020 SNPRM. 
Part 30.6 in the final rule states, "the EPA shall conduct independent peer review consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review and the exemptions described therein. The EPA will evaluate whether or not to conduct additional peer review of individual studies identified as pivotal science if the studies have already undergone peer review."
Section 30.7 (formerly 40 CFR 30.9 in the 2018 proposed rule and the 2020 SNPRM) of the final rule requires the Agency to document the rationale for any exemptions granted by the Administrator in the significant regulatory action or influential scientific information. This documentation would typically be provided as part of the proposed rulemaking given this will be part of the decision of what is the pivotal science for the rule. The final rulemaking will also provide clear documentation.
13.1.2.3 EPA Shall Ask Peer Reviewers to Articulate the Strengths and Weaknesses of EPA's Justification for the Assumptions Applied and the Implications of those Assumptions for the Results
Comment: Commenter (6150) supports the appropriateness of requesting that peer reviews focus on the strengths and weaknesses of the assumptions and their implications.
Commenter (6927) requests EPA more clearly define and clarify what is meant by asking "peer reviewers to articulate the strengths and weaknesses of EPA's justification for the assumptions applied and the implications of those assumptions for results.
Conversely, commenters (4592, 4595, 6133) express concerns about the requirement that peer reviews focus on the strengths and weaknesses of the assumptions and their implications.
Commenter (4595) expresses concern that the proposed rule requires EPA to ask peer reviewers to articulate the strengths and weaknesses of EPA's assumptions, implying that the agency would be implementing a separate and secondary level of peer review to address assumptions. 
Commenter (6133) states that the final paragraph of 40 CFR 30.7 appears to suggest that EPA should conduct peer review of the proposed agency action itself, rather than of the science underlying that action, stating that "EPA shall ask peer reviewers to articulate the strengths and weaknesses of EPA's justification for the assumptions applied and the implications of those assumptions for the results." According to the commenter, it is entirely unclear how "peer review" could be applied to EPA's reasoning itself, rather than the cited science, and the Proposal contains no further clarification. The commenter recommends that the EPA abandon the unlawful Proposal altogether but, if EPA does finalize any rule based on the Proposal, EPA still should abandon the unlawful approach reflected in proposed 40 CFR 30.7.
Response: EPA agreed with commenters that the statement in §30.7 of the notice of proposed rulemaking, "implications of the assumptions on the results," is unclear about how peer review could be applied to EPA's reasoning itself, rather than the pivotal science supporting the regulatory decision. Therefore, §30.6 (formerly 40 CFR 30.7 the 2018 proposed rule and the 2020 SNPRM) of the final rule was changed to "Because transparency in pivotal science includes addressing issues associated with assumptions used in analyzing dose-response data, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied and the implications of those assumptions for the results." Further, as detailed in OMB's Final Information Quality Bulletin for Peer Review, "There will be cases in which an agency determines that a more rigorous or transparent review process is necessary. For instance, an agency may determine a particular journal review process did not address questions (e.g., the extent of uncertainty inherent in a finding) that the agency determines should be addressed before disseminating that information. As such, prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." For these reasons, EPA may determine that additional peer review is necessary.
Comment: Commenter (0560) requests that EPA add the following to §30.7: "EPA shall particularly request that reviewers address alternate models and ignored or contrary evidence."
Response: EPA appreciates the recommendation. The EPA has finalized language in §30.6 (formerly 40 CFR 30.7 the 2018 proposed rule and the 2020 SNPRM) of the final rule to, "Because transparency in pivotal science includes addressing issues associated with assumptions used in analyzing dose-response data, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied and the implications of those assumptions for the results."
13.1.3 Comparison of the Approach in the Proposed Rule to Existing Scientific Review Processes
13.1.3.1 Existing Peer-Review Mechanisms
 13.1.3.1.1 Peer Review is Not Adequate
Comment: Commenter (0040) states that study results need to go through the peer review publication and replication processes so that the conclusions are based on strengths of evidence and are weighed according to their merit. The commenter asserts that it is through this laborious process that science progresses. 
Commenters (0563, 2142, 4534, 6045, 6176, 6365, 6867) suggest that peer review, in and of itself, is not sufficient to validate studies.
Commenter (0563) notes that, while some opponents of the proposal argue that peer review ensures the quality of studies published in refereed journals, this widespread perception is false. The commenter quotes Richard Smith:
      ... almost no scientists know anything about the evidence on peer review. It is a process that is central to science -- deciding which grant proposals will be funded, which papers will be published, who will be promoted, and who will receive a Nobel prize. We might thus expect that scientists, people who are trained to believe nothing until presented with evidence, would want to know all the evidence available on this important process. Yet not only do scientists know little about the evidence on peer review but most continue to believe in peer review, thinking it essential for the progress of science. Ironically, a faith based rather than an evidence-based process lies at the heart of science. 
Commenter (2142) asserts that the peer review system is flawed because reviewers do not dig deeply into the data or analysis as they have little incentive to do so. The commenter states that reviewers often do not have access to the data or algorithms, making it impossible to truly check the results. The commenter states that the high percentage of invalidated peer-reviewed studies in the medical field, where one would think the scrutiny to be intense, shows the failure of relying solely on peer review. 
Commenter (4534) states that, if the data has been peer reviewed and is available for the regulated parties to validate during the review and comment period, then all parties should be satisfied with the process. According to the commenter, utilizing data that has not been peer reviewed or validated could lead to incorrect conclusions, which impact recommendations to maintain or further improve the environment that we live in.
Commenter (6045) asserts that peer-reviewed scientific research data that are not publicly available are not rigorous enough for use in decision making. The commenter asks why should a secretly "insider-peer-reviewed" elitist group evaluate and approve something for public policy that affects millions of citizens, as well as billions of U.S. dollars in taxpayers' money? 
Commenter (6176) states that journal peer review is not a substitute for making data available to the public. The commenter states that not all peer review processes are the same or of adequate quality. The commenter adds that, unless relevant underlying data and methodology are made available, the public has no way to know whether a specific peer review process has been effective and properly addressed potential problems.
Commenter (6867) asserts that pre-publication peer review is not an adequate substitute for public availability of data, methodology, and computational code. The commenter states that, in light of overwhelming evidence that a significant portion of studies published in prominent peer reviewed journals are not reproducible, it is dubious to claim that any particular research is valid and relevant unless -- at a minimum -- other researchers and the public have access to the underlying data, methodology, and computational code. The commenter states that empirical evidence indicates that peer reviewers routinely fail to identify even major errors and peer review in less prominent journals may often occur in name only. The commenter states that the best available metascience -- science about science -- indicates that pre-publication peer review is not adequate to ensure the validity of published scientific claims.
Commenter (6365) states that the experience of the lead NAAQS rulemaking (73 FR 66964) demonstrates the salutary purpose that transparency would serve in the regulatory process. The commenter provides that, in that rule, EPA relied heavily on Lanphear et al. (2005) although the underlying data were not available to EPA or to stakeholders for replication and validation at the time of publication. The commenter notes that, due to the lack of availability, there was no complete reanalysis of that data for nearly a decade and EPA relied on the study in both the 2008 and 2016 lead NAAQS reviews. The commenter states that the lack of available data has been noted in the scientific literature as preventing alternative statistical analyses to be performed, which is particularly important given that key confounding factors such as parental education, intelligence, birthweight, smoking, and race were not controlled in several of the Lanphear cohorts and may contribute to an overestimation of health effects. The commenter provides that, when the data underlying Lanphear was reanalyzed numerous errors that were not discovered in the peer-review process and that were not previously known to EPA were uncovered and analyzed. According to the commenter, although EPA ultimately determined that these errors did not materially affect the NAAQS review documents that relied on the Lanphear study, the Agency included a memorandum in the subsequent NAAQS rulemaking docket that acknowledged those errors and considered their impact on the regulatory process.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions.
The EPA finds that these comments have merit, in part. However, in this rule, the EPA is not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review does not typically include reanalysis of the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information) and, thus, peer review is not considered a replacement for the data availability requirements of this rule. 
The EPA would also like to clarify that peer review is but one aspect that is considered in evaluating studies for potential use in EPA products. In addition to determining whether a study has been adequately peer reviewed, the EPA also independently evaluates study quality using the following factors: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. The EPA may also consider additional factors based on the purpose of the assessment. For example, when selecting studies for the dose-response assessment, the EPA would evaluate the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints.
Comment: Commenter (6362) highlights the NAS review of the 2010 draft formaldehyde IRIS assessment peer review as an example of where the IRIS assessment failed to reflect the best available science. The commenter expresses that the NAS stated that EPA's claims regarding all leukemias, myeloid leukemia or related hematopoietic cancers were not supported. The commenter also provides that EPA's preliminary conclusions appeared subjective and that no clear scientific framework had been used by EPA to reach its conclusion. The commenter adds that the NAS recommended that EPA revisit its determination of causality for specific cancers, using a methodology that integrates lines of evidence and addresses the specific criticisms in the NAS report. The commenter notes that the NAS also made numerous recommendations for improving the overall process and application of science used in all assessments generated by the IRIS program. Now, according to the commenter, seven years since that NAS report was published, EPA continues to revise its assessment while not disclosing how emerging scientific evidence or modern risk assessment methods are being employed.
Response: These comments are not directly relevant to this rulemaking. Any concerns about specific projects should be directed to the appropriate EPA programmatic or regional office.
13.1.3.1.2 Appropriate Review Processes Already Exist
 Overall Support for Current Review Processes
Comment: Commenters (0061, 1364, 1433, 1773, 1943, 1973, 2249, 2408, 2425, 2905, 2908, 2911, 4842, 4845, 4884, 5164, 5165, 6103, 6122, 6133, 6135, 6149, 6153, 6164, 6171, 6174, 6188, 6191, 6358, 6367, 6379, 6446, 6464, 6868, 6876, 6898, 6915, 6922, 6974, 6901, 6916, 6920, 8272, 8275, 8801, 9227) indicate that, in peer review, appropriate processes and procedures for designing and conducting studies already exist, including existing peer review procedures, binding contracts, science assessments, use of peer-reviewed literature and collaborative methods. Comments include:
 EPA's existing processes, including peer review, already help ensure that studies used by EPA are scientifically sound, the proposed rule is not needed to add credibility or reliability to the development of EPA models and standards. (6915)
Commenters (2425, 5165, 6358, 6868, 8272) state that the process of peer-reviewed research is already sufficiently transparent by design and that there are specific existing rigorous mechanisms already in place, including Institutional Review Board (IRB) review, peer review of funding requests, and peer review by journals before publication:
 Regarding the IRB, commenter (2425) states that EPA requires an IRB to review all scientific studies occurring in an academic institution prior to beginning the study. The IRB closely reviews all study methods to ensure that the methods are sound and study subjects are protected and provides details of requirements. Once a study is complete, data involving human subjects must always be kept for a minimum of five years after study completion and be available for IRB review. (2425)
 Commenters (6358, 8272) state that peer review processes for funding are often highly competitive and involve critiques of study design that, if addressed, can lead to research with fewer potential biases or unmeasured confounders. (6358, 8272)
 Commenters (2425, 5165, 6358, 8272) contend that in order to publish the data in a scientific journal, scientists not associated with the research study must review the study design to ensure that it is scientifically sound and that there are many rigorous aspects of this review. 
 Commenter notes that this includes assessing for "good laboratory practices" and "standardized test methods." In this rigorous peer-review process, which is already required for studies to be considered by EPA, the commenter states that experts in the field of study not only assess the study's design and methods, but its statistical analysis and conclusions in an objective manner to determine the study's merit and significance. The commenter asserts that, if any of these areas are deemed to be unsatisfactory, the study will not be published. Additionally, the commenter states that, when studies are published, the authors must detail all methods and statistical tests used and this information is presented to the public. (2425)
 Commenter (5165) notes that when the results of a scientific study are submitted for publication, the uncertainties, assumptions, parameters and theories utilized by the scientists are laid out in the publication; peer review analyzes all these components to establish validity. Thus, commenters (6358, 8272) state that peer reviews for journal publications often involve critiques that lead to more nuanced discussions of a study's limitations and the conclusions that can be drawn given those limitations. 
 Study manuscripts go through multiple rounds of rigorous peer review to ensure the validity of study methods, the fidelity of adhering to the scientific methods, and the added value of the findings yielded from the study. The commenter provides that, during the peer review process, scientists are frequently asked to perform additional analyses and present additional results to convince the peer reviewers that the findings are robust, and their conclusions are supported by the data. The commenter states that, for certain types of studies without identifying patient data, such as many clinical trials and basic laboratory research studies, it is becoming the norm among top tier journals to request data as part of the publication process. The commenter adds that many journals also use plagiarism detection software and submit images to manipulation detection software to ensure data integrity. (6868)
 Commenters note that there are also mechanisms for review and comment post-publication of studies in journals. Commenters state that investigators or institutions in possession of data sets often make them available to researchers who meet appropriate criteria, such as demonstrating the ability to safeguard privacy for research participants. The commenters express that other researchers may use the data either to reanalyze findings or to examine additional research questions. (6358, 8272)
 Commenters (6358, 6868, 8272) state that studies that generate unexpected findings and provocative implications provoke discussion. Commenter (6868) states that (1) letters to editors of science journals question the sample size, the study construction, the analytical method, the generalizability and nearly every conceivable aspect of a study; and (2) scientific meetings host pro/con debates about how to interpret provocative data and how far to apply its implications. The commenter (6868) states that this post-publication debate helps generate new questions to ensure that our understanding of the data is correct. Commenters (6358, 8272) state that re-analyses of data may be undertaken at times in order to strengthen confidence in particularly when there are unexpected or consequential findings, but typically confidence in findings is strengthened by having evidence from different studies accumulate.
 EPA already has in place a robust process for determining studies' reliability. The commenter states that EPA has spent decades creating and implementing an elaborate system to ensure the best scientific research serves as the basis for its regulatory actions. The commenter states that this system already includes vigorous peer-review requirements, ethical standards, and independent review by scientific professionals., The commenter states that according to EPA's own IQGs, EPA "maintain[s] a robust quality system that addresses [information collected through contracts with EPA; information collected through grants and cooperative agreements with EPA; and information submitted to EPA as part of a requirement under a statute, regulation, permit, order or other mandate] by including regulatory requirements for quality assurance for EPA contracts, grants, and assistance agreements." The commenter notes that this includes an Agency-wide Quality System that requires, among other things, a quality assurance manager and Quality Assurance Project Plan (QAPP) for projects and tasks that involve the use of environmental data, and a Peer Review System that requires major scientifically- and technically-based Agency work product to be peer reviewed. (6188)
 Commenters (0061, 1433, 1773, 1943, 6164, 6191, 6358, 6367, 6464, 6868, 6164, 8272, 8801) assert that the peer review process is the gold standard for scientific understanding and advancement for decades and assert that peer review is the integrity and reliability standard for ensuring unbiased, rigorous vetting of research and scholarly content prior to publication. Commenter (6164) states that the peer review process remains the global gold standard of academic achievement and affords the type of informed discourse necessary for the objectivity, rigor, and legitimacy of scientific information. Commenter (9227) states that publication in a peer-reviewed scientific journal is the way that scientists communicate their findings to other scientists and is considered the hallmark of scientific quality. Commenter (6103) contends that the peer review process is more than adequate to ensure studies are reliable to use in the policymaking process. 
 EPA already uses collaborative methods to get to the truth of needed environmental protections. The commenter notes that the Volkswagen diesel passenger vehicle transgression was discovered by West Virginia University students operating in a laboratory using methods not then sanctified by the U.S. EPA Auto Emissions Laboratory. (4884)
 EPA has rightly continued to rely on the robust peer-reviewed literature to inform the air quality standard-setting process year after year, incorporating the best available scientific evidence in epidemiology, toxicology, and exposure assessment to set the outdoor air quality standards at levels that protect public health and the environment. (6133)
 Existing procedures and safeguards commonly used to conduct, disseminated, and synthesize scientific research are more than adequate in ensuring the validity and integrity of the science used to inform EPA decision-making. (6135)
 Existing policy vehicles under established statutory authority have proven effective in providing a consensus scientific basis through open and transparent processes that support a regulatory framework for radiation protection that balances the beneficial uses of ionizing radiation with the health and safety of workers, patients, and the public. (6149)
 The review process currently utilized by EPA in evaluation of scientific research to inform policy is already transparent and involves rigorous peer-review and evaluation by independent experts. (6171)
 EPA has a proven record of using the best available science and research to set standards under its foundational laws that protect public health and the environment, which the scientific peer review process and our transparent and participatory rulemaking process can well ensure. (6174)
 Commenter is confident that longstanding practices of peer review undertaken by leading journals that publish toxicology studies of public health importance have been and will continue to be of sufficient rigor to assure the integrity and reliability of such research. (6367)
 EPA-implemented statutes have established independent scientific oversight and scientific review processes to ensure that EPA standards and regulations are founded in the best available science. (6379)
 Procedures are already in place that assure use of the best available science for EPA's regulatory activity. The commenter states that these procedures include that much of the scientific research considered in the regulatory context, and all research published in reputable journals, has already been subject to extensive peer review. The commenter states that the method that is accepted in the scientific community for assessment of the strength of scientific research is peer review. The commenter notes that, since scientific studies are regularly peer reviewed, including peer review by expert panels such as the SAB, the proposed rule unnecessary. (6446)
 Commenter encourages EPA to continue the current open and science-based development process and continue to actively involve states. (6901)
 Formal peer review, whether or not prior to publication, also validates the outcome of scientific studies, as reflected in the OMB Guidelines presumption of objectivity and usefulness for scientific studies that have undergone formal peer review. (6916)
 The proposal calls into question established processes such as EPA's IRIS review program and NAAQS program. The commenter states that EPA more-than-adequately explains its decisions and analyses both in recommending NAAQS revisions and in quantifying chemical toxicities in IRIS. (6920)
 The proposed rule is unnecessary as there are processes in place to rigorously review the scientific integrity of the studies that are used in regulatory science. The commenter states that protocols and data analysis plans are published and publicly available. The commenter states that findings are peer reviewed by scientists prior to publication and additional data are available upon request for review. The commenter states that external advisory boards with relevant expertise review the studies that are being considered in the EPA regulatory processes. (6876)
 The processes for developing human health assessments and setting and reviewing standards by EPA have been routinely scrutinized by organizations such as the NAS and the Government Accountability Office (GAO). The commenter states that EPA has incorporated their recommendations and improved its approach over time. (6920)
Response: The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. EPA does not take the position that studies for which the underlying data are not publicly available are per se invalid. Rather, EPA acknowledges that where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty. As discussed in the final rule preamble, the EPA clarifies that the rule requirements are not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Revie . The preamble to the Bulletin states that, "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review by either journals or independent advisory committees, such as the EPA's SAB, is typically conducted without access to the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information). Should peer reviewers have access to the underlying data, they could conduct a more in-depth review that may include independently reanalyzing the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty in the original study. Thus, peer review is not considered a replacement for the data availability requirements of this rule and implementation of this rule could provide independent peer reviewers as well as subject matter experts within the EPA and the public with the opportunity to evaluate pivotal science in ways that otherwise would not have typically occurred with existing EPA or peer review processes. 
The EPA is, therefore, finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate additional peer review, consistent with the OMB Bulletin for Peer Review, of individual studies identified as pivotal science if the studies have already undergone peer review. If the Agency conducts peer review on pivotal science, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied in analyzing dose-response data and the implications of those assumptions for the results.
The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Implementation of this rule will also increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Advisory Boards
Comment: Commenters (1973, 2336, 5165, 5172, 6046, 6119, 6121, 6164, 6188, 6367, 6373, 6446, 6894, 6920, 6922, 8801, 9227) suggest that advisory boards already provide assessment of the adequacy of the approaches taken in the studies, including the EPA SAB and EPA's CASAC. Commenter (6922) provides that the primary function of EPA's SAB is to review the quality and relevance of scientific and technical information being used by EPA or proposed as the basis for its regulations. 
Commenter (6145) states that in addition to all of the already existing laws and guidance under which EPA is directed to review scientific information in its promulgation of regulations, EPA also has "several mechanisms for vetting science through several expert panels, including the EPA SAB, the EPA Clean Air Scientific Advisory Committee, and the EPA FIFRA Scientific Advisory Panel . . .."
Commenter (2336) asserts there are some aspects of EPA's duties that need to be insulated from political considerations, at least to the extent that the scientific integrity of a study can be judged independently of the political ramifications of using it to formulate regulations. The commenter states that, to fill that need, Congress provided for a 48-member SAB in the CAA. The commenter states that, although the Board's recommendations are advisory, the EPA Administrator, as a practical matter, is obligated to provide a cogent and convincing rationale for rejecting the Board's determinations. 
Commenters (5165, 6164, 9227) express concern that the proposal does not acknowledge that EPA already has institutional mechanisms to review and vet scientific information through panels of scientific experts. The commenters assert that these expert panels were established to review the quality and relevance of scientific and technical information being used by EPA or proposed as the basis for EPA regulations. Commenter (5165) states that, by ignoring the existence of these bodies in the proposed rule, EPA suggests that it does not trust its own scientific advisors which tends to undermine public confidence in EPA decision making, rather than to bolster it. Commenter (6164) urges EPA to use these resources and clarify how this expertise will be incorporated into rulemaking under this proposal. Commenter (9227) adds that a SAB workgroup noted:
      The proposed rule fails to mention that EPA has mechanisms for vetting science through several expert panels . . . For example, the EPA CASAC routinely reviews and evaluates epidemiologic and toxicological studies that are the basis for dose-response relationships used in risk and exposure assessments for air pollutants regulated under the National Ambient Air Quality Standards. Although such mechanisms do not typically engage in reanalysis of original data using the same methods as the original investigators, they do entail a rigorous review process that goes beyond the typical journal peer review procedures.
Commenter (6446) states that procedures are already in place that assure use of the best available science for EPA's regulatory activity. The commenter provides that these procedures include the SAB which is required by its authorizing legislation to make every effort to maximize public participation and transparency, "including making the scientific and technical advice of the [SAB] and any investigative panels . . . publically available in electronic form on the website of the Environmental Protection Agency." The commenter notes that the proposed rule does not mention the SAB, or its important function in providing expert assessments of whether scientific research offered as the basis for regulatory action is, in fact, the best available science. 
Commenter (6894) states that the CASAC was established in 1977 to explicitly provide a diverse, independent science peer review of all the considered science so the Administrator would have a direct report on the science and policy options to consider in parallel to those of staff. The commenter states that, along with public comment, this proven process has long provided the credible "process" only vaguely described in the proposal.
Commenter (1973) states that the EPA SAB review process already provides assessment of the adequacy of the approaches taken in the studies considered in setting a standard. The commenter provides the following example regarding the setting of air quality standards: 
      The process involves a summary of the science by the EPA in the Integrated Science Assessment (ISA), which is reviewed by experts, and then provided to the external EPA Clean Air Science Advisory Committee (CASAC) for review. If, based on how the analysis was done, what confounders were controlled for, or other criteria, the reviewers of the ISA, members of CASAC, or the general public find a study wanting, they are free to raise that issue in public comments to EPA on the draft ISA, which EPA is required to respond to substantively. Subsequently they can again provide comments to CASAC when it reviews the draft ISA. This has happened routinely in the past, where many comments on the adequacy of individual studies, on possible flaws in measurement techniques or analytical methods have been raised and debated in past NAAQS reviews under the current review process for setting the NAAQS. There is no need to exclude most human studies to achieve a goal that is already being met.
Response: Peer review by either journals or independent advisory committees, such as the EPA's SAB, is typically conducted without access to the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information). Should peer reviewers have access to the underlying data, they could conduct a more in-depth review that may include independently reanalyzing the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty in the original study. Thus, peer review is not considered a replacement for the data availability requirements of this rule and implementation of this rule could provide independent peer reviewers as well as subject matter experts within the EPA and the public with the opportunity to evaluate pivotal science in ways that otherwise would not have typically occurred with existing EPA or peer review processes.
The EPA is finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate peer review, consistent with the OMB Bulletin for Peer Review and the EPA's Peer Review Handbook, of individual studies identified as pivotal science if the studies have already undergone journal peer review. If the Agency conducts peer review on pivotal science, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied in analyzing dose-response data and the implications of those assumptions for the results." The EPA Peer Review Handbook, 4th Edition, describes the process for peer reviews conducted by a Federal Advisory Committee and the National Academy of Sciences.
Peer Review is Rigorous
Comment: Commenter (6922) states that the EPA's Peer Review Handbook provides that if a regulation is supported by a scientific and technical work product, the underlying work product should be peer reviewed unless it meets listed exemption criteria. The Handbook explains that a critical element in ensuring that decisions are based on sound and defensible science is to have an open and transparent peer review process. 
Commenters (1459, 1973, 2353, 2425, 2479, 4844, 5170, 5180, 6111, 6874, 6915, 6920, 6925) assert that scientific articles have already been subjected to a high degree of scrutiny and strong filters of quality control during the peer review process. 
Commenter (6915) asserts that the existing peer review process ensures the reliability and validity of the scientific information relied upon by EPA in the regulatory process. The commenter states that existing policies and procedures for peer review include the following:
 EPA vets scientific studies through several independent expert panels, including the SAB, the EPA CASAC, the EPA FIFRA Scientific Advisory Panel, and the EPA Chemical Assessment Advisory Committee. The CASAC routinely reviews and evaluates epidemiological and toxicological studies that are the basis for dose response relationships used in risk and exposure assessments for air pollutants regulated under the NAAQS; the Chemical Assessment Advisory Committee reviews toxicological assessments of various chemicals for inclusion in EPA's IRIS database; and the NAS has reviewed EPA risk assessment practices numerous times.
 Each of these independent committees or panels is required to be staffed by a "fairly balanced" mix of regulators, academics, and industry/consultant representatives who bring a well-balanced perspective to the process. See Federal Advisory Committee Act, 5 U.S.C. App. 2 § (5)(b)(2), (c).
 OMB bulletin entitled "Final Information Quality Bulletin for Peer Review," 70 Fed. Reg. 2664-02 (Jan. 14, 2005), is applicable to all federal agencies, including EPA, and establishes government-wide guidance aimed at enhancing the practice of peer review of government science documents. The bulletin was subject to extensive public and agency comment on two prior draft versions. It includes guidance to federal agencies on what information is subject to peer review, the selection of appropriate peer reviewers, opportunities for public participation, and related issues. The bulletin also defines a peer review planning process that provides for public participation whenever possible and permits the public and scientific societies to comment about which scientific reports and studies merit especially rigorous peer review.
Commenter (1459) suggests that whether the data can be made publicly available is not as important as whether the data presented supports the conclusions and whether the experimental design and data analysis were free from error. The commenter states that, as such, the process of peer review by other experts in the field is a far more rigorous evaluation of a study's quality than the publish-ability of its data ever could be.
Commenter (6920) states that the current process is adequate. The commenter provides that the EPA currently uses robust, transparent processes to evaluate the best available scientific research, characterize the health hazards of chemicals and air pollution, and set standards to protect public health and the environmental. The commenter adds that EPA relies on peer-reviewed studies that describe the underlying data, methods, assumptions, sensitivity, and uncertainty of the results, and that these studies are cited and are published and available for public review. 
Commenter (1973) states that EPA already has in place a detailed review process, that, together with the information already made public about the data and methods used in each study, allows EPA and its external science advisory committees, to reach that judgment. 
Commenter (6925) states that the longstanding process of peer review includes an evaluation of the methodologies, assumptions, data, and inferences used in a study, which helps to ensure the appropriateness and validity of that study.
Commenters (2425, 4844) assert that scientific studies used by EPA to make regulatory changes are already rigorously examined prior to being published in peer-reviewed scientific journals. The commenters provide that EPA requires an IRB to review all scientific studies occurring in an academic institution prior to beginning the study. The commenters add that the IRB very closely reviews all study methods to ensure that the methods are sound and study subjects are protected. According to the commenters, once a study is complete, data involving human subjects must be kept for a minimum of five years after study completion and must be available for IRB review at all times. The commenters note that, in order to publish the data in a scientific journal, scientists not associated with the research study must review the study design to ensure that it is scientifically sound. The commenters add that this includes assessing for "good laboratory practices" and "standardized test methods." The commenters assert that, in this rigorous peer-review process, which is already required for studies to be considered by EPA, experts in the field of study not only assess the study's design and methods, but its statistical analysis and conclusions in an objective manner to determine the study's merit and significance. According to the commenters, if any of these areas are deemed to be unsatisfactory, the study will not be published, and that when studies are published, the authors must detail all methods and statistical tests used and this information is presented to the public. 
Commenter (5170) states that that there are already transparent procedures that include review of research designs and findings by independent scientific experts and making the studies available to the public. The commenter notes that published research findings are evaluated and debated by scientists, health professionals, scientific organizations, industry as well as the public to analyze the appropriate conclusions that may be drawn. The commenter adds that, where studies make use of private data (for example, medical histories), independent scientists review and evaluate the studies.
Commenters (6153, 6874) state that the scientific community has developed robust mechanisms and procedures for designing and conducting studies in ways that minimize bias, drawing appropriate conclusions from research findings, and advancing knowledge. The commenters note that peer review processes for funding are often highly competitive and involve critiques of study design that, if addressed, can lead to research with fewer potential biases or unmeasured confounders. The commenters contend that peer reviews for journal publication often involve critiques that lead to more nuanced discussions of a study's limitations and the conclusions that can be drawn given those limitations. 
Commenter (6111) asserts that there are ample and adequate safeguards in place at the leading journals to ensure "transparency"  -  the ability of other researchers to question, challenge, and validate the results of published studies. The commenter notes that this would include the journals' policies on treatment of data from research published years and even decades ago. Similarly, commenter (5180) asserts that the scientific studies published in scientific journals have already been critiqued and validated by experts familiar with the relevant scientific field as part of the scientific peer-review process.
Response: The EPA agrees with commenters that the scientific community and government agencies are making great strides in data transparency; however, improvements can still be made over existing policies and mechanisms. The requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
Further, the EPA would like to clarify that the rule requirements are not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review.The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review by either journals or independent advisory committees, such as the EPA's SAB, is typically conducted without access to the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information). Should peer reviewers have access to the underlying data, they could conduct a more in-depth review that may include independently reanalyzing the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty in the original study. Thus, peer review is not considered a replacement for the data availability requirements of this rule and implementation of this rule could provide independent peer reviewers as well as subject matter experts within the EPA and the public with the opportunity to evaluate pivotal science in ways that otherwise would not have typically occurred with existing EPA or peer review processes. 
The EPA is, therefore, finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate peer review, consistent with the OMB Bulletin for Peer Review and the EPA's Peer Review Handbook, of individual studies identified as pivotal science if the studies have already undergone journal peer review. If the Agency conducts peer review on pivotal science, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied in analyzing dose-response data and the implications of those assumptions for the results.
Existing Scientific Review Processes are Effective Even Without Availability of Raw Data
Joint Statement
Comment: Commenters (1306, 6110, 6122, 6876, 6909, 6915, 9227) refer to language in a Joint Statement by editors of several scientific journals and assert that effective peer review of scientific studies does not require that all underlying data be made public. 
Commenters (1306, 6122, 6876, 9227) generally concur with statements from the editors, including the following: 
 The merits of studies relying on data that cannot be made publicly available can still be judged.
 Reviewers can have confidential access to key data and as a core skill, scientists are trained in assessing research publications by judging the articulation and logic of the research design, the clarity of the description of the methods used for data collection and analysis, and appropriate citation of previous results.
Commenter (6110) states that the editors noted that many scientific journals already have policies to ensure transparency as much as possible and, in cases where such transparency is not possible, reviewers can be given confidential access to the raw data so that they can check and replicate the findings. The commenter asserts that the EPA proposal does not allow for such situations. 
Commenter (6122) states that, because they conduct their own research and extensive review of others' research, scientists do not need the raw data to determine if a study was conducted correctly, but rather are trained to do so by examining data collection and analysis methods and to determine whether the results can be replicated. The commenter notes that when doctors prescribe a medicine that they have deemed effective, it is not because they have reviewed and analyzed the raw data of patients in the clinical trials themselves; instead, they have reviewed the published papers on the medication and have seen both that the trials were done correctly and that the results were significant enough to prescribe the treatment. The commenter states that this is no different with research scientists. 
Response: The final rule does not categorically exclude studies based on the availability of the underlying dose-response data. The EPA would also like to clarify that the final rule does not require the EPA to collect, store, or publicly disseminate data and models underlying pivotal science. Further, EPA does not take the position that studies for which the underlying data are not publicly available are per se invalid. Rather, EPA acknowledges that where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty.
Further, the EPA would like to clarify that the rule requirements are not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review by either journals or independent advisory committees, such as the EPA's SAB, is typically conducted without access to the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information). Should peer reviewers have access to the underlying data, they could conduct a more in-depth review that may include independently reanalyzing the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty in the original study. Thus, peer review is not considered a replacement for the data availability requirements of this rule and implementation of this rule could provide independent peer reviewers as well as subject matter experts within the EPA and the public with the opportunity to evaluate pivotal science in ways that otherwise would not have typically occurred with existing EPA or peer review processes. 
The EPA is, therefore, finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate peer review, consistent with the OMB Bulletin for Peer Review and the EPA's Peer Review Handbook, of individual studies identified as pivotal science if the studies have already undergone journal peer review. If the Agency conducts peer review on pivotal science, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied in analyzing dose-response data and the implications of those assumptions for the results.
Study Validation is Addressed Through the Peer Review Process
Comment: Commenters (0569, 1481, 1793, 1796, 4590, 5173, 5180, 6103, 6109, 6357) assert that the design, methods, and assumptions of research studies are vitally important for validating scientific results and are typically addressed through peer review prior to publication. 
Commenters (0569, 4590, 5173) state that once the results of the research study are published, the scientific community and the public at large may respond to the study and challenge the premises, methodology, analyses, and/or conclusions. Commenters (0569, 4590, 5173) state that such challenges could include other research aimed at testing the validity of the findings. Commenters (0569, 4590, 5173) state that this process is one of the long-standing fundamental checks on the validity of scientific research. 
Commenter (5180) asserts that the scientific studies published in scientific journals have already been critiqued and validated by experts familiar with the relevant scientific field as part of the scientific peer-review process. 
Commenter (4590) states that current scientific practice recognizes that providing publicly available data and models is desirable, where doing so is consistent with researchers' legal and ethical obligations. The commenter notes that, in contrast, the proposed EPA rule requires public availability of data and models from scientific studies as the basis of decision-making, but does not recognize adequately either the value of peer review and subsequent research in addressing validity, the balancing of privacy concerns against public disclosure, or the importance of other context in evaluating research.
Commenter (1481) asserts that the appropriate use of statistics can be assessed as accurately and more efficiently from the published methodology and statistics than from the overwhelmingly large original data sets. The commenter states that open raw data sets offer no benefit to the public, since proper statistical methodology requires field specific expertise. The commenter adds that, thus, there is no useful transparency added to the studies but the policy comes at the cost of confidentiality. 
Commenter (1793) states that decision makers do not need to see raw data that went into studies in order to trust scientific evidence. The commenter notes that they can look at the methods, design, and results in order to assess the quality of the science. According to the commenter, this rule fundamentally misrepresents how science works. The commenter asserts that scientific studies already undergo intense scrutiny and peer-review from experts in the field before they are published in a scientific journal.
Commenter (4838) asserts that studies with confidential data can still be appropriately peer-reviewed through the use of confidentiality agreements, and subject to rigorous scientific scrutiny over their methods and conclusions. Commenter (1796) states that alternate methods exist to ensure that expert reviewers can evaluate the studies for methodological rigor, as well as for obtaining confidential access to data where necessary and appropriate. Commenter (4595) asserts that there are credible procedures for testing results and verifying outcomes with methodologies that do not require access to raw data. 
Commenter (6103) asserts that the value of any particular scientific study should be defined by the peer review process, not by whether the raw data are accessible. The commenter states that researchers have legal and ethical obligations to keep personal information private and businesses should not have to compromise their intellectual property. The commenter states that the public already has access to the science EPA relies on and can fully access methods, summary data, results, and interpretation as can all peer reviewers. The need for access to raw data is entirely unnecessary and would cause many problems by precluding the use of critical studies. 
Commenter (6109) states that, while EPA appears to suggest that in the absence of publicly available raw data, there is no mechanism to verify a study's validity, this is incorrect. The commenter states that the peer review process is rigorous and allows scientists to examine each other's work and analytical methodologies. The commenter states that, in addition, independent verification and confirmation by other studies can lend credence to studies based on confidential information.
Response: EPA does not take the position that studies for which the underlying data are not publicly available are per se invalid. Rather, EPA acknowledges that where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty.
Further, the EPA would like to clarify that the rule requirements are not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review by either journals or independent advisory committees, such as the EPA's SAB, is typically conducted without access to the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information). Should peer reviewers have access to the underlying data, they could conduct a more in-depth review that may include independently reanalyzing the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty in the original study. Thus, peer review is not considered a replacement for the data availability requirements of this rule and implementation of this rule could provide independent peer reviewers as well as subject matter experts within the EPA and the public with the opportunity to evaluate pivotal science in ways that otherwise would not have typically occurred with existing EPA or peer review processes. 
The EPA is, therefore, finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate peer review, consistent with the OMB Bulletin for Peer Review (Ref. 8), of individual studies identified as pivotal science if the studies have already undergone peer review. If the Agency conducts peer review on pivotal science, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied in analyzing dose-response data and the implications of those assumptions for the results As discussed in the preamble to the final rule, the rule requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Release of Raw Data (Including Privacy Data) is Not Required Under Existing Rigorous Peer Review Processes
Comment: Commenters (1461, 1973, 2910, 5418, 6103, 6111, 6922) contend that existing peer review processes that do not require the release of privacy data are sufficient.
Commenter (1461) states that in cases where data cannot be made available to the public, scientific journals and standards can maintain rigor by careful peer review, including allowing confidential access to key data to reviewers. Commenter (1973) asserts that the existing system has worked well and has provided the EPA with studies of mortality and hospital admissions studies from the same U.S. data by multiple investigating teams, while still maintaining data privacy. Commenter (2910) notes that pooled anonymous data has been used successfully for years to advance medical science. Commenter (5180) states that federal agencies like the EPA do not need to see the raw data that went into studies in order to trust the scientific evidence and consider it in their rulemaking.
Commenter (6111) asserts that the public health, medical, and scientific research community does not regard the public disclosure of all raw data as necessary. The commenter states that, for example, the Committee on Publication Ethics (COPE), which has over 12,100 member journals and editors covering all areas of scholarly inquiry, has established 10 core practices. The commenter provides that COPE's core practice #5 on data and reproducibility provides that "[j]ournals should include policies on data availability and encourage the use of reporting guidelines and registration of clinical trials and other study designs according to standard practice in their discipline." The commenter adds that the simplicity and generality of this core practice statement signals that the question of standards for data transparency, data access, data sharing, data peer review, and replication and reproducibility practices are far from settled. The commenter asserts that there is no one-size-fits-all approach to the critical questions of data transparency, data sharing, and reproducibility. 
Commenter (5418) notes that the approach advocated in the proposed rule is inconsistent with professional best practices in their respective disciplines for conducting, reviewing, and confirming the results/findings of studies, especially those based on confidential personal health data of study participants. The commenter notes that they publish their research results in the most reliable, highest-quality, peer-reviewed medical and scientific journals, including Lancet, Nature, Science, New England Journal of Medicine, Journal of the American Medical Association, Cell, and Environmental Health Perspectives and they conduct peer reviews of the work of other researchers. 
Commenters (6915, 9227) assert that, in a letter filed in this docket, the Presidents of the National Academies of Science, Engineering, and Medicine (NASEM) observe that the public availability of data is not necessary to ensure the integrity of regulatory science and is not a sufficient criterion for excluding a particular study from consideration. Commenter (9227) states the Presidents' letter notes: "The National Academies have developed a long-standing body of work that demonstrates scientific literature can be evaluated in a transparent and objective manner without complete disclosure of the underlying data." Commenter (9227) states the letter goes on to explain: "If the study data are not available, their absence may affect how the study is rated and used in the [agency's] analysis, but the study should not necessarily be eliminated from the assessment."
Commenter (6925) states that the complete public release of data is not necessary to ensure validity or transparency in science nor the regulatory process. The commenter notes that transparency has long been a requirement in science, with the understanding that the methodology is described in a manner that would allow the study to be reproduced. 
Response: The EPA clarified 40 CFR 30.5 to give equal consideration to dose-response data that are publicly available or available through restricted access. As discussed in the final rule, underlying dose-response data are considered available if they can be accessed through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that are unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator, as detailed under 40 CFR 30.7. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
The final rule is limited to dose-response data and does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. The final rule does not require the EPA to collect, store, or publicly disseminate the dose-response data underlying pivotal science. The EPA recognizes that researchers and other third parties may voluntarily consider the EPA's position on data availability, as described in this rule, as they make their own decisions about how to conduct research and the extent to which they make data available. Researchers may choose to make more data available, but the EPA recognizes that these parties will weigh their own benefits and costs and make choices that they deem appropriate. Further, EPA does not take the position that studies for which the underlying data are not publicly available are per se invalid. Rather, EPA acknowledges that where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty.
Further, the EPA would like to clarify that the rule requirements are not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review by either journals or independent advisory committees, such as the EPA's SAB, is typically conducted without access to the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information). Should peer reviewers have access to the underlying data, they could conduct a more in-depth review that may include independently reanalyzing the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty in the original study. Thus, peer review is not considered a replacement for the data availability requirements of this rule and implementation of this rule could provide independent peer reviewers as well as subject matter experts within the EPA and the public with the opportunity to evaluate pivotal science in ways that otherwise would not have typically occurred with existing EPA or peer review processes. 
The EPA is, therefore, finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate peer review, consistent with the OMB Bulletin for Peer Review and the EPA's Peer Review Handbook, of individual studies identified as pivotal science if the studies have already undergone journal peer review. If the Agency conducts peer review on pivotal science, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied in analyzing dose-response data and the implications of those assumptions for the results As discussed in the preamble to the final rule, the rule requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Radiation Protection Scientific Peer Review 
Comment: Commenter (6149) states that optimization of technical resources is critical in evaluating radiation protection scientific bases due to the limited availability of experts in the fields of radiobiology, radiation epidemiology, and radiation health physics. The commenter contends that the current system of review utilizing the National Academies, National Research Council, and National Council on Radiation Protection (NCRP) to leverage radiation protection expertise provides sufficient independent scientific review for regulatory action. The commenter states that, while the field of study is extensive and the breadth of regulatory impact is large, the number of experts in the fields of radiobiology, radiation epidemiology, and related health physics areas is extremely limited. The commenter notes that the National Academies, National Research Council, and NCRP have proven the capability to establish committees and provide qualified independent reviews of the scientific basis for radiation protection within this limited population of technical experts. According to the commenter, the EPA, nor any single agency, lacks adequate expertise of its own necessary to provide a meaningful independent review without engaging the resources available to National Academies, National Research Council, and NCRP. According to the commenter, these organizations are already performing this function, mandating an additional independent review by the EPA represents an additional regulatory burden without a commensurate benefit.
Response: As described in 40 CFR 30.5 of the final rule, the EPA will continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies -- based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements or quantitative analyses of an EPA significant regulatory action or influential scientific information will be identified as pivotal science. As described in preamble discussion of costs and benefits, the EPA recognizes there will be some incremental review and administrative costs to the Agency but anticipates these will be small.
Further, the EPA would like to clarify that the rule requirements are not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review by either journals or independent advisory committees, such as the EPA's Science Advisory Board, is typically conducted without access to the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information). Should peer reviewers have access to the underlying data, they could conduct a more in-depth review that may include independently reanalyzing the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty in the original study. Thus, peer review is not considered a replacement for the data availability requirements of this rule and implementation of this rule could provide independent peer reviewers as well as subject matter experts within the EPA and the public with the opportunity to evaluate pivotal science in ways that otherwise would not have typically occurred with existing EPA or peer review processes. 
The EPA is, therefore, finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate peer review, consistent with the OMB Bulletin for Peer Review and the EPA's Peer Review Handbook, of individual studies identified as pivotal science if the studies have already undergone journal peer review. If the Agency conducts peer review on pivotal science, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied in analyzing dose-response data and the implications of those assumptions for the results.
13.1.3.1.2 Studies Can Be Judged Without Data Being Made Public
Comment: Commenters (0040, 0569, 1433, 1879, 1943, 2477, 2906, 4845, 5173, 6098, 6111, 6133, 6152, 6188, 6194, 6446, 6909, 6922, 8281-PH26, 9227) do not agree with the proposal's premise that open access to data needs to be included in the Scientific Review process to validate the quality of a scientific study. Some of these commenters provide examples (e.g., American Cancer Society's (ACS) Cancer Prevention Study II) of reviewed and replicated study results that support their position. Commenters (1973, 2170, 5165, 6046, 6122, 6133) assert that key information about studies is already publicly available under existing peer review and that studies relying on data that cannot be made publicly available can still be judged with regards to the quality and transparency of the interpretation of findings with existing peer review protocols. Comments include:
 While the rule's premise is that availability of individual level data will improve the quality of the science, validation and analysis of data represent only one aspect of the overall criteria used to judge the credibility of science. (0569, 5173)
 Public access to the underlying data is not necessary in order to further EPA's proclaimed interests in ensuring the validity of the scientific information relied upon and "the public's ability to understand and meaningfully participate in the regulatory process." (6922)
 Transparency is not a key test of the quality and legitimacy of a scientific study. The commenter states that legitimacy and quality is determined through peer review, reliance on the modern scientific method, and robust collaboration and debate within the scientific community. The commenter contends that the fact that the underlying data cannot be made public does not impact the validity of the results of these peer-reviewed studies. (1433)
 While researchers rely heavily and critically on primary research papers, there is no issue whatsoever evaluating the science in those papers without having access to the primary data. (1879)
 The process of good science does not require all data to be available, just a good experimental design to test the hypotheses with a thorough review of the methods used to reach the conclusions. (2477)
 The scientific journals that have implemented data availability policies have found that mere availability of data, without documentation of the methods used to collect, process, and analyze such data, provides little to help the stated aims of promoting reproducibility or public involvement in policymaking. The commenter notes that interoperability standards for collection and reporting must first be agreed upon within the scientific community and the burden of making data available in such a way as to meaningfully advance the goals of the rule must be assessed. (2906)
 The unspoken premise of the proposal is that unless EPA can guarantee full public access to a study's underlying data, the study must be deemed unreliable and should play no role in assessing a pollutant or chemical's effects on human health. The commenter argues that this premise ignores the many ways in which the scientific community, regulators, and the public have traditionally determined the quality and relevance of study results. The commenter provides that study reports typically explain the protocols used to gather data, the methods used for data analysis, the doses or exposure concentrations at which effects were and were not observed, the nature, severity, and incidence of such effects, and any unusual occurrences that may affect interpretation of the results. According to the commenter, this information plays an important role in the peer review process, informing the judgment of independent reviewers as to whether a study is worthy of publication in the scientific literature. The commenter states that Agency reviewers likewise consider these indicators of reliability in deciding how much weight a study deserves in making judgments about hazard and risk. The commenter states that, in its narrow focus on a single criterion for study acceptability, the proposal departs from this comprehensive, multi-faceted approach for determining the "best available science" to inform decision-making. (4845)
 Verification of method requires collection of new data, not use of the raw data. The commenter states that independent studies test the same method across many data sets. The commenter asserts that when you validate a protocol you are not testing the raw data within the system you are testing the system for accuracy using new data obtained in the same manner. (6098)
 The professional public health, medical, and scientific research community have long recognized that whether or not the raw data underlying a study is released does not determine the quality of the study. The commenter states that, rather, it is the scientific method that is determinative. The commenter adds that the proposed rule fails to take into account the fact that studies are reliable and constitute the best available science when they comply with professionally-established best practices for describing the methodology, sampling size, sampling procedure and assumptions utilized and the results are consistent with those of other studies. (6111)
 The proposed rule would replace EPA's existing successful process, which is in step with established scientific procedures, with one driven by the public availability of data, and unconcerned with well-established hallmarks of reliable scientific research, like peer review requirements. The commenter argues that this emphasis is inappropriate, given the courts' agreement that publicizing the data underlying studies upon which EPA relies "would be impractical and unnecessary." The commenter contends that the proposed rule would reduce, not enhance, the quality of data available to EPA as it considers important regulatory actions. (6188)
 EPA proposed this rule under the false premise that only scientific studies for which all underlying data is made publicly available can be adequately evaluated. The commenter states that this is simply not the case, and any discussion about mandating transparency through a bright line rule such as the one currently being proposed should be grounded in this fact. The commenter adds that, in addition, where necessary and appropriate, reviewers (as well as other researchers seeking to reproduce or extend scientific analysis) can have confidential access to key data in conformity with privacy requirements. (6909)
 The Agency's arbitrary, single-minded focus on considering studies for which certain data and models are publicly available (but only the dose-response studies relevant to health protective regulation, not the ones supporting registration of chemicals) stands in stark contrast to the way the scientific community validates research findings. The commenter states that the scientific community, and scientific journals look to a range of attributes when assessing the quality of a scientific study, including whether the study has been peer reviewed, whether the scientists used rigorous scientific methods, and whether the study's results have been reproduced or replicated. The commenter states that the scientific community recognizes that the quality of a study is not determined by whether the underlying data is publicly available and has long utilized a variety of tools for ensuring the integrity and rigor of research findings. (9227)
 The proposal fails to recognize is that disclosure of data addresses only one method of validating scientific research -- and a relatively less important aspect at that. According to the commenter, disclosure of data for a given study -- the focus of the proposal -- permits independent researchers to determine whether the data and methodology used in that study can be applied to generate the same results. The commenter notes that this may help protect against sources of error or misrepresentation in a particular study, however, both EPA and independent researchers have recognized that such reanalysis does not by itself validate a particular study. The commenter provides that, rather, a study's evidentiary weight rests both on the strength of its methodology, as well as whether similar results can be obtained by applying the study's methodology to a relevant, but different dataset or population, or by using a distinct methodology to interrogate the same hypothesis. (9227)
 Commenters (1973, 6122) assert that scientists already include model selection, assumptions, and uncertainty in work that is peer reviewed. Commenter (1973) adds that studies that have undergone the peer review process required for journal publication have also had to address comments based on review and comment by a group of experts unrelated to the study. Commenter (6122) asserts that the main point of peer review that is addressed under current peer review mechanisms is to validate the methodology and ensure results were interpreted correctly under the given assumptions. 
 Commenters (2170, 6133) cite the following from a Science magazine article in support of their position that the merits of studies relying on data that is not publicly available can still be judged:
 Importantly, the merits of studies relying on data that cannot be made publicly available can still be judged. Reviewers can have confidential access to key data and as a core skill, scientists are trained in assessing research publications by judging the articulation and logic of the research design, the clarity of the description of the methods used for data collection and analysis, and appropriate citation of previous results.
 Peer-review can and does occur without the study data being made public, including confidential data whereby the researcher can work with the peer-reviewers to provide datasets with study participant confidentiality ensured. (6046)
 While many peer-reviewed scientific journals have recently adopted policies that support data sharing, these standards recognize the array of workflows across scientific fields and make the case for data sharing at different levels of stringency that consider exceptional circumstances, such as data sets featuring personal identifiers and other data that cannot be shared openly. (6133)
 The scientific community has already adopted several protocols and guidelines specifically designed to improve the reporting and evaluation of studies and to improve the quality and transparency of the interpretation of findings. According to the commenter, these protocols and guidelines, such as CONSORT, ARRIVE and STROBE, do not require public access to all study data but still improve the scientific basis of evaluating studies. (1973)
 The following ESCAPE study as an example where the peer review process is completely transparent, without the need for privacy violations. (1973)
 For example, the ESCAPE study consisted of 22 cohorts for which the methodology was well documented in papers describing how the recruitment was done, who was eligible to be recruited, who was not eligible to be recruited, how many people were recruited, where the recruitment was done, what kind of data were collected on the people, including medical data and data on risk factors for disease, and they have all produced detailed summary statistics of those data, and their correlations. Separate papers describe in detail how the land use regression models that produced the exposure were fit, what the protocol was for collecting monitoring data, locating the monitors, what land use variables were used and where they were obtained from, etc. The statistical methods for fitting the land use regressions, and what variables were used were similarly described in detail. The papers that matched these two sets of data to examine the association of air pollution with health also described which data were used, how they were used, and what statistical models were fit. This is a completely transparent process, without the need for privacy violations. The same is true for the other cohort studies that EPA has relied on. They are perfectly transparent as to the nature of the population studied, how the recruitment was done, what data was collected on the individuals, and how the analysis was done. 
Response: EPA does not take the position that studies for which the underlying data are not publicly available are per se invalid. Rather, EPA acknowledges that where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty.
Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that are unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator, as detailed under 40 CFR 30.7. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
As discussed in the preamble to the final rule, the requirements of the final rule build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. 
This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. The EPA recognizes that researchers and other third parties may voluntarily consider the EPA's position on data availability, as described in this rule, as they make their own decisions about how to conduct research and the extent to which they make data and models available. Researchers may choose to make more data and models available, but the EPA recognizes that these parties will weigh their own benefits and costs and make choices that they deem appropriate.
The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information.
Comment: Commenter (6922) notes that, for decades, EPA has relied upon epidemiological studies to help determine what pollutants or chemicals need to be regulated, and how rigorously to do so. The commenter states that the EPA's reliance on those studies was reasonable  -  even though the underlying data was not public  -  because the studies underwent peer review to ensure these studies and their results were sound, replicable, and reliable. In other words, the commenter asserts that not having the raw data did not, and does not, make the science unsound. To the contrary, the commenter contends that the studies that include or rely on confidential health information often are among the best science that the Agency has to evaluate the effects of pollutant exposure on humans, and the costs imposed on individuals and society as a result of that exposure. Similarly, the commenter provides that the EPA routinely relies on data provided by regulated industries that is treated as confidential business information (CBI) and is therefore protected from public disclosure because it may include customer or other personally-identifiable information, including electric and gas consumption. According to the commenter, this is no different from relying on public health studies that rely on data that is protected from disclosure for various reasons. 
Response: EPA does not take the position that studies for which the underlying data are not publicly available are per se invalid. Rather, EPA acknowledges that where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty. As discussed in the preamble to the final rule, the requirements of the final rule build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. In addition, the EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (6133) states that the proposed rule does not require that any information actually be independently validated before EPA may consider it or base regulatory decisions on such verification. Accordingly, there is an irrational disconnect between EPA's insistence that information be "publicly available for independent validation" and the proposal's claim that this ensures EPA will consider and use the "best available science." The commenter notes that the EPA itself has not outlined a process by which "dose response data and models" would be validated, and the proposal does not seriously consider the methodological complications of partial redaction of underlying study data.
Response: As discussed in the preamble of the final rule, where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Comment: Commenter (6194) states that, where EPA alleges that the proposal is consistent with the focus on transparency in these and other laws, regulations, and policies, the Agency is really conflating standards of quality with public dissemination of data. The commenter states that, rather than recommend foregoing the use of the information based on confidential information, OMB provides agencies mechanisms for validating information when data are not available. The commenter states that, when it comes to its own research, EPA "strives to increase access to its research results" but recognizes that, "[w]hether research data are fully available to the public or available to researchers through other means does not affect the validity of the scientific conclusions from peer-reviewed research publications." 
Response: EPA does not take the position that studies for which the underlying data are not publicly available are per se invalid. Rather, EPA acknowledges that where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty.
In addition, the EPA clarified 40 CFR 30.5 to give equal consideration to dose-response data that are publicly available or available through restricted access. As discussed in the final rule, underlying dose-response data are considered available if they can be accessed through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that are unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator, as detailed under 40 CFR 30.7. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
This rule build[s] upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. As discussed more fully in the preamble of the final rule, the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
Comment: Commenter (9227) asserts that the absence of publicly available underlying data does not make the results of a study invalid or even suggest that the study is likely to be invalid, nor has EPA presented evidence to suggest that studies with publicly available underlying data are more likely to represent strong science than studies without such data availability. The commenter states that key reasons why researchers do not make data for some studies publicly available have nothing to do with scientific quality. The commenter states that, while reanalyzing study results using the same data is one way to help validate those results, it is neither the primary nor a sufficient way to do so. The commenter states that EPA's apparent conflation of data availability and best available science is not based on any evidence cited by EPA, is contrary to the evidence before EPA, and is simply arbitrary. The commenter provides the following additional information regarding data availability and study quality:
   EPA's Preamble to the Integrated Science Assessments provides another discussion of how EPA evaluates study quality, and similarly, does not call out publicly available data:
 [T]he individual study quality is evaluated by considering the design, methods, conduct, and documentation of each study, but not the study results. This uniform approach aims to consider the strengths, limitations, and possible roles of chance, confounding, and other biases that may affect the interpretation of individual studies and the strength of inference from the results of the study.
 A statement by the American Statistical Association on p-Values: Context, Process, and Purpose further emphasizes the multiple considerations related to quality, stating "Researchers should bring many contextual factors into play to derive scientific inferences, including the design of a study, the quality of the measurements, the external evidence for the phenomenon under study, and the validity of assumptions that underlie the data analysis." Similarly, the letter filed by the Presidents of the NASEM in this docket lists multiple reports conducted since 2007 that have examined EPA's scientific assessment processes and "that advise EPA on the scientific bases of regulatory decisions related to human health and the environment." According to the NASEM Presidents, These reports encourage EPA to consider all available science in the rule-making process and provide guidance about how the agency could be more transparent in describing how evidence is gathered and evaluated. . . . Individual study quality should be evaluated based on information that is available in standard journal articles, such as the study design elements, analytical techniques, and statistical methods. Researchers may be contacted to answer questions about the conduct of the study or be asked to provide additional data. If the study data are not available, their absence may affect how the study is rated and used in the analysis, but the study should not necessarily be eliminated from the assessment.
 The commenter asserts that the proposal conflates transparency with quality, which is inconsistent with OMB's Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies, which does not consider transparency an absolute requirement, provides that the objectivity of information that cannot be made publicly available can be verified through other types of "robustness checks," and provides an example of the Harvard Six Cities Study where reanalysis was achieved even when underlying data could not be made publicly available.
Response: This rulemaking is designed to build upon open data initiatives in the federal government and scientific community and advances the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. EPA does not take the position that studies for which the underlying data are not publicly available are per se invalid. Rather, EPA acknowledges that where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. 
As discussed more fully in the preamble of the final rule, the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
13.2.3.2 Consistency with Existing Peer Review Policies
 13.2.3.2.1 Proposed Rule is Not Aligned with Existing Review Policies
Comment: Commenters (0559, 6117, 6375, 6464, 9227) express concern that the proposal is not aligned with existing policies.
Commenter (9227) states that EPA's Peer Review Handbook acknowledges that "EPA strives to ensure that the scientific and technical bases of its decisions meet two important criteria: (1) they are based upon the best current knowledge from science, engineering, and other domains of technical expertise; and (2) they are credible." The commenter states that this policy clearly impacts EPA's regulatory actions, and thus will be impacted by the Proposal. According to the commenter, the EPA fails to analyze the impact the Proposal will have on its ability to comply with this policy and fails to explain why it is changing course or justify its decision to do so. 
Commenter (6375) suggests that EPA consider updating and utilizing its own peer-review policy in addition to that provided in the peer-reviewed scientific literature. The commenter suggests that, for proprietary data/CBI that are submitted to EPA to support new chemical registrations and other regulatory actions, EPA has full data access from which to conduct its own peer reviews in accordance with its own policy. 
Commenters (0559, 6464) express that policies should be consistent with standards for determining accepted science that is used by the scientific community and other research agencies. Commenter (6464) states that establishing a different and arbitrary set of standards, would be overstepping the federal government's role. 
Response: As discussed in the preamble to the final rule,  the rule requirements are not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review by either journals or independent advisory committees, such as the EPA's Science Advisory Board, is typically conducted without access to the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information). Should peer reviewers have access to the underlying data, they could conduct a more in-depth review that may include independently reanalyzing the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty in the original study. Thus, peer review is not considered a replacement for the data availability requirements of this rule and implementation of this rule could provide independent peer reviewers as well as subject matter experts within the EPA and the public with the opportunity to evaluate pivotal science in ways that otherwise would not have typically occurred with existing EPA or peer review processes. 
The EPA is, therefore, finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate peer review, consistent with the OMB Bulletin for Peer Review and the EPA's Peer Review Handbook, of individual studies identified as pivotal science if the studies have already undergone journal peer review. If the Agency conducts peer review on pivotal science, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied in analyzing dose-response data and the implications of those assumptions for the results.
13.2.3.2.2 Requirements of this Rule as Compared to Peer Review in Environmental Statutes
Comment: Commenter (6133) states that under longstanding federal case law, when Congress authorizes an approach in one section of a statute using specific language but does not do the same in other sections of a statute, courts presume that Congress acted purposefully and did not mean to address or authorize that approach in those other statutory sections. See, e.g., Dean v. United States, 556 U.S. 568 (2009) ("It is generally presumed that Congress acts intentionally when including particular language in one section of a statute but not in another." (citing Russello v. United States, 464 U.S. 16, 23 (1983)). According to the commenter, not only are there no implied grants of authority to an agency in the other statutory sections, but the congressional decisions to authorize the approach elsewhere in the statute give even greater force to the conclusion that the agency has not been given authority where Congress did not use the same or similar authorizing language. 
Commenter (6133) asserts that the EPA approach proposed in §30.7 is even more unlawful than would be the case, independently, under this case law. The commenter provides that the proposed §30.7 says "EPA shall conduct independent peer review on all pivotal regulatory science used to justify regulatory decisions, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664) and the exemptions described therein." 83 Fed. Reg. at 18,774/2. The commenter adds that the EPA proposes to bind itself ("shall") to conduct a particular form of peer review on "all pivotal regulatory science" based on an unenforceable, non-binding OMB Final Information Quality Bulletin for Peer Review that has never been the subject of notice and comment rulemaking and that itself is not authorized by any federal law. To the contrary, the commenter contends that treating the content of the unenforceable bulletin as a binding regulation would itself violate the federal statutes that EPA implements, because those statutes do not codify the bulletin, and EPA would be unlawfully codifying a mere policy preference. 
Commenter (6133) asserts that the Proposal identifies no statutory authority for EPA to conduct independent peer review for any, much less all, "pivotal regulatory science" consistent with the dense content (and exceptions) of the OMB bulletin. The commenter provides that the Proposal identifies no statutory authority for EPA to bind itself, and future administrations, to conduct peer review only in this fashion, unless and until future notice-and-comment rulemaking is undertaken. The commenter further provides that the Proposal identifies no suggestion in statutory language or legislative history that Congress intended EPA to conduct binding peer review consistent with this OMB bulletin, notwithstanding that Congress has known about this bulletin since 2005. 
Commenter (6133) states that when Congress writes statutes it knows how to create the legal authority for peer review and identify who shall conduct that peer review; including what role, if any, that EPA or other parties will play, and how that peer review may be conducted. The commenter asserts that not only do no provisions in the law authorize EPA's Proposal in §30.7, but existing environmental statutes require that EPA use peer-reviewed science, regardless of whether it would meet EPA's definition of the term in §30.7. 
Commenter (6133) provides the following examples in support of their assertion:
 CAA.
      §7511b. Federal ozone measures.
      [...]
      (g) Ozone design value study. The Administrator shall conduct a study of whether the methodology in use by the Environmental Protection Agency as of November 15, 1990, for establishing a design value for ozone provides a reasonable indicator of the ozone air quality of ozone nonattainment areas. The Administrator shall obtain input from States, local subdivisions thereof, and others. The study shall be completed and a report submitted to Congress not later than 3 years after November 15, 1990. The results of the study shall be subject to peer and public review before submitting it to Congress. 
      42 U.S.C. §7511b.
      §7412. Hazardous air pollutants. 
      [...]
      (p) Mickey Leland National Urban Air Toxics Research Center. 
      [...]
      (3) Scientific Advisory Panel. The Board of Directors shall be advised by a Scientific Advisory Panel, the 13 members of which shall be appointed by the Board, and to include eminent members of the scientific and medical communities. The Panel membership may include scientists with relevant experience from the National Institute of Environmental Health Sciences, the Center for Disease Control, the Environmental Protection Agency, the National Cancer Institute, and others, and the Panel shall conduct peer review and evaluate research results. The Panel shall assist the Board in developing the research agenda, reviewing proposals and applications, and advise on the awarding of research grants. 
      42 U.S.C. §7412.
 CWA.
      §1321. Oil and hazardous substance liability. 
      (a) Definitions. 
      [...]
      (27) the term "best available science" means science that-- (A) maximizes the quality, objectivity, and integrity of information, including statistical information; (B) uses peer-reviewed and publicly available data; and (C) clearly documents and communicates risks and uncertainties in the scientific basis for such projects; 
      33 U.S.C. §1321. 
 SDWA.
      §300g-1. National drinking water regulations. 
      [...]
      (b) Standards. 
      [...]
      (3) Risk assessment, management, and communication.
      (A) Use of science in decision-making. In carrying out this section, and, to the degree that an Agency action is based on science, the Administrator shall use--
      (i) the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices; and(ii) data collected by accepted methods or best available methods (if the reliability of the method and the nature of the decision justifies use of the data). 
      (B) Public information. In carrying out this section, the Administrator shall ensure that the presentation of information on public health effects is comprehensive, informative, and understandable. The Administrator shall, in a document made available to the public in support of a regulation promulgated under this section, specify, to the extent practicable--
      (i) each population addressed by any estimate of public health effects;
      (ii) the expected risk or central estimate of risk for the specific populations;
      (iii) each appropriate upper-bound or lower-bound estimate of risk;
      (iv) each significant uncertainty identified in the process of the assessment of public health effects and studies that would assist in resolving the uncertainty; and
      (v) peer-reviewed studies known to the Administrator that support, are directly relevant to, or fail to support any estimate of public health effects and the methodology used to reconcile inconsistencies in the scientific data.
      [...]
      (12) Certain contaminants.
      [...]
      (B) Sulfate.
      (i) Additional study. Prior to promulgating a national primary drinking water regulation for sulfate, the Administrator and the Director of the Centers for Disease Control and Prevention shall jointly conduct an additional study to establish a reliable dose-response relationship for the adverse human health effects that may result from exposure to sulfate in drinking water, including the health effects that may be experienced by groups within the general population (including infants and travelers) that are potentially at greater risk of adverse health effects as the result of such exposure. The study shall be conducted in consultation with interested States, shall be based on the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices, and shall be completed not later than 30 months after August 6, 1996. 
      42 U.S.C. §300g-1. 
      §300j-2. Grants for State programs.
      [...] 
      (d) New York City watershed protection program.
      (1) In general The Administrator is authorized to provide financial assistance to the State of New York for demonstration projects implemented as part of the watershed program for the protection and enhancement of the quality of source waters of the New York City water supply system, including projects that demonstrate, assess, or provide for comprehensive monitoring and surveillance and projects necessary to comply with the criteria for avoiding filtration contained in 40 C.F.R. 141.71. Demonstration projects which shall be eligible for financial assistance shall be certified to the Administrator by the State of New York as satisfying the purposes of this subsection. In certifying projects to the Administrator, the State of New York shall give priority to monitoring projects that have undergone peer review. 
      42 U.S.C. §300j-2.
 RCRA.
      §6939a. Exposure information and health assessments.
      [...]
      (b) Health assessments.
      (2) Whenever in the judgment of the Administrator, or the State (in the case of a State with an authorized program), a landfill or a surface impoundment poses a substantial potential risk to human health, due to the existence of releases of hazardous constituents, the magnitude of contamination with hazardous constituents which may be the result of a release, or the magnitude of the population exposed to such release or contamination, the Administrator or the State (with the concurrence of the Administrator) may request the Administrator of the Agency for Toxic Substances and Disease Registry to conduct a health assessment in connection with such facility and take other appropriate action with respect to such risks as authorized by section 9604(b) and (i) of this title. If funds are provided in connection with such request the Administrator of such Agency shall conduct such health assessment.
      [...]
      (e) Periodic reports. The Administrator of such Agency shall issue periodic reports which include the results of all the assessments carried out under this section. Such assessments or other activities shall be reported after appropriate peer review. 
      42 U.S.C. §6939a. 
 CERCLA.
      §9604. Response authorities.
      [...]
      (i) Agency for Toxic Substances and Disease Registry; establishment, functions, etc.
      [...]
      Any toxicological profile or revision thereof shall reflect the Administrator of ATSDR's assessment of all relevant toxicological testing which has been peer reviewed. The profiles required to be prepared under this paragraph for those hazardous substances listed under subparagraph (A) of paragraph (2) shall be completed, at a rate of no fewer than 25 per year, within 4 years after October 17, 1986. A profile required on a substance listed pursuant to subparagraph (B) of paragraph (2) shall be completed within 3 years after addition to the list. The profiles prepared under this paragraph shall be of those substances highest on the list of priorities under paragraph (2) for which profiles have not previously been prepared. Profiles required under this paragraph shall be revised and republished as necessary, but no less often than once every 3 years. Such profiles shall be provided to the States and made available to other interested parties.
      [...]
      (7)(A) Whenever in the judgment of the Administrator of ATSDR it is appropriate on the basis of the results of a health assessment, the Administrator of ATSDR shall conduct a pilot study of health effects for selected groups of exposed individuals in order to determine the desirability of conducting full scale epidemiological or other health studies of the entire exposed population. (B) Whenever in the judgment of the Administrator of ATSDR it is appropriate on the basis of the results of such pilot study or other study or health assessment, the Administrator of ATSDR shall conduct such full scale epidemiological or other health studies as may be necessary to determine the health effects on the population exposed to hazardous substances from a release or threatened release. If a significant excess of disease in a population is identified, the letter of transmittal of such study shall include an assessment of other risk factors, other than a release, that may, in the judgment of the peer review group, be associated with such disease, if such risk factors were not taken into account in the design or conduct of the study.
      [...]
      (13) All studies and results of research conducted under this subsection (other than health assessments) shall be reported or adopted only after appropriate peer review. Such peer review shall be completed, to the maximum extent practicable, within a period of 60 days. In the case of research conducted under the National Toxicology Program, such peer review may be conducted by the Board of Scientific Counselors. In the case of other research, such peer review shall be conducted by panels consisting of no less than three nor more than seven members, who shall be disinterested scientific experts selected for such purpose by the Administrator of ATSDR or the Administrator of EPA, as appropriate, on the basis of their reputation for scientific objectivity and the lack of institutional ties with any person involved in the conduct of the study or research under review. Support services for such panels shall be provided by the Agency for Toxic Substances and Disease Registry, or by the Environmental Protection Agency, as appropriate. 
      42 U.S.C. §9604. 
 EPCRA.
      No mentions of peer review. 
 FIFRA.
      §136w. Authority of Administrator. 
      [...]
      (e) Peer review. The Administrator shall, by written procedures, provide for peer review with respect to the design, protocols, and conduct of major scientific studies conducted under this subchapter by the Environmental Protection Agency or by any other Federal agency, any State or political subdivision thereof, or any institution or individual under grant, contract, or cooperative agreement from or with the Environmental Protection Agency. In such procedures, the Administrator shall also provide for peer review, using the advisory panel established under subsection (d) of this section or appropriate experts appointed by the Administrator from a current list of nominees maintained by such panel, with respect to the results of any such scientific studies relied upon by the Administrator with respect to actions the Administrator may take relating to the change in classification, suspension, or cancellation of a pesticide. Whenever the Administrator determines that circumstances do not permit the peer review of the results of any such scientific study prior to the Administrator's exercising authority under section 136d(c) of this title to immediately suspend the registration of any pesticide to prevent an imminent hazard, the Administrator shall promptly thereafter provide for the conduct of peer review as provided in this sentence. The evaluations and relevant documentation constituting the peer review that relate to the proposed scientific studies and the results of the completed scientific studies shall be included in the submission for comment forwarded by the Administrator to the advisory panel as provided in subsection (d). As used in this subsection, the term "peer review" shall mean an independent evaluation by scientific experts, either within or outside the Environmental Protection Agency, in the appropriate disciplines. 
7 U.S.C. §136w.
      §136w-8. Pesticide registration service fees. 
      (a) Definition of costs. In this section, the term "costs", when used with respect to review and decision-making pertaining to an application for which registration service fees are paid under this section, means -- 
      (1) costs to the extent that -- 
      (A) officers and employees provide direct support for the review and decision-making for covered pesticide applications, associated tolerances, and corresponding risk and benefits information and analyses; 
      (B) persons and organizations under contract with the Administrator engage in the review of the applications, and corresponding risk and benefits information and assessments; and 
      (C) advisory committees and other accredited persons or organizations, on the request of the Administrator, engage in the peer review of risk or benefits information associated with covered pesticide applications; 
      (2) costs of management of information, and the acquisition, maintenance, and repair of computer and telecommunication resources (including software), used to support review of pesticide applications, associated tolerances, and corresponding risk and benefits information and analyses; and 
      (3) costs of collecting registration service fees under subsections (b) and (c) and reporting, auditing, and accounting under this section. 
      7 U.S.C. §136w-8.
 Toxics Substances Control Act (TSCA).
      §2625. Administration.
      [...]
      (h) Scientific standards. In carrying out sections 2603, 2604, and 2605 of this title, to the extent that the Administrator makes a decision based on science, the Administrator shall use scientific information, technical procedures, measures, methods, protocols, methodologies, or models, employed in a manner consistent with the best available science, and shall consider as applicable -- 
      (1) the extent to which the scientific information, technical procedures, measures, methods, protocols, methodologies, or models employed to generate the information are reasonable for and consistent with the intended use of the information; 
      (2) the extent to which the information is relevant for the Administrator's use in making a decision about a chemical substance or mixture; 
      (3) the degree of clarity and completeness with which the data, assumptions, methods, quality assurance, and analyses employed to generate the information are documented;
      (4) the extent to which the variability and uncertainty in the information, or in the procedures, measures, methods, protocols, methodologies, or models, are evaluated and characterized; and 
      (5) the extent of independent verification or peer review of the information or of the procedures, measures, methods, protocols, methodologies, or models. 
      15 U.S.C. §2625.
      §2617. Preemption.
      [...]
      (b) New statutes, criminal penalties, or administrative actions creating prohibitions or other restrictions. 
      (1) In general. Except as provided in subsections (c), (d), (e), (f), and (g), beginning on the date on which the Administrator defines the scope of a risk evaluation for a chemical substance under section 2605(b)(4)(D) of this title and ending on the date on which the deadline established pursuant to section 2605(b)(4)(G) of this title for completion of the risk evaluation expires, or on the date on which the Administrator publishes the risk evaluation under section 2605(b)(4)(C) of this title, whichever is earlier, no State or political subdivision of a State may establish a statute, criminal penalty, or administrative action prohibiting or otherwise restricting the manufacture, processing, distribution in commerce, or use of such chemical substance that is a high-priority substance designated under section 2605(b)(1)(B)(i) of this title.
      [...]
      (f) Waivers. 
      [...]
      (2) Required exemptions. Upon application of a State or political subdivision of a State, the Administrator shall exempt from subsection (b) a statute or administrative action of a State or political subdivision of a State that relates to the effects of exposure to a chemical substance under the conditions of use if the Administrator determines that -- 
      (A)(i) compliance with the proposed requirement of the State or political subdivision of the State would not unduly burden interstate commerce in the manufacture, processing, distribution in commerce, or use of a chemical substance; 
      (ii) compliance with the proposed requirement of the State or political subdivision of the State would not cause a violation of any applicable Federal law, rule, or order; and 
      (iii) the State or political subdivision of the State has a concern about the chemical substance or use of the chemical substance based in peer-reviewed science; or 
      (B) no later than the date that is 18 months after the date on which the Administrator has initiated the prioritization process for a chemical substance under the rule promulgated pursuant to section 2605(b)(1)(A) of this title, or the date on which the Administrator publishes the scope of the risk evaluation for a chemical substance under section 2605(b)(4)(D) of this title, whichever is sooner, the State or political subdivision of the State has enacted a statute or proposed or finalized an administrative action intended to prohibit or otherwise restrict the manufacture, processing, distribution in commerce, or use of the chemical substance. 
      15 U.S.C. §2617.
      §2605. Prioritization, risk evaluation, and regulation of chemical substances and mixtures.
      [...]
      (b) Risk evaluations. 
      [...]
      (E) Metals and metal compounds. In identifying priorities for risk evaluation and conducting risk evaluations of metals and metal compounds, the Administrator shall use the Framework for Metals Risk Assessment of the Office of the Science Advisor, Risk Assessment Forum, and dated March 2007, or a successor document that addresses metals risk assessment and is peer reviewed by the Science Advisory Board. 
      15 U.S.C. §2605. 
Response: EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. If implementing this procedural rule would result in conflicts with existing environmental statutes, and their implementing regulations, this rule will yield to the EPA statutes and regulations.
Comment: Commenter (1981) states that, where the issue of raw data collection has been raised, the courts have permitted the EPA to use studies like the fish surveys. The commenter notes that in Coal of Battery Recyclers, the United States Court of Appeals for the District of Columbia Circuit held that the EPA could use non-public data in order to support its regulation of the amount of lead in ambient air. The commenter adds that, in another case using non-public data, epidemiological studies protected elderly Americans from cardiopulmonary disease caused by a contaminated respiratory environment. The commenter provides that the Uinta Basin ozone concentrations in Colorado were found by the EPA to exceed the NAAQS on the basis of non-public data. 
Commenter (1981) states that, occasionally, studies use data that is not publicly available, often because the research focuses on medical issues and patient privacy is protected under other federal laws. The commenter provides that this research is reviewed by experts in the field before it is ever considered for policy development. The commenter states that it is both pragmatic and efficient to recognize that the EPA delegates review of raw data to experts. The commenter states that it was for this reason that in Coal of Battery Recyclers, the court made a distinction between the EPA's "reliance on a study's results from its reliance on the raw data underlying such results." The commenter contends that the proposed rule unrealistically suggests that the EPA cannot and will not utilize the peer review that it has delegated in cases like Battery. 
Response: The rule does not preclude EPA from considering studies with non-publicly available data, so it does not conflict with those decisions.
The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted access in this rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. This rule does not require the EPA to disclose or host data, but to determine if data and models are available and to give greater consideration to those studies for which data and models are available. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Further, although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science are unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, a study with underlying dose-response data that are not available should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. The EPA could also give pivotal science with underlying dose-response data that are unavailable for independent validation full consideration upon receiving an exemption from the requirements of this rule from the Administrator, as detailed under 40 CFR 30.7. One of the criteria the Administrator may use in his/her determination to grant a waiver is that making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, CBI, or national security. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
Comment: Commenter (6194) states that EPA's own SAB, which Congress created in 1978 to provide scientific advice to EPA, 42 U.S.C. § 4365(a), challenged EPA's failure to solicit SAB's input on the proposal -- pointing out that it was not even aware of the proposal until it was published in the Federal Register and discussed in news articles. According to the commenter, given that EPA should have provided the proposal to another federal agency to review, EPA was also required to make the proposal available to the SAB, along with any relevant scientific and technical information on which the proposal was based. 42 U.S.C. § 4365(c). 
Commenter (6916) asserts that EPA failed to confer with its chartered SAB, and seek its input, as required under the Environmental Research Development and Demonstration Authorization Act (ERDDAA), 42 U.S.C. § 4365(c)(1), even though EPA claims authority under the CAA, and admits in the proposal that it is a "significant regulatory action" submitted to OMB for review. The commenter states that, as EPA's SAB recently wrote to the Administrator, a proposal like this, which "focus[es] on the EPA's foundational policies related to the use of science in rulemaking and policy development," is within the SAB's purview and should have been submitted to it for review." The commenter states that EPA's failure even to submit the proposal to SAB review, never mind take account of the SAB review results, is evidence of the arbitrary and capricious nature of this whole undertaking. 
Response: The EPA received comments from the SAB on the 2018 NPRM and 2020 SNPRM on April 24, 2020. The EPA considered these comments (EPA-SAB-20-005) in development of the final rule. In brief, the SAB consensus was that it supported the precept of enhancing public access to scientific data and analytical methods to ensure scientific integrity, consistency, and robust analysis and their consensus recommendations were that the EPA develop additional policy and/or guidance documents; further describe how pivotal science would be selected, made available, and peer reviewed; consider establishing an office on data sharing with the support of a peer review panel; provide greater clarity in the definitions of key terms, including data, independent validation, and publicly available; develop specific criteria for the Administrator's exemptions under the rule; consider applying rule requirements only to information developed after the effective date of the rule; seeking input from experts to identify best practices to ensure efficiency, utility, and protection of data that are made available; and consider funding reanalysis work. 
As noted in the preamble for the final rule, this rule sets the overarching structure and principles for transparency of pivotal science in significant regulatory actions and influential scientific information and technical considerations, including those noted by the SAB, could be addressed in subsequent implementation guidelines and/or statute-specific rulemakings. Further, in response to comments from SAB and the public, the EPA clarified and modified several key definitions, including data, independent validation, and publicly available, in the final rule. The EPA also provided criteria for the Administrator's exemption in 40 CFR 30.7 of the final rule. The EPA would like to clarify that this rule does not direct or require any outside entity or the EPA to establish data sharing mechanisms and does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science. Comments related to making data available and future funding opportunities are not within the scope of this rulemaking. Finally, the EPA is maintaining that the rule provisions apply equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. However, in consideration of instances where data cannot be made available, the EPA amended the rule to provide technical considerations (40 CFR 30.5(d)) that can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability in the absence of maximum transparency and included an exemption criterion -- that the development of the dose-response data was completed or updated before the effective date of the rule -- to 40 CFR 30.7 to account for the concerns raised by the SAB and other public commenters. 
Comment: Commenter (6916) asserts that the proposal is not consistent with OMB and EPA Guidelines under the Data Quality Act. The commenter states that the Data Quality Act, in force since the early 2000s, directs transparency in analysis and a process for questioning the results of science used in Agency decision making.
Commenter (6916) asserts that longstanding statutory authorities govern the quality, objectivity, utility, and integrity of scientific data relied on by federal agencies, as implemented by policies and formal guidelines in place for decades. The commenter states that these authorities have been effective at ensuring the quality of studies relied on by EPA, offering a process for questioning study results and for the controlled reanalysis or use in new studies of sometimes confidential or proprietary human health data and models. The commenter notes that adding additional regulatory requirements and constraints is both impractical and unnecessary.
Response: The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. This rule is designed to build upon OMB M-19-15, which highlights the need to characterize the sensitivity of an agency's conclusions to analytic assumptions, as well as other federal guidance documents that require greater data transparency.
These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. As described in preamble discussion of costs and benefits, the EPA recognizes there will be some incremental review and administrative costs to the Agency but anticipates these will be small. Furthermore, as the final rule makes clear, if implementing this procedural rule would result in conflicts with existing environmental statutes, and their implementing regulations, this rule will yield to the EPA statutes and regulations. 
Comment: Commenters (6119, 6916) assert that the FOIA also offers a pathway for the release of data that have been the basis of Agency decisions. Commenter (6916) states that the FOIA exempts from "public" disclosure are "medical files and similar files the disclosure of which would constitute a clearly unwarranted invasion of personal privacy."
Response: The rulemaking does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (0844) notes that the EPA proposal is consistent with policies on scientific integrity espoused by previous administrations. The commenter states that, for example, President Obama encouraged, "transparency in the preparation, identification, and use of scientific and technological information in policymaking" in a March 2009 Presidential Memorandum.
Response: The EPA agrees that the Agency and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. The requirements in the final rule build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information.
13.2.3.3 Other Existing Means to Determine the Validity of Studies
 13.2.3.3.1 Administrative Procedure Act (APA)
Comment: Commenters (4534, 6108, 6153, 6171, 6358, 6446, 6901, 6922) suggest that the current process is already transparent because EPA rulemaking provides opportunity for public review and comment.
Commenter (4534) asserts that the comment period provides opportunities for industry to validate or challenge the assumptions that have been made based on the data used to develop the regulation. The commenter states that regulations' quality should also improve as the authors will know that industry will be validating their conclusions from the data provided. 
Commenter (6108) states that, if EPA has concerns about whether to use a study because it appears that the conclusions are based on faulty science, it is the EPA's duty to investigate the study further and make its concerns known in the rulemaking process. The commenter adds that, if the public has concerns on a study used by the EPA, those concerns and any counter studies may be raised in the rulemaking process.
Commenter (6446) asserts that the ultimate safeguard to assure that EPA regulatory action is based on high quality science is the fact that EPA's regulatory action is subject to the notice and comment requirements of the APA (APA, 6 1.J.S.C. § 553) and ultimately to judicial review. The commenter states that, to the extent that scientific information proposed as a basis for regulatory action falls short, the APA provides members of the public, regulated businesses and other scientists an opportunity to comment to the agency about those shortcomings and to submit contrary studies and information for agency consideration. The commenter notes that, following adoption of a regulation, interested parties may seek judicial review to determine whether it is arbitrary, capricious or lacking in evidentiary support. (5 1.1.S.C. § 706.) The commenter asserts that the APA requirements, and the opportunity for judicial review of EPAs regulatory action, protect against the adoption of regulations that lack high-quality scientific support. 
Commenter (6901) notes that states can openly question the validity of studies during the comment period for the proposed rule. The commenter suggests that the rulemaking docket provides open access and transparency now for states to examine the basis for new rules. The commenter contends that states can review what scientific studies were used in the preparation of the proposed rule and enough detail is provided to judge whether these studies support EPA's conclusions. The commenter adds that, during this time, states can also recommend additional studies that they believe EPA should consider in developing the final regulations. The commenter notes that, in the future, EPA can enhance the opportunity for input by consistently allowing a minimum 90-day comment period for new/revised rules. According to the commenter, early involvement by states, as co-regulators, in the early stages of the regulatory development process (pre-proposal) will allow states even more opportunity to provide input on the science used to support the new rules. The commenter adds that, beyond the science, involving states as early as possible in the regulatory development process means the resulting regulations can be effectively implemented and public health protection enhanced.
Commenter (6922) states that the EPA follows a robust public notice and comment process for its significant rulemakings. Where the public has valid reason to doubt the assumptions or data underlying a particular study, or is aware of a study that EPA has not considered, the commenter notes that the public can share conflicting studies and other information with EPA during the public participation process of the rulemaking. The commenter asserts that the EPA is obliged to exercise independent reasoned judgment to disregard information the Agency deems unreliable or biased. The commenter states that this is the traditional, well-settled process of reasoned administrative rulemaking.
Response: The EPA agrees that existing mechanisms within the EPA, the federal government, and in the scientific community are increasingly pushing towards greater data sharing and increasing scientific rigor. The EPA supports these efforts and is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. The EPA believes that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. 
As described in the final rule, the EPA will use existing factors (including soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review) to evaluate study quality. When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence (consistent with Agency guidelines on hazard identification and dose-response assessment) to support a relationship between exposure and effect, the EPA will identify a subset of those studies for use in characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect. This will be based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both will be identified as pivotal science. Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. In compliance with the rule, the EPA shall give greater consideration to evidence, findings, and conclusions of pivotal science where the underlying dose-response data are available in a manner sufficient for independent validation. When the dose-response data underlying pivotal science are not publicly available in a manner sufficient for independent validation, the EPA will give less consideration to the studies' evidence, findings, and conclusions unless the Administrator grants an exemption request.
Although current methods and processes do allow for public comment and, where appropriate, peer review in the development of significant regulatory actions and influential scientific information, subject matter experts, peer reviewers (including the SAB and other science advisory panels), and the courts generally have not had access to the underlying data necessary to fully evaluate scientific conclusions. EPA does not take the position that studies for which the underlying data are not publicly available are per se invalid. Rather, EPA acknowledges that where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. The EPA is confident that this approach will result in significant regulatory actions and influential scientific information that are based on the highest quality, most relevant studies and ensure adequate human health and environmental protections, while maximizing transparency and maintaining consistency with the statutes the EPA administers.
13.2.3.3.2 Bradford Hill Criteria (BHC)
Comment: Commenter (4541) asserts that, given its constrained mission, flawed paradigm, political pressures on it to chase the impossible goal of zero risk, and evidence of actual corruption, there can be no confidence in any science produced by EPA in justification of its regulations. The commenter states that much of the controversy over the impact of air pollution on humans is due to the assumption by policymakers, regulators, and advocates that evidence of an association between a chemical in air or water and a human health effect is evidence that the chemical causes that effect. The commenter states that, because distinguishing between coincidence and correlation, on the one hand, and causal relationships on the other can be very difficult in matters of public health, an English epidemiologist named Sir Austin Bradford Hill (1897 - 1991) established in 1965 what has become known as the BHC, nine minimal conditions necessary to provide evidence of a causal relationship between a physical cause (in this case an air pollutant) and a health effect (illness or mortality). The commenter suggests that a better way to determine the validity of studies is to consider the BHC and other requirements of the scientific method. 
Response: The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. The EPA's attention to data transparency is consistent with federal initiatives on open data and responsive to the broader interest in greater data and model transparency observed in the numerous transparency initiatives in the scientific community and federal government, as well as the criticism the EPA has received from members of the public, scientific community, and Congress on the transparency of the scientific basis for EPA's decisions in previous influential scientific information assessments and regulatory actions. The EPA's continued progress toward maximizing transparency is vital to building and maintaining trust with the public and credibility in the Agency's decisions.
13.2.3.3.3 FOIA
Comment: Commenter (6955) asserts that the proposal does not mention current FOIA procedures for providing access to researcher data from federal research and from privately funded research submitted to EPA for consideration. The commenter states that, hence, there is no discussion of the differences, between the two approaches and no claim that the new proposal offers any added value to the current situation. The commenter contends that, by omitting mention of the OMB and IQA FOIA options, the proposed rule gives the impression it is a public-access rule, rather than a study exclusionary rule. 
Response: FOIA processes are not the subject of this rulemaking however, in response to comments received, the EPA is modifying the definition in the final rule to add the following at the end of the "publicly available" definition: "The public must be able to access the information on the date of publication of the proposed rule (or, as appropriate, a supplemental notice of proposed rulemaking, or notice of availability) for the significant regulatory action or on the date of dissemination of the draft influential scientific information for public review and comment."
13.2.3.3.4 Good Laboratory Practices (GLP)
Comment: Commenter (0044) states that, under its GLP inspection program, EPA already has a program "available to the public for validation in a manner that honors legal and ethical obligations to reduce the risks of unauthorized and re-identification." The commenter provides that the following is taken from EPA's Good Laboratory Practices Standards Compliance Program: 
      EPA conducts inspections as part of its GLP program to monitor compliance with the regulations to assure that studies submitted to the Agency in support of a pesticide registration or under a testing consent agreement for an industrial chemical were done with integrity, are of good quality and valid.
Commenter (6161) recommends the Agency adopt a requirement that studies that warrant consideration as key or pivotal studies conform with GLP or "the spirit of GLP" in order to promote the goal of relying on reproducible data. The commenter states that, whether in the "pure" sciences or regulatory sciences, the principal reason for requiring transparency of scientific data is to enable reproducibility studies and validation. The commenter states that, for the Agency to require transparency but ignore findings of irreproducibility defeats the purpose. The commenter notes that the Agency requires that scientific testing data submitted by industry as the basis for product registrations, hazard evaluations, or other purposes comply with GLP. The commenter states that the reason for the requirement is to ensure that the Agency's evaluations are based on data of sufficient quality, rigor and reproducibility. The commenter states that, recognizing that not all provisions specified by formal GLP programs can be applied to all studies, the concept of the spirit of GLP is often adopted by regulatory agencies and study sponsors. The commenter provides the following additional information regarding GLP:
      The Organization of Economic Cooperation and Development (OECD) defines GLP as ...a quality system concerned with the organizational process and the conditions under which non-clinical health and environmental safety studies are planned, performed, monitored, recorded, archived and reported. The definition of GLP makes clear that the principles of GLP are applicable to all scientific studies, not just to "guideline studies." In the environmental sciences, the regulated community often adopts a requirement that a laboratory, contractor, or grantee follow "the spirit of GLP" as a condition of funding a study that may serve a regulatory purpose. In practice, conforming with the spirit of GLP involves preparation of a protocol that describes the details of a study. The description includes safety precautions, lists of required equipment, sampling procedures, chains of custody, analytical methods, sample archiving, QA and QC measures, calculating and reporting results, and predefining and documenting criteria for including or excluding data to help avoid bias. As a study progresses, changes may be required. A GLP-compliant protocol describes procedures for amending the protocol to record the changes. In this way, protocols promote successful study replication and assist in the types of detailed peer reviews that can (and should) be a feature of regulatory science.
      In the environmental sciences, adherence to GLP principles is not uniform. Scientific studies sponsored by the regulated community to meet regulatory requirements are likely to conform with GLP. However, it has been PCTC's experience that studies in the environmental sciences conducted in some academic institutions and non-regulatory government agencies show little or no evidence of conforming with GLP. We have found protocols prepared by government agencies that would be rejected out-of-hand by EPA if submitted by industry in support of product registration or evaluation. Scientist-to-scientist requests and FOIA responses sometimes demonstrate the complete absence of protocols or even something as simple as an outline of a study design. Quality control measures for analytical chemistry are generally in place, but quality management for other aspects of environmental studies seems to be the exception rather than the rule.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information, and does not prescribe conformity with GLP or other laboratory practices. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. Also, the rulemaking does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenter (2907) states that the proposed regulation directs the EPA to use industry-sponsored studies by requiring it to use studies that follow "standardized test methods" and "good laboratory practices." According to the commenter, these are industry standard practices, not academic research practices which use state-of-the-art methods. The commenter notes that standardized test methods are industry-sponsored studies designed to gain regulatory approval of the test substance. The commenter also notes that industry test studies have been required to follow good laboratory practices since the 1970s after flagrant test violations and fraud were identified. The commenter contends that, in contrast, academic research is the cutting edge of methods development, and must meet IRB approval, which sets high scientific and ethical standards of conduct. 
Commenter (6109) notes that the proposal states that where "available and appropriate, EPA will use peer-reviewed information, standardized test methods, consistent data evaluation procedures, and good laboratory practices to ensure transparent, understandable and reproducible scientific assessments." The commenter states that, with this statement, EPA is displaying a bias toward industry-backed research over research from independent and university scientists. The commenter states that GLP, a protocol that federal regulators apply to industrial laboratories, is infrequently used outside of industry because GLP is expensive and does not reflect science quality. The commenter states that GLP regulations mandate methods for archiving data, record keeping, equipment calibration, etc., but they do not influence research design or methodology. The commenter states that academic researchers (who are frequently federally funded), on the other hand, follow scientific processes that are subject to the rigors of peer review and independent replication. The commenter states that, if EPA is to truly use the "best available science," it must focus on studies that use appropriate protocols and techniques, not merely favored record keeping processes. 
Commenter (6122) asserts that adoption of "Good Laboratory Practices" would prevent the EPA from using the best available science. The commenter states that independent research that does not follow GLPs could be deemed "qualitative" by the EPA and therefore, by definition under EPA guidance, not valid for use in establishing thresholds. According to the commenter, this "qualitative" research can be fundamental in establishing cause and effect, result in seminal publications, and often funded by the United States government. The commenter asserts that EPA must use the most sensitive study the passes the rigor of independent peer-review in establishing protective thresholds. The commenter contends that GLPs are not an acceptable way of judging scientific quality and should only be used as an inclusion criterion for research where there appears to be some incentive for fraud.
Commenter (6122) asserts that, while GLPs became guidelines to prevent fraud in industry science by requiring high numbers of replicates and model test systems, public health studies from universities or other non-industry sources do not need  --  and would not benefit  --  from GLPs. The commenter notes that providing raw data or using extensive replicates is rarely a requirement for scientific publication, yet published research is how most scientists communicate their findings and it is the way science is able to progress in every single field. The commenter adds that the precarious funding of most academic and governmental research precludes the inclusion of many GLPs, since following them makes it cost prohibitive to conduct studies but in no way makes conclusions less scientifically valid. The commenter contends that there is no reasonable or rational basis for ignoring a peer-reviewed, published study that does not follow GLPs. The commenter provides that a 2012 Scientific Advisory Panel on atrazine stated that: "
      In the view of the Panel, the test design elements [GLPs] should not be applied so strictly to the published literature as to disqualify all studies that do not meet all of these criteria...In the Panel's analysis, the EPA's strict application of the test design elements to the published literature was flawed and many of the test design elements should be relaxed for review of the published literature.
      "A study could be considered a high-quality study and very useful in risk assessment (even quantitative assessment), even if some of these design elements are not met."
Commenter (6940) does not support the proposed requirements to use GLP. The commenter asserts that GLP does not ensure "good" science or use of optimal study design and analyses, while government-supported, peer-reviewed science had evolved to improve reproducibility. The commenter states that optimal regulatory science depends upon both scientifically valid data and credible data. GLP is only designed to generate and analyze data following specific protocols. The commenter notes that peer-reviewed science, on the other hand, can address both data needs required for regulatory science. The commenter adds that this is true now, more so than ever, given that the National Institutes of Health (NIH) has new requirements for outlining components contributing to rigor and reproducibility. The commenter provides the following are two examples of the lack of scientific (vs. procedural) rigor inherent in GLP:
      First, toxicological assessments of bisphenol A (BPA) by scientists following GLP and those conducting experimental toxicology studies came to disparate conclusions over the low dose toxicity of BPA. However, the GLP studies failed to use a sensitive strain of mouse and failed to show effects of a positive control, compromising the scientific validity of the studies. Further, adverse outcomes measured in GLP studies of BPA tend to be of low sensitivity. On the other hand, hundreds of independent studies by multiple experimental toxicologists have confirmed the low dose toxicity of BPA. 
      Second, GLP is used as an approach in pharmaceutical safety testing, where its shortcomings also have been revealed. Guth and Pugsley present an example of a GLP-compliant study of cardiovascular effects of a therapeutic drug using instrumented dogs (telemetry-based data acquisition). Significant flaws in the experimental design (e.g. use of non-naïve dogs, failure to acclimate the dogs to the instrumentation or the study environment, insufficient power in the study design, insufficient demonstration of the signal quality from the telemetry system) which led to high variability in the data and difficulties in interpreting the data. 
Response: The EPA would also like to clarify that the purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information, and does not prescribe conformity with GLP or other standard practices. The EPA is also not categorically excluding any studies from consideration when promulgating final significant regulatory actions and finalizing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will subsequently consider the availability of underlying dose-response data when evaluating pivotal science and give greater consideration to studies where the underlying dose-response data are available for independent validation. Including this review of dose-response data availability for pivotal science is critical to ensuring confidence in the EPA's decision-making because it supports the scientific reanalysis and scrutiny by independent third parties for pivotal science. This approach will result in final significant regulatory actions and influential scientific information based on the highest quality studies that maximize transparency, leading to human health and environmental protections consistent with the statutes the EPA administers.
13.2.3.3.5 Implement Existing Guidelines, Plans, Policies, and Programs
 General 
Comment: Commenter (4595) states that there exist federal guidelines that provide access to and ensure the quality of scientific information used for federal policies and regulation which also emphasize the necessity to maintain needed privacy or assurance of suitable confidentiality. The commenter states that there are also procedures for providing protected, limited access to the full raw data to independent researchers for independent validation without making the full data publicly available. The commenter states that these procedures negate the need for this proposed rule, as they respect privacy laws and bona fide practices used by the scientific community.
Response: The EPA is finalizing internal procedural requirements for how the Agency will consider the availability of underlying dose-response data of pivotal science when promulgating a significant regulatory action or developing influential scientific information that relies on dose-response data. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Implementation of this rule, which will occur in forthcoming statute-specific transparency regulations or programmatic regulations, will increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information. The requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
Comment: Commenter (4839) states the proposed transparency rule is not based on scientific best practices, and it is inconsistent with transparency processes administrated by major business organizations, academia and the federal government. The commenter states that, for example, EPA proposes that "data and models" become publicly available and be subject to additional, time-consuming and expensive validation and analysis; by contrast, the American Petroleum Institute (API) has maintained for decades a transparency policy that enables a university investigator receiving a grant to retain ownership of the scientific data generated in a study it has funded. The commenter states that, similarly, the American Chemistry Council (ACC) has a long-standing policy through its Long-Range Research Initiative that provides external scientific investigators to retain ownership of their data even while encouraging these researchers to publish their results in the open scientific literature. The commenter states that EPA, other federal agencies and universities have long abided by policies with similar principles and practices.
Response: The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. This rule is designed to build upon OMB M-19-15 (Ref. 18), which highlights the need to characterize the sensitivity of an agency's conclusions to analytic assumptions, as well as other federal guidance documents that require greater data transparency (Ref. 18). The EPA's attention to data transparency is also responsive to the broader interest in greater data and model transparency observed in the numerous transparency initiatives in the scientific community and federal government, as well as the criticism the EPA has received from members of the public, scientific community, and Congress on the transparency of the scientific basis for EPA's decisions in previous influential scientific information assessments and regulatory actions (Refs. 19, 20, 21, 22, 23). The EPA's continued progress toward maximizing transparency is vital to building and maintaining trust with the public and credibility in the Agency's decisions.
Comment: Commenter (6899) states that EPA's goal to strengthen regulatory science transparency is attainable without adding this regulation to the federal collection. The commenter states that EPA must require enforcement of the existing federal rules that strengthen this kind of transparency. The commenter states that, for example, the IRB process and the Health Insurance Portability and Accountability Act of 1996 (HIPAA) Privacy Rule requirements exist to protect privacy, confidentiality, safety and security when conducting scientific studies and reporting scientific results. The commenter states that EPA and other federal agencies already spend a significant amount of taxpayer dollars to enforce these federally required processes. 
Response: This is a rule of internal procedure promulgated under the EPA's housekeeping authority. This rule does not direct or require any outside entity or the EPA to establish data sharing mechanisms and does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science. This rule does not impact rules developed and overseen by other federal organizations, including rules governing the IRB process and privacy under HIPAA.
Comment: Commenter (6122) contends that, even where EPA may have some legitimate concern about underlying data, there are existing approaches that address those concerns without this rule, such as allowing EPA itself to examine underlying data in a secure manner. In short, the commenter suggests that the EPA can ameliorate any concerns about underlying data either by continuing to address these issues on a case-by-case basis, or by developing guidance for reviewing underlying data in a manner that both preserves confidentiality and any other legitimate interests.
Response: As discussed in the preamble of the final rule, the EPA, the federal government, and the scientific community are increasingly pushing towards greater data sharing and increasing scientific rigor. The EPA supports these efforts and is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. The EPA believes that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety.
Food and Drug Administration (FDA)
Comment: Commenter (5041) suggests that, instead of requiring all scientific research used to make EPA policy, regulation or guidance be based on publicly available datasets, EPA can adopt the long established and proven processes and procedures used by FDA. The commenter states that this ensures that privacy laws and concerns are met while having the most robust numbers of studies to ensure sound scientific policy/regulation/guidance making where the public may bear the cost or harm of said policy/regulation/guidance. The commenter states that the FDA decision making process to either approve or clear new/changed medical devices has a proven track record in ensuring that good decisions are made using the current regulatory policies, regulations, and/or guidance, i.e., publicly available datasets are not required. The commenter notes that, like the FDA, EPA uses studies that may include participant privacy issues, including ensuring protected health information (PHI), is not disclosed as required by law. The commenter provides that every study using human participants includes participant consent ensuring their privacy is ensured while stating that domestic and/or international regulatory agencies may have controlled/private access to the data when ensuring that the conclusion(s) made by the study sponsor are correct and meet the study objective(s) and endpoint(s). According to the commenter, peer-reviewed and published scientific research has also never required studies to be supported by publicly available datasets. The commenter notes that, while there are examples of conclusions being revised based on further independent review of the data or of researchers dry-labbing (falsifying) the data, we have a substantive history of the peer-review process ensuring reliable evidence is presented to the public.
Response: This is a rule of internal procedure promulgated under the EPA's housekeeping authority. This rule does not impact rules developed and overseen by other federal organizations (e.g., FDA), including rules governing the IRB process and privacy under HIPAA. The EPA is requiring that, when promulgating significant regulatory actions or developing influential scientific information, the Agency will determine which studies constitute pivotal science and give greater consideration to those studies determined to be pivotal science where the underlying dose-response data are available in a manner sufficient for independent validation. Focusing the final rule requirements on the underlying dose-response data is intended to address public comments concerning clarity of the rule, potential unintended consequences, and the potential for far-reaching impacts. The requirements provide a workable framework for EPA to evaluate pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
IRIS
Comment: Commenter (6133) asserts that the EPA chemical assessment program, called the IRIS already uses credible transparent methods to provide the public with reliable, transparent, credible chemical hazard assessments and toxicity values. The commenter states the program received high praise from its last two reviews by the NAS (NAS 2014 and NAS 2018), as well as from the SAB (SAB 2017) for its continuous improvements and successes in its methods for evaluating and integrating scientific evidence from various streams including human studies, animal studies, and mechanistic studies. The commenter contends that the proposed rule would undermine decades of expert work to advance successful data evaluation methods described in the systematic review approach now underway in the EPA IRIS program. 
Response: The EPA disagrees that implementation of the final rule requirements would undermine the Agency's work. When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence (consistent with Agency guidelines on hazard identification and dose-response assessment) to support a relationship between exposure and effect, the EPA will identify a subset of those studies for use in characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect. This will be based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both will be identified as pivotal science.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA may still use pivotal science when the underlying data are unavailable after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
2013 White House Office of Science and Technology Policy (OSTP) Memorandum/EPA's Plan to Increase Access to Results of EPA-Funded Scientific Research
Comment: Commenters (6191, 6924) support implementation of the 2013 White House OSTP memorandum and EPA's Plan to Increase Access to Results of EPA-Funded Scientific Research. Commenter (6924) states that, to increase public access to science funded and used by the government, EPA should ensure it is in full compliance with the provisions of the OSTP memo. Commenter (9224) asks what is the status of the OSTP plan is and what does the proposed rule cover that this document does not regarding data funded and produced by EPA? Commenter (6191) states that the OSTP plan reflects an understanding of the need for maintaining balance between the desire to public access to data and the responsibility to protect data providers and their subjects; however, the proposed rule extends the intent for public access beyond reasonable expectations. 
Commenter (6191) provides that two sections of the OSTP plan that address current sharing of information are:
      1. "EPA has a long history of collaboration in scientific research and is a leader in providing access to environmental information to encourage better decisions and a more informed public. Transparency is a core EPA value. The Agency already makes publicly available much of the Agency's scientific and technical work, including information that supports regulatory decisions. For example, EPA provides all materials and scientific information supporting each regulation in public dockets, which are publicly available for comment at www.regulations.gov. In addition, the Agency maintains an enterprise dataset metadata catalog (the Environmental Dataset Gateway (EDG) at https://edg.epa.gov/metadata/catalog/main/home.page​) through which thousands of EPA datasets are publicly available. EPA's Enterprise Information Management Policy (EIMP), adopted March 3, 2015, codifies the Agency's approach to facilitating access to data held in EPA information systems. Appendix D describes, in more detail, a small sample of EPA's extensive ongoing efforts to be transparent and to increase access to environmental information." (page 4)
      2. "While the Agency strives to increase access to its research results, it recognizes, consistent with the OSTP Memo, that Federal agencies have a responsibility to protect confidentiality and personal privacy, respect proprietary interests and property rights, and balance between the value of providing long-term access and its associated costs. It is important to recognize that some research data cannot be made fully available to the public but instead may need to be made available in more limited ways, e.g., establishing data use agreements with researchers that respect necessary protections. Whether research data are fully available to the public or available to researchers through other means does not affect the validity of the scientific conclusions from peer-reviewed research publications." 
Response: EPA has implemented the public access requirements for EPA researchers, grantees and contractors as required by the February 22, 2013, OSTP memorandum "Increasing Access to the Results of Federally Funded Scientific Research" and the 2016 EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research." EPA researchers, grantees and contractors are required to make peer reviewed journal articles and the underlying data resulting from EPA-funded research accessible to the public. This final rule applies to pivotal science that was not funded by the EPA and requires the EPA to evaluate the availability of an earlier stage of data (the data that are ready to be analyzed to extract relevant information) than the stage of data required to be available under the 2016 EPA plan. As discussed more fully in the preamble of the final rule, the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
CONSORT, ARRIVE and STROBE
Comment: Commenter (6924) suggests EPA focus on implementing existing initiatives and guidelines for improving data sharing and transparency at federal agencies. The commenter states that, through collaborative efforts, academia, medical scientists, industry and publishers have developed several protocols and guidelines specifically designed to improve reporting and evaluation of studies that will improve the quality and transparency of the interpretation of findings. The commenter adds that protocols and guidelines, such as CONSORT, Animal Research: Reporting of In Vivo Experiments (ARRIVE) and Strengthening the Reporting of Observational studies in Epidemiology (STROBE), do not require public access to all study data and will still improve the scientific basis of evaluating studies. The commenter suggests that EPA utilize these existing tools to evaluate study quality and build on this work already done by the scientific community.
Response: The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. This rule is designed to build upon OMB M-19-15, which highlights the need to characterize the sensitivity of an agency's conclusions to analytic assumptions, as well as other federal guidance documents that require greater data transparency. As discussed more fully in the preamble of the final rule, the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
NIH Guidelines
Comment: Commenter (6924) suggests that data sharing guidelines from the NIH are applicable to the sharing of final research data for research purposes. The commenter states that, as the NIH guidelines have already undergone an extensive deliberative process with input from the scientific community, EPA should reference these guidelines in its grants and/ or cooperative agreements.
Commenter (6126) suggests that EPA consider the NIH guidance for grantees about protocols and practices for data sharing. The commenter states that the policy, which has now been in effect for 15 years, reflects another process that EPA could consider as a model for developing a thoughtful and informed regulation about agency and researcher data sharing.
Response: EPA has implemented the public access requirements for EPA researchers, grantees and contractors as required by the February 22, 2013, OSTP memorandum "Increasing Access to the Results of Federally Funded Scientific Research" and the 2016 EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research." EPA researchers, grantees and contractors are required to make peer reviewed journal articles and the underlying data and models resulting from EPA-funded research accessible to the public. The final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science, nor does it require any researcher or other outside entity to provide data or models to the EPA or the public. Therefore, the Agency will not be developing information or protocols on data sharing.
National Center for Health Statistics Research Data Centers
Comment: Commenter (6924) suggests that providing secure access to data for research purposes also improves data sharing and is an alternative to requiring publicly available data. The commenter recommends that EPA consider the National Center for Health Statistics Research Data Centers as a model -- these centers have successfully provided access to sensitive information without breaches of privacy/ confidentiality. 
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
13.2.3.3.6 Methodology and Assumptions
Comment: Commenters (1796, 6111, 6121, 6935) suggest that review of a study's scientific methodology and assumptions can be sufficient and is important when validating scientific results. 
Commenter (6935) asserts that robust discussions of methodology and assumptions can provide sufficient information for stakeholders to critique and evaluate the validity of studies and models, which may be sufficient for the purposes of determining if the study can and should be relied upon by the Agency in a regulatory context. The commenter states that, accordingly, final rules should reflect that some flexibility may be appropriate if sufficient other information about studies and models is provided or otherwise available. The commenter adds that it may not be necessary for EPA to complete a full, independent validation of a study or model if sufficient information about inputs, assumptions and methodology are made available. The commenter states that such information may provide adequate information for determining whether the study can and should be relied upon in the regulatory context, without the expense, both in terms of resources and time, of independent validation. The commenter states that recognizing this in any final rule could provide the Agency and regulated entities additional flexibility.
Response: The EPA agrees that existing methods of evaluating the methods and assumptions presented in a peer-reviewed study are useful. However, when subject matter experts have access to the underlying data, they could conduct a more in-depth review that may include independently reanalyzing the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty in the original study.
Further, although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science is unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Though not exhaustive or exclusive, the EPA has identified several factors in 40 CFR 30.5(d) that balance some of the important technical considerations the EPA will consider in addition to data availability and that are particularly relevant to the stage of the analysis where dose-response data are used. Additional discussion on these factors, which include consideration of assumptions and methodology, is provided in the preamble to the final rule.
13.2.3.3.7 Replication, Reproducibility and Reanalysis Processes of Science
 Replication
Comment: Commenters (5172, 6046, 6111, 6373) suggest EPA consider replication of scientific studies by different researchers using different data sets to validate studies that may be useful for regulatory purposes. Commenters (6111, 6373) suggest EPA corroborate results of a scientific study by conducting subsequent studies/validation studies.  
Commenter (1973) states that the publication of similar studies, by other investigating teams, using equivalent data from other countries, also provides the EPA with the assurance it needs that the results they rely on have been replicated, and by different people using different approaches. The commenter notes that, for example, Medicare data was first used to look at air pollution by one investigator, first working at EPA and then at Harvard. The commenter states that then other investigators sought the data and published their own studies. The commenter adds that, while these data may not be made publicly available as EPA proposes, the data are available to investigators with the will to apply for access and adhere to the data use rules. The commenter provides that, similarly, many studies, by many different teams of investigators have used mortality data from state mortality or national records to examine the acute effects of air pollution. The commenter asserts that there is no reason to risk revealing identified confidential medical information to accomplish a task that is already accomplished. 
Commenter (2431) asserts that replication of results using different data, not revelation of personally identifiable health data, is the strength by which we should measure our science. The commenter states that, beyond the HEI reanalysis, the findings of Six Cities have been replicated numerous times in many independent studies using different study designs, larger numbers of enrollees, broader geographical distributions across multiple regions of the world, and increasingly sharp analytical tools. The commenter notes that the number of studies replicating Six Cities' original findings demonstrate the soundness of the scientific process and have established an extensive foundation of literature whose conclusions are just as important to the weight of evidence as Six Cities itself. 
Commenters (6153, 6874) contend that true replication requires not reanalyzing existing work but independent investigators producing independent data. The commenters state that the HEI work reanalyzing an existing body of evidence  was time-consuming and expensive and that it would not be a good use of resources to require this approach for all studies contributing to the regulatory actions, or even to a few key studies.
Commenter (6876) states that the proposed rule fails to recognize the power of replication, a key criterion for defining the strength of scientific evidence. The commenter adds that replication refers to the fact that consistent findings from studies in different populations in different places strengthens the likelihood of an effect. 
Commenter (6916) states that the proposal's focus on public release of the underlying data used in studies  -  particularly "dose response data" and studies, evinces a clear lack of understanding of the way science is conducted and verified. The commenter notes that, while replicating the results of a study with the same data set certainly verifies it, as the National Research Council said in 2002, "[f]or large epidemiological studies, repeating a study is seldom either possible or desirable." The commenter asserts that more effective, instead, is some combination of new studies testing the same question in different places, and/or an independent analysis with the same data.
Commenter (8801) states that systematic replication by different researchers using different techniques collecting different data sets, yet addressing a similar fundamental health question, provides the strongest possible replication method.
Commenter 6139) states that EPA's transparency proposal is based on the inappropriate grafting of transparency rules appropriate for clinical research onto the much different world of human health environmental research on which regulation is often based. The commenter notes that environmental health studies in communities cannot and should not be held to the same gold standard for clinical research as a double-blind drug and placebo study. The commenter states that, instead we depend upon replication through consistency in the findings when investigated in other populations by different authors using different methodologies, and on biological plausibility in the form of toxicological studies. 
Response: Although the EPA agrees with commenters that meaningful insights can be obtained through similar studies in different media or context, the EPA continues to find that independent validation of the study findings and conclusions driving the EPA's dose-response assessments would provide important information. As detailed in the preamble of the final rule, there is scientific support for the usefulness of reanalyzing data, and the EPA finds this to be especially true for data that drive the quantitative requirements or analyses of EPA significant regulatory actions or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and, thus, increase the opportunity for independent subject matter experts to validate pivotal science, and as the dose-response data are better understood through independent reanalysis the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
The EPA also wishes to clarify reanalysis is not required under this rule. The rule requires the EPA to give greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Reproducibility
Comment: Commenter (0040) states that the whole scientific and statistical process of drawing conclusions is based on the idea of reproducibility. The commenter notes that this duplication of research occurs a lot, especially if findings are unexpected, controversial, or groundbreaking. According to the commenter, if results are not duplicated, then the theories are rejected. According to the commenter, only after initial studies are replicated and extended upon, a body of evidence grows and the public policy shifts accordingly.
Commenter (6920) asserts that reproducibility and independent validation are critical aspects of the scientific method and have resulted in significant advancements in our understanding of the health effects of air pollutants at different exposures and thresholds.
Response: In the final rule the terms replicate and reproduce (or other forms of the word) are no longer used. In the final rule the term replicate is not used. The EPA is clarifying that the intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze." In addition, the EPA is modifying the definition of "independent validation" in the final rule by replacing "capable of being substantially reproduced" with "produced." The EPA will not finalize the proposed 40 CFR 30.2 definition of "capable of being substantially reproduced" because the term is not used in the final rule's definition of "independent validation" or elsewhere in 40 CFR 30.
Reanalysis
Comment: Commenters (5172, 6046, 6111, 9227) suggest EPA consider reanalysis of scientific studies to validate studies that may be useful for regulatory purposes.
Commenter (6135) requests that EPA recognize that reanalysis of individual studies is not a replacement for replication of study findings in multiple distinct health studies. The commenter suggests that except in the case of exceptional circumstances, EPA follow the example of past EPA administrators in waiting for additional studies to be completed before taking regulatory action based on a single study finding.
Commenters (5017, 6111, 6133, 6137, 6351, 6373, 6916, 9227) assert that scientific results and methods can be questioned and validated without exposing protected underlying data to public scrutiny. As an example, the commenters point to the series of studies, re-analyses, verifications, and extensions of work begun in 1974 on the prospective "respiratory health effects of respirable particles and sulfur oxides on a sample of adults and children in six U.S. cities."
Commenter (6351) suggests that, instead of blocking studies, EPA refer studies to an independent third party such as the HEI. The commenter states that autonomous reviewers examined the data from the 1993 Harvard Six Cities Study and the 1995 ACS Study and developed a report on the two studies that confirmed their original findings. The commenter notes that since these studies, other research has confirmed their findings as well, including some studies that used publicly available datasets. The commenter states that similar third-party reviews could readily address concerns about existing or future studies as needed.
Similarly, commenter (9227) states that an important effort began with Harvard's "Six Cities" study, reported in (Dockery et al., 1993). The commenter explains that the researchers initially sought to reproduce their initial findings using a data base with a much larger number of subjects and cities and did indeed reproduce those findings (Pope et al., 1995). According to the commenter, by 2009 enough new evidence had accumulated for EPA's integrated assessment for particulate matter to conclude that the number of large U.S. cohort studies, together with supporting evidence from other epidemiology and toxicological studies were sufficient to infer a causal relationship between long-term PM2.5 exposures and mortality and cardiovascular effects. The commenter states that this conclusion regarding causality (the strongest finding possible under the causality classification methodology based on these studies was endorsed by the external CASAC, which noted: "The five-level classification of strength of evidence for causal inference has been systematically applied; this approach has provided transparency and a clear statement of the level of confidence with regard to causation, and we recommend its continued use in future ISAs." The commenter provides that the link between particulate matter exposure and mortality that was observed in the Six Cities study has been vetted through multiple mechanisms that have confirmed the validity of the findings without public access to the underlying data -- including extensive reanalysis using larger datasets with longer duration of follow up and different statistical methods; reproduction and corroboration with independent studies using distinct populations and methodologies; and rigorous external review by independent scientists.
Commenters (6111, 6137, 9227) refer to comments of a SAB workgroup which assert that removing confidential information so that it can be publicly disclosed is unnecessary, as studies can be validated without demanding access to confidential data. Commenters (6111, 9227) state that, as explained by the SAB, the HEI's reanalysis of the Six Cities Study, through "an unusually rigorous form of peer review and independent reanalysis, coupled with many follow-up studies, has accomplished a measure of confidence in findings without public access to data and analytic methods." Commenters (6137, 9227) note the SAB workgroup also states:
      The proposed rule fails to mention that there are various ways to assess the validity of prior epidemiologic studies without public access to data and analytic methods. For example, the HEI conducted a re-analysis of the influential Harvard Six Cities and ACS epidemiologic studies and was able to replicate its findings and to assess the robustness of the findings via sensitivity analysis.
Response: In the final rule the terms replicate and reproduce (or other forms of the word) are no longer used. In the final rule the term replicate is not used. The EPA is clarifying that the intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze." In addition, the EPA is modifying the definition of "independent validation" in the final rule by replacing "capable of being substantially reproduced" with "produced." The EPA will not finalize the proposed 40 CFR 30.2 definition of "capable of being substantially reproduced" because the term is not used in the final rule's definition of "independent validation" or elsewhere in 40 CFR 30.
The EPA agrees with commenters on the utility of findings from reanalysis studies. Indeed, one criterion for the Administrator's consideration of an exemption under 40 CFR 30.7 is that a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis.
Finally, although the EPA agrees with commenters that meaningful insights can be obtained through similar studies in different media or context, the EPA continues to find that independent validation of the study findings and conclusions driving the EPA's dose-response assessments would provide important information. As detailed in the preamble of the final rule, there is scientific support for the usefulness of reanalyzing data, and the EPA finds this to be especially true for data that drive the quantitative requirements or analyses of EPA significant regulatory actions or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and, thus, increase the opportunity for independent subject matter experts to validate pivotal science, and as the dose-response data are better understood through independent reanalysis the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
13.2.3.3.8 Systematic Review
Comment: Commenters (4831, 6125, 6158, 6915) suggest that EPA consider adoption of systematic-review methods. Commenter (6121) suggests that multiple steps of independent scientific review be conducted to ensure the integrity and objectivity of science. 
Commenter (4831) notes that the National Academies have carried out numerous studies that advise EPA on the scientific bases of regulatory decisions related to human health and the environment. The commenter states that these reports encourage EPA to consider all available science in the rule-making process and provide guidance about how the agency could be more transparent in describing how evidence is gathered and evaluated. The commenter suggests that systematic-review methods be adopted to ensure objectivity, rigor, and transparency in performing literature-based reviews (IOM, 2011; NRC, 2011, 2014). The commenter provides that NASEM, 2017a, includes four case examples of systematic reviews.
Commenter (6125) asserts that the proposal ignores the existence of "systematic review" methods for review of evidence that are being developed within the agency, most notably within the IRIS program, prompted by criticisms and recommendations from the NAS. The commenter states that the development of such methods is informed by the National Toxicology Program's Office of Health Assessment and Translation (NTP/OHAT) guidance and other similar tools (UCSF's Navigation Guide). According to the commenter, no authoritative body of experts has ever recommended requiring "raw data" in order to perform or review dose response assessments. 
Commenter (6915) states that recently a rigorous systematic review process has been developed and implemented by EPA's IRIS program to ensure even greater thoroughness and objectivity in hazard identification. The commenter asserts that, in this way, EPA already ensures that the studies and data upon which it relies are valid.
Commenter (6158) asserts that systematic review methods are one well-regarded means to develop comprehensive criteria to evaluate and "rank" different scientific studies. The commenter states that data availability is only one criterion considered and the absence of public data is not a worthy justification for the exclusion of any given study. The commenter states that study design, analytical techniques used, and statistical methods applied are all deemed to be more important criteria by which to evaluate the strength and utility of a scientific study.
Response: When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices, which may include systematic review, and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence (consistent with Agency guidelines on hazard identification and dose-response assessment) to support a relationship between exposure and effect, the EPA will identify a subset of those studies for use in characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect. This will be based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both will be identified as pivotal science.
Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA may still use pivotal science when the underlying data are unavailable after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations
Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty in the assessment. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information.
13.2.3.3.9 Transparency and Openness Promotion (TOP)
Comment: Commenters (1306, 5120, 6119, 6360) suggest that EPA consider the TOP standards and assert that the scientific community already supports TOP Standards which are able to accommodate research using strict confidentiality and private guidelines. Commenter (1306) states that these standards recognize the array of workflows across scientific fields and make the case for data sharing at different levels of stringency; and in cases where all data cannot be fully shared. 
Commenter (6360) recommends that EPA draw from (or adopt) existing transparency screening tools developed by consensus in the scientific community to further define the scope and applicability of the rule's requirements. The commenter states that one important, open source system, the TOP guidelines, have been recently adopted by over 5,000 major scientific organizations and journals. The commenter notes that, published in Science in 2015, the TOP guidelines include eight standards, each with three levels of increasing stringency. The commenter provides that claims made in a journal requiring Level III disclosure are more likely to be recognized as scientific facts than those published to Level I standards. The commenter states that, since TOP and other systems are widely adopted now and are growing in their reach, scientists understand their obligations under these systems if they want to be published in certain journals. According to the commenter, if they must turn in their data to be published in Science, for example, turning it over to EPA for public review is not a significant additional burden.
Commenter (6360) recommends that EPA require TOP Level III transparency for all significant regulatory actions and all other EPA policy decisions have at least Level II standards for transparency to the public. The commenter states that, by "significant regulatory actions," they mean "regulatory actions" as defined in Executive Order 13771 and "significant" as defined in Executive Order 12866, OMB's Good Guidance bulletin, and other relevant guidance regarding regulatory planning and review. The commenter adds that, if researchers must comply with Level III procedures to receive one publication in Science, it seems reasonable to require the same level of disclosure for EPA decisions with millions (or billions) of dollars of annual economic impact. The commenter notes that, as EPA takes comment on a proposed decision and notes that the underlying judgements in the applied science meet Level II criteria, the public could evaluate the claims and provide valuable feedback to EPA regarding the appropriate rigor of its data transparency procedures. The commenter adds that, based on these comments, EPA could decide that the final agency action requires Level III transparency. According to the commenter, the Level II and Level III requirements mirror somewhat the applicability of the proposed standards in EPA's rulemaking. The commenter suggests that the tiered levels and specific requirements in TOP offer an existing, widely used approach for EPA to consider that would allow it to prioritize its resources and its requirements with respect to applied science developers. The commenter states that EPA can calibrate the transparency requirements of particular science applications with the magnitude of the policy decision in question.
Commenter (6360) states that the TOP criteria require researchers to post data, analytic methods, and research materials to a trusted depository. The commenter states that, for EPA applications of science, this depository should be the public docket of a proposed rule or the accompanying administrative record associated with a proposed decision. The commenter recommends that, for sensitive information, EPA adopt the depository standards in common use by other federal agencies, journals, and EPA itself. According to the commenter, for example, the Census Department has established stringent criteria for researchers seeking to review data that could allow individual responses to be identified. The commenter notes that similar procedures are in place to protect data from new drug trials in journal depositories. The commenter adds that EPA has procedures for its contractors to review CBI  -  procedures that could be modified for non-government public commenters to use to review applied science that relies on sensitive data.
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are publicly available or available through restricted access. Restricted access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. The EPA agrees that the scientific community and government agencies are making great strides in data transparency; however, improvements can still be made over existing policies and mechanisms. As discussed more fully in the preamble of the final rule, the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
13.2.3.3.10 Other Available Approaches
Comment: Commenters suggest EPA consider other available approaches that can ensure the validity of scientific studies that may be useful for regulatory purposes, including the following:
 Collaborative studies, where the original study authors employ data use agreements to share the data with other researchers, who can validate and expand upon the analysis. (6925)
 Comparisons of research findings with the results of other peer-reviewed research efforts, including through meta-analyses and literature reviews that are designed to shed light on consistent findings across studies. (9227)
 Competitive grant process and grant application review. (6121, 6373)
 Confidential sharing of data with independent research teams that are able to validate results. (9227)
 Data-use agreement application review. (6373)
 Detailed description of research methods, code and non-confidential data in their published articles. (6111) 
 Expertise and judgment of career EPA scientists when considering the strength and relevance of studies included in EPA decisions. (6121)
 Federal regulations that include review requirements for research involving human subjects. (6119)
 IRB requirements. (6121, 6373)
 Risk assessment drafts are made public and can be commented on. (6046) 
 Single research questions require multiple trials. (6373)
 Solicitation of expert opinion. (6153, 6358)
 Statutory requirements. (6373)
Response: Although the EPA agrees with commenters that meaningful insights can be obtained through similar studies in different media or context, the EPA continues to find that independent validation of the study findings and conclusions driving the EPA's dose-response assessments would provide important information. As detailed in the preamble of the final rule, there is scientific support for the usefulness of reanalyzing data, and the EPA finds this to be especially true for data that drive the quantitative requirements or analyses of EPA significant regulatory actions or influential scientific information. . Implementation of this rule will more effectively share pivotal science for external consideration and, thus, increase the opportunity for independent subject matter experts to validate pivotal science, and as the dose-response data are better understood through independent reanalysis the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information. 
Further, although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science is unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Though not exhaustive or exclusive, the EPA has identified several factors in 40 CFR 30.5(d) that balance some of the important technical considerations the EPA will consider in addition to data availability and that are particularly relevant to the stage of the analysis where dose-response data are used. Additional discussion on these factors is provided in the preamble to the final rule.
Finally, the EPA would like to clarify that the approach in the final rule gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access.
Comment: Commenter (6357) states that data from clinical trials are often not publicly available but are made available for confidential safety-monitoring and peer-review. The commenter states that policies for in-depth federal agency review also exist, as exemplified by the Food and Drug Administration's (FDA's) ability to audit clinical trials. 
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (6357) states that the validity of scientific evidence is based not only on individual studies, but also on a collection of multiple sources of data that may confirm or refute individual findings. The commenter encourages EPA to consider multiple legitimate paths for validation of scientific findings to ensure that the science most relevant to the protection of human health is included in the decision-making process.
Response: Although the EPA agrees with commenters that meaningful insights can be obtained through similar studies in different media or context, the EPA continues to find that independent validation of the study findings and conclusions driving the EPA's dose-response assessments would provide important information. As detailed in the preamble of the final rule, there is scientific support for the usefulness of reanalyzing data, and the EPA finds this to be especially true for data that drive the quantitative requirements or analyses of EPA significant regulatory actions or influential scientific information. Implementation of this rule will more effectively share pivotal science for external consideration and, thus, increase the opportunity for independent subject matter experts to validate pivotal science, and as the dose-response data are better understood through independent reanalysis the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information. 
Further, although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science is unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Though not exhaustive or exclusive, the EPA has identified several factors in 40 CFR 30.5(d) that balance some of the important technical considerations the EPA will consider in addition to data availability and that are particularly relevant to the stage of the analysis where dose-response data are used. Additional discussion on these factors is provided in the preamble to the final rule.
Comment: Commenter (6941) states that the TSCA identifies independent verification as one of five scientific standards that EPA must consider (also the intended use of the information, its relevance for use in making regulatory decisions, the clarity and completeness of its documentation, and the extent to which variability and uncertainty are evaluated and characterized). The commenter notes that even when underlying data is not publicly available in a manner sufficient for independent verification, EPA's use of such information, methods, or models can be consistent with the best available science if these other standards are met.
Response: This final rule does not interpret or apply the provisions of any environmental statutes; such efforts will occur in the subsequent actions under the relevant statutes described in the preamble of this rule. As this rule makes clear, if implementing this procedural rule would result in conflicts with existing environmental statutes, and their implementing regulations, this rule will yield to the EPA statutes and regulations.
13.1.3.4 Overall Comparison to Existing Transparency Mechanisms
 13.1.3.4.1 Are Existing Mechanisms Adequate?
Comment: Commenters (1905, 2472, 5170, 6135, 6152, 6194, 6446, 6894, 6915, 6916, 6920, 9227) express concern that the rule fails to analyze whether existing mechanisms for vetting science without publicly releasing subjects' personal data are adequate. Comments include:
 The proposed rule does not make a persuasive case for replacing the well-established and successful processes for determining scientific validity. The commenter asserts that, while there have been cases of intellectual fraud or non-replicable results, it is the scientific process and the scientific community itself that has discovered and corrected these problems. (5170)
 The proposed rule does not identify any deficiencies in the long-established approach of using scientific studies to inform agency decisions. (6135)
 The proposal does not justify the need for a change to its current transparency or reproducibility practices that include peer review and existing access to the science that EPA relies upon. (6152)
 EPA has not explained why existing federal and professional rules, standards, and policies, many of which promote the objective of transparency, cannot be used to attain the purported objective of the proposal. The commenter adds that the proposal also ignores, or presents no argument for rejecting, scientific practices embodied in the host of nongovernmental methods and best practices established and adhered to by the research community to ensure the transparency, objectivity, and validity of studies, analyses, models, and reports, including by reproduction and/or replication where appropriate. (6194)
 Commenter asserts that the lack of any explanation for a need for the proposed rule is doubtless because procedures are already in place that assure use of the best available science for EPA's regulatory activity. The commenter states that these procedures include that much of the scientific research considered in the regulatory context, and all research published in reputable journals, has already been subject to extensive peer review. (6446)
 Commenter contends that there are significant and viable and obvious alternatives to this proposal which the Agency has failed adequately to consider and explain why it believes such alternatives are not sufficient. (6916)
 While the proposed rule cites the need for increased transparency, it provides little evidence that this need is not being met. The commenter states that EPA models, for example, that serve as the basis for predicting cancer risk and blood lead levels in children are well presented, explained and supported. The commenter asserts that debate continues on the accuracy of such models but there is little debate that they are transparent. (6920)
 Commenter asserts that EPA fails to explain why EPA's existing mechanisms are inadequate to ensure the scientific integrity of its actions. The commenter states that the proposal ignores both the available approaches embraced by the scientific community and the record of past EPA assessments, which reveal alternative methods for ensuring the credibility of potentially useful scientific studies. The commenter notes that the proposal says virtually nothing about the level of confidence those mechanisms afford in EPA's regulatory science. (9227)
Response: The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. This rule is designed to build upon OMB M-19-15, which highlights the need to characterize the sensitivity of an agency's conclusions to analytic assumptions, as well as other federal guidance documents that require greater data transparency. The EPA's attention to data transparency is also responsive to the broader interest in greater data and model transparency observed in the numerous transparency initiatives in the scientific community and federal government, as well as the criticism the EPA has received from members of the public, scientific community, and Congress on the transparency of the scientific basis for EPA's decisions in previous influential scientific information assessments and regulatory actions. The EPA's continued progress toward maximizing transparency is vital to building and maintaining trust with the public and credibility in the Agency's decisions. The EPA also agrees with commenters that the scientific community and government agencies are making great strides in data transparency; however, improvements can still be made over existing policies and mechanisms.
Comment: Commenter (9227) states that the proposal fails to offer a reasoned explanation for its departure from EPA's Scientific Integrity Policy that broadly requires the agency to consider all available scientific information when undertaking rulemakings. The commenter provides that EPA's own existing Scientific Integrity Policy states:
      To support a culture of scientific integrity within the Agency, this policy. . . [r]ecognizes . . . policy makers within the Agency weigh the best available science, along with additional factors such as practicality, economics, and societal impact, when making policy decisions. 
The commenter adds that the proposal conflicts with this policy by restricting what may be the best available science on a given topic from EPA's consideration solely because the underlying data cannot be made public. The commenter contends that public availability of data is neither necessary nor sufficient to ensure that studies constitute "best available science." 
Commenter (9227) states that the proposal fails to offer a reasoned explanation for its departure from EPA's which require EPA to ensure the objectivity of influential scientific information it disseminates by using "the best available science and supporting studies conducted in accordance with sound and objective scientific practices." The commenter provides that EPA considers information to be disseminated when EPA prepares and distributes information to support an Agency decision or regulation or when EPA distributes information in a way that suggests EPA agrees with it, that it supports EPA's viewpoint, or if in the distribution EPA proposes to use it to support or formulate a regulation or agency decision. According to the commenter, thus, the proposal conflicts with the Guidelines by restricting scientific studies that EPA may use to support regulations, which may cause it to disseminate other information to support its regulations that is not based on the best available science. 
Commenter (9227) expresses that the proposal fails to offer a reasoned explanation for its departure from EPA's Peer Review Handbook which acknowledges that "EPA strives to ensure that the scientific and technical bases of its decisions meet two important criteria: (1) they are based upon the best current knowledge from science, engineering, and other domains of technical expertise; and (2) they are credible." The commenter notes that EPA's Science Policy Council Handbook on Risk Characterization also requires reasonableness in the agency's risk assessments, which is achieved when "the characterization is based on the best available scientific information." The commenter asserts that these policies clearly impact EPA's regulatory actions, and thus will be impacted by the proposal. According to the commenter, EPA completely fails to analyze the impact the proposal will have on its ability to comply with these policies and fails to explain why it is changing course or justify its decision to do so. The commenter contends that the proposal fails to even acknowledge that the agency is changing positions.
Response: In the final rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. The EPA may still use pivotal science when the underlying data are unavailable after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator. Title 40 CFR 30.5(d) provides additional factors that the Agency shall use when determining the consideration to afford pivotal science with data that is unavailable for independent validation. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations. The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety.
Comment: Commenter (6915) states that a fundamental premise of the proposed rule is that only studies for which the underlying data are publicly available are valid for decision-making. The commenter notes that this premise is inconsistent with generally accepted practices for conducting and evaluating scientific research. The commenter asserts that the rationale for the premise is not provided: 
      EPA presents no evidence for the conclusion that its current criteria for selecting studies result in scientifically invalid conclusions or overly stringent regulations. 
The commenter adds that the D.C. Circuit has already rejected EPA's proposed approach of excluding studies relying on non-public data as "impractical and unnecessary" when raised by a trade association as part of a challenge to an air quality standard. Am. Trucking, 283 F.3d at 372. 
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. The rule does not categorically exclude studies -- even studies where the underlying dose-response data are not available for independent validation and only applies to dose-response data. The rule requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
13.1.3.4.2 How Would the Proposal Remedy Any Existing Deficiencies?
Comment: Commenters (6125, 6135, 6152, 6194, 6446, 6894, 9227) express concern that the proposed rule does not provide any rationale for how the proposed rule, if promulgated, would remedy any specific deficiencies. Comments include:
 Both the discussion of purportedly desirable attributes in studies to be used for assessment and how the agency should use alternative low dose models are far less detailed and nuanced than either EPA's voluminous existing guidance on risk assessments or recommendations on these topics contained in several NAS reports relating to risk assessment, including the 2007 "Silver Book." According to the commenter, it is impossible to tell from reading the proposal what the real problems, if any, might be, nor how commenters should respond to EPA's proposed fixes. In short, the proposal's description of the "fix" is as inadequate and off the mark as its description of the "problem." (6125)
 EPA has not pointed to a single regulatory action over the thousands it has implemented that was based on faulty science and that this proposal would remedy. The commenter states that, historically, EPA has stated that a review of raw data would be warranted "[o]nly in extreme cases -- for example where there are credible allegations of fraud, abuse or misconduct." The commenter notes that EPA has advocated this position in court proceedings. (6194)
 EPA does not explain why the proposal is preferable to existing recommendations and methods vetted and supported by the scientific community, which endeavor to promote transparency through open science policies and secure data-sharing systems. (6194)
 The proposed rule advances no reason why the accepted methods used in the scientific community to evaluate scientific research should be replaced with the single, threshold requirement in the proposed rule. (6446)
 The proposed rule seems not to trust the present process but offers no alternative or specific critique. (6894)
 EPA's proposal does not question whether this rule would add anything to the current efforts, or whether it would have any effect whatsoever in increasing public accessibility of data. (9227)
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes.
Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information. The EPA's attention to data transparency is consistent with open data initiatives in the federal government and responsive to the broader interest in greater data and model transparency observed in the numerous transparency initiatives in the scientific community and federal government, as well as the criticism the EPA has received from members of the public, scientific community, and Congress on the transparency of the scientific basis for EPA's decisions in previous influential scientific information assessments and regulatory actions. The EPA's continued progress toward maximizing transparency is vital to building and maintaining trust with the public and credibility in the Agency's decisions.
13.1.3.4.3 Existing Transparency Efforts
Comment: Commenters (2492, 6188, 9227) express concern that the proposal does not address the existing steps being taken by the scientific community to make underlying data publicly available where feasible.
Commenter (2492) states that there are now many effective protocols and technologies for making human subjects research highly transparent while protecting private data. The commenter states that EPA should consult with scientists and scientific institutions to learn more about the current best practices in promoting transparent research.
Commenter (9227) states that there are already various ongoing initiatives to make scientific data and models more commonly publicly available, including those noted by EPA. The commenter also refers to the ongoing implementation of the 2016 Plan to Increase Access to Results of EPA-Funded Scientific Research. The commenter states that this Plan aims to maximize access to "research data underlying a publication" resulting from EPA-funded research. The commenter states that it is worth emphasizing the Plan also exempts "research data [that] cannot be released due to one or more of constraints, such as requirements to protect confidentiality, personal privacy, proprietary interest, or property rights." The commenter states that the Plan also explicitly indicates that:
      [i]t is important to recognize that some research data cannot be made fully available to the public but instead may need to be made available in more limited ways, e.g., establishing data use agreements with researchers that respect necessary protections. Whether research data are fully available to the public or available to researchers through other means does not affect the validity of the scientific conclusions from peer reviewed research publications. 
Response: The EPA agrees with commenters that the scientific community and government agencies are making great strides in data transparency; however, improvements can still be made over existing policies and mechanisms. The rule requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. The EPA's continued progress toward maximizing transparency is vital to building and maintaining trust with the public and credibility in the Agency's decisions.
13.1.3.5 Parallel Scientific Process at Odds with the Current Process
Comment: Commenter (1409) argues that this regulation reduces the methods of scientific research--often peer-reviewed research that has stood the test of time--and thus would diminish the EPA's understanding of science and its knowledge in making regulations.
Commenter (6125) states that the proposed rule opens the door to EPA's scientific practices being determined by politically appointed regulators, and not scientists. The commenter asserts that, by contrast, EPA's current science policy guideline-based approach provides the agency with the flexibility to move forward to incorporate new science information in a timely manner, whether it is developed by industry, the academic community, or the agency itself. 
Commenter (6125) asserts that the proposed rule approach is a breach of the fundamental principle of separating risk assessment from risk management, as championed in the 1983 NAS report. 
Commenter (6924) suggests that risk assessment and other scientific methods utilized by the agency should not be the subject of rulemaking, which would freeze those methods in time and prevent EPA from staying abreast of rapidly advancing scientific methods and thus access to the best available science.
Response: As more fully discussed in the preamble of the final rule, the EPA disagrees that this rule is politically motivated, as transparency assumes no political ideology, nor would this rule result in decreased human health or environmental protections, as the benefits of greater data transparency and the significance of reanalyzing and validating study results are well-documented in scientific literature. The EPA believes that data transparency is vital for individuals who have not contributed to the study to be able to verify the quality and strength of published studies and that the opportunity to independently validate the pivotal science that the EPA relies upon is important in furthering scientific understanding and the Agency's mission. When data are widely available, researchers can validate research results and help identify and correct unintended errors, as well as reanalyze the data for new and different purposes, examine novel questions, provide new scientific insights, and improve model development. The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety.
Comment: Commenter (6133) asserts that EPA, other federal agencies, EPA scientific advisory bodies, the NAS, and EPA's SAB have repeatedly and consistently relied upon the best available, peer-reviewed, independent, credible scientific studies -- for which the underlying data are not publicly available -- and found that science to be valid, reliable, trustworthy, and a reflection of the "best available science" that EPA claims as its concern in the proposal. The commenter states that the proposal arbitrarily excludes prior research, studies, and data that do not meet its applicability criteria based on concerns that were never announced to researchers or the public, or deemed necessary by any government agency, at the time the research, studies, or data-gathering were undertaken. The commenter asserts that the proposed rule is strikingly at odds with those scientific practices and their history, with nothing in the rulemaking docket to support casting aspersions on the practices or history sufficient to prohibit EPA from considering such science.
Response: Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations. 
13.2 SNPRM Independent Peer Review
13.2.1 Existing Peer Review Processes and Protocols Are Sufficient
Comment: Commenters (10748, 10866, 11193, 11330, 11332, 11350, 11358, 11381, 11400, 11403, 11404, 11495, 11479, 11892, 11920, 12719, 12727, 13788, 14397) point out that there are already existing peer review methods in place and/or that the EPA already employs a number of methods for evaluating the quality of studies it uses in rulemaking and assuring that the Agency relies on the best available science; these include formal peer review, review by the SAB and other advisory committees and external bodies, systematic frameworks that govern the Agency's review and evaluation of scientific studies that inform its work, Data Quality Objective (DQO) requirements and other stipulations for grantees, quality assurance plans, and the public comment process. Commenters (11330, 12727, 13788) argue that EPA has failed to justify why these mechanisms are insufficient and/or are failing. Commenter (12727) adds that the EPA has also not sufficiently explained how the proposed rule would "improve transparency and the scientific integrity of the regulatory outcomes in an effective and efficient manner."
Commenter (10748) states that peer review processes are already in place as all published scientific studies must undergo a rigorous peer-review process to ensure sound science and the government should let scientific journals and experts continue to decide the quality of studies. 
Commenter (11193) states that the scientific community has long established rules for evaluating science using processes like independent peer review, independent advisory boards, and institutions like the National Academies to review issues of scientific importance. The commenter asserts that these independent peer review methods have, over the decades, led to the greatest scientific advancements in the history of the world and notes that this has happened largely due to the efficacy and thoroughness with which this process evaluates all the available science, including the participation of agency, academic, and industry scientists meeting and using a thorough process for review. 
Commenter (12719) states that publications in peer reviewed journals have had their protocols, data, and statistical analyses assessed during the peer review process leading up to publication. The commenter states that the EPA's current process for reviewing such data already allows for stakeholders to challenge the use of the studies in decisions without making the data publicly available for "independent validation."
Commenter (12727) states that EPA has developed a well-earned reputation for grounding its decisions in rigorous science over approximately five decades of operation spanning administrations of both parties. The commenter notes that this reputation has not come by accident -- rather, it has resulted from deliberate efforts by EPA to utilize time-tested tools including formal peer review; close scrutiny of agency policies, decisions, and studies by the agency's advisory committees and external bodies such as the National Academies; and systematic frameworks that govern the agency's review and evaluation of the scientific studies that inform its work (such as the Preamble to the Integrated Science Assessment and systematic review protocols accompanying IRIS toxicological reviews). The commenter adds that on top of these well-established mechanisms, the ability of the public to comment on the Agency's use of science -- and seek judicial review of arbitrary actions -- constitutes a further safeguard to ensure the Agency acts on the basis of the best available science. The commenter states that, like the original proposal, the Supplemental Notice fails to explain why these existing mechanisms are insufficient -- or to address other information that has emerged since the publication of the original proposal that underscores this basic flaw in the rule. The commenter notes that EPA's own SAB issued a final report on this proposal in April 2020 that sharply criticized EPA's failure to consider the effectiveness of existing transparency mechanisms -- and warned that the proposal could "reduce scientific integrity" and "decrease efficiency."
Commenter (11330) states that EPA presents no evidence at all that peer review, a system that has literally built American scientific might, is failing. In fact, the commenter notes that only two out of ten thousand papers are retracted in the United States. The commenter believes the system is strong, fair, and leads to positive scientific and public health outcomes. 
Commenter (11350) states that EPA science assessments generally include an exhaustive and critical review of relevant studies and a full explanation of how they are being interpreted. The commenter notes that extensive information about each study is typically part of the public record even if all underlying data may not be included. The commenter adds that EPA assessments are normally subject to public comment and independent peer review and members of the regulatory community are free at any time to replicate studies they deem flawed, or to independently seek access to underlying data and reanalyze them. The commenter concludes that the so-called problem that the proposed seeks to fix is largely imaginary and that the stakes for EPA science and the protection of public health are simply too high to finalize this deeply problematic and unnecessary proposal.
Commenter (11479) notes that "evaluation and review" is where the agency already independently validates the research without using underlying data by asking the following questions:
      a) To what extent has there been independent verification or validation of the study method and results? What were the conclusions of these independent efforts, and are they consistent? b) To what extent has independent peer review been conducted of the study method and results, and how were the conclusions of this review taken into account? c) Has the procedure, method or model been used in similar, peer reviewed studies? Are the results consistent with other relevant studies? d) In the case of model-based information, to what extent has independent evaluation and testing of the model code been performed and documented?
The commenter (11479) states that these guiding questions ensure that the Agency does not have to fully reanalyze every study and can instead focus on the weight of the evidence approach, which "considers all relevant information in an integrative assessment that takes into account the kinds of evidence available, the quality and quantity of the evidence, the strengths and limitations associated with each type of evidence and explains how the various types of evidence fit together." The commenter concludes that including data availability in the weighting injects an arbitrary parameter, unrelated to study quality and reliability.
Commenter (11920) states that access to raw data at the stage described in the supplemental notice is typically not a requirement even for peer-review, as scientists can evaluate the robustness and accuracy of findings based on the presentation of processed data and description of the methods and analysis used. The commenter notes that the editors-in-chief of leading scientific journals, including Science, Nature, Proceedings of NAS, Public Library of Science, and Cell, have denounced the original proposed rule noting that "the merits of studies relying on data that cannot be made publicly available can still be judged...As a core skill, scientists are trained in assessing research publications by judging the articulation and logic of the research design, the clarity of the description of the methods used for data collection and analysis, and appropriate citation of previous results."
Commenter (11911) states that EPA does not assess the rule's redundancy given that many peer-reviewed journals have already updated their data transparency policies to encourage data availability.
Commenter (10866) states that truly transparent regulatory procedures should be identical or similar to the principles utilized by virtually all medical professional associations, utilizing a structured multi-step process that EPA could readily adopt. The commenter summarizes these steps as follows:
      a. Develop the question. Find all of the relevant evidence based on pre-established criteria for reviewing the literature; identify the methodologic characteristics of each publication -- e.g., anecdote (lowest form of evidence), randomized controlled trial with appropriate blinding (very high level of evidence). Determine risk of bias: high bias from industry-funded studies; low bias from NIH-funded clinical trials.
      b. Interpret the evidence. Understand association between variables and statistical precision. Synthesize evidence. Formulate evidence-based conclusion.
      c. Create rule in accordance with established procedures including substantial opportunities for public hearings and comments.
Commenter (11892) suggests that independent peer review would be impractical for some specific wetland data, as much of the data used to influence wetland management and regulatory decisions has undergone some sort of peer review through a journal or university committee, or was collected with approved quality control plans, often already approved by EPA. Therefore, the commenter asserts, independent peer review of wetland-related data is not necessary for all wetland data as current procedures and practices are sufficient. The commenter recommends that EPA provide written justification of its findings after EPA conducts an independent peer review to justify significant regulatory decisions. 
Commenter (11403) states that there is no need for EPA to impose additional measures to ensure the reliability of science. The commenter states that peer review is a central component of the scientific process that creates transparent and valid science. The commenter notes that when leading scientific bodies and journals such as NAS, Public Library of Science, Science, and Nature expressed opposition to this type of rulemaking in 2018, EPA should have immediately scrapped this proposal in its entirety. The commenter states that the proposed rule has not addressed the core issue of undermining peer-reviewed science. The commenter notes that the editors of the esteemed journals said in a 2018 statement that the merits of research can still be judged without the sharing of raw data because scientists that conduct peer review are trained "as a core skill" to assess "research publications by judging the articulation and logic of the research design, the clarity of the description of the methods used for data collection and analysis, and appropriate citation of previous results." The commenter adds that in other words, because they conduct their own rigorous review of others' research, scientists do not need raw data to determine if a study was conducted correctly but rather are trained to do so by examining data collection and analysis methods to determine whether the results can be replicated. The commenter states that when doctors prescribe a medicine they have deemed effective, it is not because they have reviewed and analyzed the raw data of patients in the clinical trials themselves, but instead, they have reviewed the published papers on the medication and have seen both that the trials were done correctly and that the results were significant enough to prescribe the treatment. The commenter states that his is no different with research scientists.
Commenter (11404) states that the scientific community has its own process that has been thoroughly developed and extensively vetted. The commenter states that Dr. Nancy Cartwright, through her research, presents a list of criteria used to define "credible science": it is credible because 1) of the scientific method, 2) science is rigorous, 3) science produces general truths by generalizing from well-tested results, 4) science is objective 5) because of the integrity of the scientific community and 6) because interplay between overlapping areas of scientific research serves as an invaluable feedback mechanism. The commenter suggests that the six defining characteristics, at a high level, encompass the rigorous, multi-year processes of feedback and peer review, discussion with colleagues, replication, and publication and adds that science that meets the criteria is therefore credible and trustworthy. The commenter notes that this list does not include a requirement to make all data "publicly available in a manner sufficient for independent validation," nor is such a requirement included in the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines either, as that would be overly burdensome and unnecessary.
Commenter (13788) states that EPA has long relied on peer review and already has extensive policies in place regarding influential scientific information to assess the quality of the scientific research. The commenter states that EPA provides no explanation as to why those policies are not sufficient to allow validation of scientific studies and information relied on by the Agency. The commenter suggests that simply stating that "EPA shall conduct independent peer review on all pivotal regulatory science used to justify significant regulatory decisions and on all pivotal science underlying influential scientific information, consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review," fails to explain why this does not already occur under the Agency's past and current practice. The commenter concludes that if validation is truly the aim of this rulemaking, EPA must explain why existing practices for peer review are insufficient to validate scientific studies as EPA has failed to do so, indeed, the supplemental proposal contains very little discussion of peer review at all.
Response: The EPA agrees with the comments, in part. All Agency technical work products go through rigorous internal review and, when appropriate, peer review. The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. Adequate peer review is peer review that is consistent with the requirements of the OMB Peer Review Bulletin and EPA's Peer Review Handbook. The EPA is subject to the OMB Bulletin and its provisions have been incorporated into the Agency's peer review handbook (Ref. 44). The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will lead to consistent implementation of those procedures across the Agency.
The EPA will continue conducting peer reviews according to the peer review policies and practices that were in place before the rule goes into effect. As discussed in the final rule preamble, the rule requirements are not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review does not typically include reanalysis of the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information) and, thus, peer review is not considered a replacement for the data availability requirements of this rule. Therefore, as detailed in the preamble of the final rule, EPA is finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate peer review, consistent with the EPA's Peer Review Handbook, of individual studies identified as pivotal science if the studies have already undergone journal peer review. If the Agency conducts peer review on pivotal science, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied in analyzing dose-response data and the implications of those assumptions for the results. 
If implementing this procedural rule would result in conflicts with the guidance found in the Agency's existing Peer Review Handbook, the conflicting requirements found in the Handbook will yield to this final rule. The EPA disagrees that the rule adds another layer of peer review that could result in unnecessary delays. EPA clarified in 40 CFR 30.6 that "the EPA will evaluate whether or not to conduct additional peer review of individual studies identified as pivotal science if the studies have already undergone peer review." This should ensure that no unnecessary peer reviews are conducted that could cause undue delay. 
This rulemaking is not intended to modify the Agency's interpretations of "best available science." The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. Further, the EPA plans to promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
13.2.2 Concerns Related to Transparency in EPA's Peer Reviews 
Comment: Commenter (11381) states that the absence of provisions requiring the Agency to obtain third-party information and ensure that their own work is reproducible undermines the effectiveness of EPA's highly developed peer review program. The commenter states that a key element in the supplemental proposal is continued reliance on peer review of pivotal science, but peer review of all types is shackled by a lack of transparency. The commenter states that EPA's own peer review program includes no expectation or requirement that Agency documents submitted for review will be transparent and the supplemental proposal maintains these deficiencies intact, doing nothing to ensure that the Agency will affirmatively comply with its own final rule with respect to even its own peer reviewers. The commenter notes that, instead, new Part 30 would assign to peer reviewers a limited and much less important task: "articulat[ing] the strengths and weaknesses of EPA's justification for the assumptions applied and the implications of those assumptions for the results," which is what Agency peer reviewers have done for years and does nothing to enhance transparency.
Commenter (11381) states that governments use peer review to show that the science behind their policy decisions is correct and notes that government peer review is always owned by the sponsoring agency and that agency chooses the reviewers, writes their charge, and decides whether to respond to their comments. The commenter states that government peer review panels often have limited expertise. The commenter adds that a panel consisting of 15 members may seem to be rich in expertise, but that expertise is rapidly diluted when the review task before them is broad or the available time is short. The commenter states that government-wide peer review guidelines require agencies to limit peer review to scientific and technical matters and avoid asking peer reviewers to opine on policy.
Commenter (11381) recommends that the quality of Agency-sponsored peer review be enhanced by ensuring that the information they provide reviewers is fully transparent. The commenter states that peer reviewers, such as members of EPA's SAB, are a scarce resource and their limited time would be much better spent if the Agency demonstrated compliance with the final transparency rule before seeking peer review. The commenter also recommends that the Agency should strengthen its peer review program by requiring Agency offices to comply with the IQG before seeking peer review. The commenter states that EPA's Peer Review Handbook includes no such requirement, a deficiency that has been as persistent as it is unjustifiable. The commenter recommends that the Handbook, and the policy on which it is based, should be immediately revised to make IQG compliance a prerequisite for peer review.
Response: The EPA is subject to the OMB Bulletin and its provisions have been incorporated into the Agency's peer review handbook (Ref. 44). The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will lead to consistent implementation of those procedures across the Agency.
It should be noted that in this final rule, rather than considering all studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact as "pivotal science," the EPA is balancing transparency and feasibility by focusing on those studies that inform the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect. Thus, "pivotal science" includes only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). These are the studies that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information. If peer review under section 30.7 is warranted, it would be conducted on "pivotal science" and not on every dose-response study considered by the Agency in developing a significant regulatory action or influential scientific information. The EPA is finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate peer review, consistent with the OMB Bulletin for Peer Review and the EPA's Peer Review Handbook, of individual studies identified as pivotal science if the studies have already undergone journal peer review. If the Agency conducts peer review on pivotal science, the EPA shall ask peer reviewers to articulate the strengths and weaknesses of the justification for the assumptions applied in analyzing dose-response data and the implications of those assumptions for the results.
13.2.3 Supplemental Proposal Violates the Principles of the OMB Final Information Quality Bulletin for Peer Review 
Comment: Commenter (12715) states that the supplemental violates the principles of OMB Memorandum M-05-03, Final Information Quality Bulletin for Peer Review, which establishes government-wide guidance "aimed at enhancing the practice of peer review of government science documents." The commenter states that contrary to this OMB memorandum, the SNPR would prevent or limit EPA's reliance on peer-reviewed research unless the underlying data can be made available for public review. The commenter notes that for studies with restricted data that would require a tiered access system to achieve public availability, EPA does not state how such systems will work in practice other than to note that it is conducting a pilot study using the Research Data Center's "secure data enclave" to host EPA datasets in a restricted use environment. The commenter states that development of standard data repositories "is still ongoing," and that EPA itself "does not intend to make all data and models underlying pivotal regulatory science and pivotal science publicly available. The commenter notes that it is not clear if EPA will make data and models publicly available via some tiered access system where it does have access to data and models, or if EPA intends that scientists and other researchers must themselves establish and maintain a tiered access system that ensures privacy protections for sensitive data. The commenter states that despite all these uncertainties regarding the mechanisms for public access, the supplemental proposal claims, with no support, that studies where the data is publicly available -- through a complex tiered access system or otherwise -- are superior to studies where the data is not publicly available, regardless of the rigor of the underlying science and the thoroughness of the peer review process. The commenter asserts that disregarding or giving less weight to studies simply because the underlying data is not publicly available undermines OMB principles of peer review. Rather than excluding evidence from consideration absent the raw data's availability and pursuing adoption of a tiered access system that has yet to be developed or shown to be workable, the commenter recommends that EPA should continue to adhere to existing, time-tested, and well-established peer review requirements and guidelines to ensure that a consistent method is used to evaluate the quality of any data utilized by the agency for finalizing influential scientific information and significant regulatory decisions.
Response: The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. Adequate peer review is peer review that is consistent with the requirements of the OMB Peer Review Bulletin and EPA's Peer Review Handbook. The EPA is subject to the OMB Bulletin and its provisions have been incorporated into the Agency's peer review handbook. The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will lead to consistent implementation of those procedures across the Agency. The EPA disagrees that the final rule violates the principles of OMB's Bulletin for Peer Review (Ref. 8). The EPA is finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate peer review, consistent with the EPA's Peer Review Handbook, of individual studies identified as pivotal science if the studies have already undergone journal peer review. 
Although not exhaustive or exclusive, the EPA has identified several factors that balance some of the important technical considerations the EPA will consider in addition to data availability and that are particularly relevant to the stage of the analysis where dose-response data are used. The EPA will thus consider the following factors:
 the quality of the study relative to other studies for which the dose-response data are available; 
 the extent to which there are other studies for which the dose-response data are available;
 the sensitivity of the conclusions in the significant regulatory action or influential scientific information based on the use of the study;
 the extent to which the study is fit for the purpose or intended use relative to other studies for which the dose-response data are available;
 the use of exposures or doses in a range and duration that is relevant for the intended use and that minimizes the need for extrapolations;
 the extent to which the study is supported by other scientific evidence;
 the extent to which the study accounts for unique scientific considerations;
 the extent to which the study minimizes the use of defaults and assumptions, uses appropriate and strong statistical methods, and includes a robust representation of uncertainty and confidence intervals; and
 the study's consideration of a broad range of parametric dose-response or concentration-response models, a robust set of potential confounding variables, nonparametric models that incorporate fewer assumptions, various threshold models across the dose or exposure range, and models that investigate factors that might account for spatial heterogeneity.
The EPA would like to clarify that the final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available for independent validation. As discussed in the final rule, underlying dose-response data are considered available if they can be accessed through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. The rule does not prescribe a type of data sharing mechanism. By placing greater emphasis on the availability of dose-response data underlying pivotal science, the rule will allow for greater scientific scrutiny as EPA decision makers are developing significant regulatory actions and influential scientific information and increases the likelihood that any errors will be identified and corrected. Greater transparency is also inherently valuable as a principle of good government and provides benefits to the public at large, including reducing the risk of errors in EPA analyses and in the science such analyses rely upon. The ability for independent subject matter experts to validate pivotal science will facilitate more effective comment and engagement with the public during development of future significant regulatory actions and influential scientific information.
13.2.4 Resource Burdens of Peer Review Requirements 
Comment: Commenter (11193) suggests that if every comment made by a member of an advisory panel or every comment made by a peer reviewer required that the underlying data, models, and analysis for that comment to be made publicly available, this long-established process would grind to a halt, weakening the foundations of scientific decision-making. 
Commenter (11919) asks that, as states rely on EPA research, EPA consider how it will meet the additional workload required in the proposed regulatory text at 40 CFR 30.7, calling on EPA to conduct independent peer review, without impacting the important work of EPA research scientists. 
Commenter (11388) states that the proposal implies that EPA will have to conduct additional Agency-led peer review for every applicable study. The commenter does not agree with this provision as most of these studies have already undergone expert, independent peer review and creating an additional level of peer review to be conducted by the agency adds unnecessary hours and labor to EPA's decision-making procedures. The commenter recommends this requirement be removed from the final rule.
Commenter (12430) notes that as a result of the changes in the supplemental proposal, EPA would effectively have to conduct independent review for all regulatory or influential science, regardless of the peer review process to which that science has already been subjected, which comprises a comprehensive and highly burdensome new task for the agency. The commenter states that EPA would have to coordinate the peer review (or perhaps conduct the peer review itself), pay for it, wait for it to be completed, and ensure that it complied with the mandate under the supplemental proposal to "articulate the strengths and weaknesses of U.S. EPA's justification for the assumptions applied and the implications of those assumptions for the results." The commenter states that the supplemental proposal fails to acknowledge this burden or explain where EPA would find the resources and staff to perform these tasks. The commenter notes that California statute requires "external scientific peer review of the scientific basis for any rule proposed for adoption by any board, department, or office within" CalEPA. The commenter adds that in California statute, however, "scientific basis" means "those foundations of a rule that are premised upon, or derived from, empirical data or other scientific findings, conclusions, or assumptions establishing a regulatory level, standard, or other requirement for the protection of public health or the environment" The commenter states that in contrast, the supplemental proposal would require EPA to independently peer review, not the foundations of a science-based rulemaking, but the actual studies, data, and models underlying both every rule and every piece of "influential scientific information."
Commenter (10748) recommends that providing technical and financial support is essential for this rule, but states that it would also be costly to the Agency and is duplicative since there is already a peer review process in place within the scientific community that is trustworthy.
Response: The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. Adequate peer review is peer review that is consistent with the requirements of the OMB Peer Review Bulletin and EPA's Peer Review Handbook. The EPA is subject to the OMB Bulletin and its provisions have been incorporated into the Agency's peer review handbook. The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will lead to consistent implementation of those procedures across the Agency. 
Further, the EPA would like to clarify that the rule requirements are not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble of the bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review does not typically include reanalysis of the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information) and, thus, peer review is not considered a replacement for the data availability requirements of this rule. 
Therefore, the EPA is finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will evaluate whether or not to initiate peer review, consistent with the the EPA's Peer Review Handbook, of individual studies identified as pivotal science if the studies have already undergone journal peer review. 
The preamble of the final rule specifies, "given the costs of the current robust process for identifying and reviewing scientific studies and documentation that are existing Agency practice, as well as that the determination of dose-response data availability is limited to pivotal science underlying significant regulatory actions and influential scientific information, the EPA anticipates that the incremental costs of this rule will be small relative to current administrative costs for developing significant regulatory actions or influential scientific information."
13.2.5 Considerations of Bias and Independence of Peer Reviewers
Comment: Commenter (11193) asserts that putting the authority of the peer preview process in the hands of a political appointee, who would have the authority to waive requirements for the data to be public on a case-by-case basis, rather than in the hands of an apolitical team of scientists is, itself, indefensible scientifically, as this would allow politics, and not science, to determine the basis for regulatory policies designed to protect the environment and the American people.
Commenter (10866) suggests that an additional pertinent consideration and criteria, if EPA were to adopt procedures similar to the principles utilized by medical professional associations, includes whether the study is published in a peer-reviewed journal. The commenter observes that in this sphere it should be noted that there are industry-sponsored journals that are peer reviewed; these studies should not be considered because of the risk of bias. The commenter also suggests that the impact factor of the journal should be considered, noting that in general, the highest impact factor journals are the sources of the most carefully reviewed and reliable studies. The commenter recommends that these studies should receive careful attention but should not be immune to the deficiencies noted above.
Commenter (11920) asserts that this rulemaking must also be considered in the context of the Administration's broader assault on science. The commenter suggests that this supplemental notice is designed to cast unjustified doubt on science that has gone through the independent peer-review process, while promoting unscientific criteria for evaluating science. The commenter states that EPA has failed to show any compelling need or justification for the proposed regulation, nor has it demonstrated that the purported benefits of this regulation would outweigh the substantial costs in compliance as well as increased risks to human health and the environment. The commenter supports the language requiring independent peer review in §30.7, adding that it is imperative that the "peer review" be truly independent and that there is transparency as to the peer review process (i.e., who are the peer reviewers, do they have a direct or indirect interest in the subject matter being reviewed, do they have established positions that they have already taken on the "science/subject matter" that they are reviewing, etc.). The commenter recommends that EPA strengthen the "independence" and transparency of the peer review process by identifying who does the peer review, whether they have a direct or indirect interest in the subject matter, previous positions the reviewer may have taken on the matter being reviewed, and affiliation of the reviewer, including any connections between the reviewer and his/her organization and the subject matter.
Commenter (11381) asserts that government peer review panels often include stakeholders who have key policy interests, but little or no expertise with respect to the scientific or technical issues under review and that this practice is inconsistent with government-wide peer review guidance. The commenter states that stakeholders are essential for advisory groups chartered to give policy advice, but they do not belong on scientific peer review panels.
Commenter (11381) suggests that government peer review focuses inordinately on a select group of potential conflicts of interest, typically restricted to financial interests related to for-profit enterprises while other financial interests, such as financial interests related to non-profit enterprises and financial interests with respect to non-profit or agency funding, are ignored. The commenter states that conflicts of intellectual interest are desirable, for it is where such interests arise that peer review can be expected to be most rigorous. The commenter adds that on the other hand, scientists with intellectual interests aligned with an agency, or with researchers whose pivotal science is involved, are least likely to perform a review with care and are also are most likely to defend the scientific information under review for nonscientific reasons. The commenter concludes that the primary conflict of interest in government peer review is provided by employees of the sponsoring agency as they are by far the most personally invested in obtaining specific outcomes.
Response: This rulemaking is not intended to modify the Agency's interpretations of "best available science." The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. Adequate peer review is peer review that is consistent with the requirements of the OMB Peer Review Bulletin and EPA's Peer Review Handbook. The Agency will continue conducting peer reviews according to the peer review policies and practices that were in place before the rule goes into effect. The EPA Peer Review Handbook, 4[th] Edition, describes how peer reviewers are selected by the Agency. As discussed in the final rule preamble, the EPA would like to clarify that the rule requirements are not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." If implementing this procedural rule would result in conflicts with the guidance found in the Agency's existing Peer Review Handbook, the conflicting requirements found in the Handbook will yield to this final rule.
The EPA appreciates the recommendation related to selecting a balanced peer review panel; however, the recommendation is not relevant to this rule and will not be considered in finalizing the rule. The EPA will continue to conduct peer reviews according to the peer review policies and practices that were in place before the rule goes into effect. The EPA Peer Review Handbook, 4[th] Edition, describes how peer reviewers are selected by the Agency.
The EPA is finalizing the language at 40 CFR 30.6 (formerly 40 CFR 30.7 in the 2018 proposed rule and the 2020 SNPRM) to clarify that the Agency will conduct independent peer review consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review and the exemptions described therein. The EPA will evaluate whether or not to conduct additional peer review of individual studies identified as pivotal science if the studies have already undergone journal peer review. 
Although not exhaustive or exclusive, the EPA has identified several factors that balance some of the important technical considerations the EPA will consider in addition to data availability and that are particularly relevant to the stage of the analysis where dose-response data are used. The EPA will thus consider the following factors:
    the quality of the study relative to other studies for which the dose-response data are available; 
    the extent to which there are other studies for which the dose-response data are available;
    the sensitivity of the conclusions in the significant regulatory action or influential scientific information based on the use of the study;
    the extent to which the study is fit for the purpose or intended use relative to other studies for which the dose-response data are available;
    the use of exposures or doses in a range and duration that is relevant for the intended use and that minimizes the need for extrapolations;
    the extent to which the study is supported by other scientific evidence;
    the extent to which the study accounts for unique scientific considerations;
    the extent to which the study minimizes the use of defaults and assumptions, uses appropriate and strong statistical methods, and includes a robust representation of uncertainty and confidence intervals; and
    the study's consideration of a broad range of parametric dose-response or concentration-response models, a robust set of potential confounding variables, nonparametric models that incorporate fewer assumptions, various threshold models across the dose or exposure range, and models that investigate factors that might account for spatial heterogeneity.
Further, the EPA plans to promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
The preamble of the final rule specifies, "given the costs of the current robust process for identifying and reviewing scientific studies and documentation that are existing Agency practice, as well as that the determination of dose-response data availability is limited to pivotal science underlying significant regulatory actions and influential scientific information, the EPA anticipates that the incremental costs of this rule will be small....relative to current administrative costs for developing significant regulatory actions or influential scientific information."
 Chapter 14: Implementation (Including Implementation Benefits and Costs)
The Agency states in the notice of proposed rulemaking (NPRM) that it believes the benefits of this proposed rule justify the costs. The proposal provides that the net benefits from wrongful cost estimations and greater reproducibility would cover the costs of implementation.
While the supplemental notice does not expand the benefits and costs discussion from the original rule, the expansion of the scope established by the supplemental notice of proposed rulemaking (SNPRM) drew many additional comments on the anticipated benefits and costs. 
Several comments were received on both the NPRM and the SNPRM on the benefits and costs that would be incurred from the rule's implementation. In this section, implementation topics are organized under sections 14.1 NPRM Implementation and 14.2 SNPRM Implementation.
Section 14.1: NPRM Implementation
       Lack of Clarity How the Rule Would be Implemented 
       Incorporating Stronger Data and Model Access Requirements in Grants and Cooperative Agreements 
       Building Upon Other Federal Agencies' Policies Regarding Grantee and Cooperator Requirements
       Platform for Increasing Public Access to EPA-Funded Data
       Methodologies and Technologies Designed to Provide Protected Access to Data
       Other Implementation Issues that May Require Special Consideration 
       Timing of Implementation 
       Impacts on Other Federal Agencies, States and Tribes 
       Communications 
       Implementation Costs 
Section 14.2: SNPRM Implementation
       Inter-organizational Efforts
       The EPA's Readiness to Implement the Rule
       Implementation Timing/Process
       Implementation Costs and Benefits
       Implementation Bias
       Rule Includes Insufficient Implementation Detail/Clarity
The specific comments provided in this Chapter reflect the comment letters that provide arguments supported by detailed rationale, examples, and references/documentation. The Docket ID numbers for these commenters are cited throughout this Chapter. In addition, some general comment letters may also include comments related to comments included and responded to in this Chapter. These comments are summarized in Appendix B. Note that points and issues raised in these general comment letters may also be raised by the more detailed comments summarized in this Chapter. The responses provided in this Chapter adequately address these more general comments.
14.1 NPRM Implementation
14.1.1 Lack of Clarity How the Rule Would be Implemented 
14.1.1.1 No Part of the Agency Should Be Responsible for Implementing Rule Requirements
Comment: Commenter (0671) contends that no part of the Agency should be responsible for implementing this proposed regulation because it runs counter to the text and intent of the Clean Air Act (CAA) and amendments. Commenter (8281-PH20) asserts that EPA should focus on implementing existing initiatives and guidelines for improving data sharing and transparency at the federal government. 
Response: The EPA is authorized to issue this rule under its authority to promulgate housekeeping regulations governing its internal affairs. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. Some of these subsequent actions will be substantive rules issued under the associated environmental statutes and will be subject to judicial review.
14.1.1.2 General Lack of Specificity of Implementation and Implementation Responsibilities
Comment: Commenters (5165, 6126, 6144, 6147) express concern regarding the lack of specificity of implementation responsibilities within the EPA in the proposed rule. Commenters (5165, 6144, 6351, 6909) assert that there is little detail on how EPA itself would implement this rule and/or how it would coordinate with other entities on shared responsibilities to review the best available science to protect public health and the environment. 
Commenter (6351) states that the proposal fails to discuss how EPA would implement the proposal. The commenter notes that the proposal offers no process for public hearing or even consultation with the SAB over implementation. As written, the commenter suggests that the proposal would require review and assessment of volumes of existing research and revisions to internal processes yet to be determined. It also, according to the commenter, seems to give arbitrary decision-making authority to the Administrator to determine the fate of such research.
Response: Implementation guidelines are forthcoming. Further, the EPA plans to promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. 
14.1.1.3 Disclosure Avoidance and De-Identification of Confidential Data
Comment: Commenter (6126) states that the proposed rule does not specify who will be responsible for disclosure avoidance and de-identification of identifiable, confidential data. The commenter provides that some research subjects have been promised confidentiality as a condition of participation in studies, and researchers may be legally and ethically obligated to provide it. The commenter asserts that there is no current organizational structure at EPA that is well-suited to implement this regulation and manage the important privacy and confidentiality issues that may arise. The commenter states that, similar to the recommendations of the Commission on Evidence-Based Policymaking (Commission), EPA currently lacks a leadership structure well-suited for providing oversight and coordination of data resources. The commenter makes the following recommendations: 
 EPA should assign responsibilities to senior leaders for data and evidence activities. 
 EPA should develop a Confidential Information Protection and Statistical Efficiency Act (CIPSEA)-approved statistical unit capable of managing the relevant de-identification and other confidentiality issues. If EPA were to be approved for a CIPSEA unit by Office of Management and Budget (OMB) and the agency was willing to meet the requirements of CIPSEA for a unit within the agency, strong confidentiality protections could be in place.
Commenter (6126) notes the following recommendations of the Commission and encourages EPA to identify individuals or develop a statistical unit that can perform the functions:
 Recommendation 3-3 suggests that a senior official in agencies be identified to ensure sufficient data stewardship and implementation of privacy-protective measures. 
 Recommendation 5-1 suggests that agencies should identify a chief evaluation officer to help coordinate evidence-building activities. 
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose obligations on the EPA or any other party to make data available or requirements to ensure that personally identifiable information (PII) and confidential business information (CBI) are not disclosed. The Agency will continue to follow existing procedures for protecting PII and CBI as appropriate. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
14.1.1.4 Review and Enforcement of Confidentiality Agreements or Memoranda of Understanding Established for Data Access
Comment: Commenter (6126) states that the proposed rule does not specify who will review and enforce any confidentiality agreements or memoranda of understanding established for data access. The commenter expresses concern that EPA's existing organizational infrastructure may be insufficient to fulfill such obligations and necessary activities to protect privacy and confidentiality.
Response: The final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available.
14.1.1.5 Adjudication of Issues that May Arise
Comment: Commenter (6362) suggests EPA may also want to consider establishing an office similar to that of National Institutes of Health (NIH) Office of Research Integrity to adjudicate any issues that may arise in the administration of its practices under this rule.
Response: The EPA has a Scientific Integrity Official who implements the Agency's scientific integrity policy. The Scientific Integrity Official also adjudicates any allegations of violation of the Agency's scientific integrity policy.
Comment: Commenter (4534) states that the process to deny a request for data should be substantial. The commenter states that, in recent years we have witnessed several branches of the government not release data or records that were requested by the public or other governmental agencies for political reasons. The commenter provides that the denial was framed that the release of information would be a national security risk.
Response: This is a rule of internal procedure and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
14.1.1.6 Establishment of an Epidemiological Study Council
Comment: Commenter (6862) proposes that an Epidemiological Study Council be established with the goal of creating a transparent document reflecting a thorough review of the study. The commenter suggests there should be a middle ground where the Agency determines that the study can be used for regulatory decisions when all supporting data are not available. The commenter asserts that the decision to use the study in the absence of an independent review of the original data should not be the responsibility of one individual. The commenter provides that the responsible individuals would be identified and would defend their decisions from objections from either industry or public interest groups. The commenter suggests the following proposal to justify the use of an epidemiological study when not all of the supporting data are available or to otherwise state clearly why the study cannot be used without the submission of additional data:
      Each epidemiology study occurring in the open literature or otherwise needs to have a supporting formal Record of Epidemiological Study Review (ROESR) that clearly delineates the justification for a decision to include or not include the study in the regulatory decisions for the chemical or environmental situation. The production of the ROESR will consist of the separate and independent reports of several Sub-Committees that will be integrated by the Epidemiological Study Peer Review Council (ESPRC) and signed by the Council Chairperson and each Council member. It is important to have independent Sub-discipline committees review the study so that the biases of other sub-disciplines are minimized. The Sub-committee members should be drawn from the relevant staff throughout the Agency as well as other government agencies as needed. Each member of the Sub-committees will sign their respective Sub-committee reports. The roles of the six suggested Sub-Committees and Council Chairperson are as follows:
      1. Ethics Evaluation Sub-Committee. This sub-committee will evaluate all aspects of the ethical treatment of the individuals in the cohorts. This includes that it will identify how additional data can be provided to the Agency in a manner that will assure the identities of the individuals are protected.
      2. Endpoint Evaluation Sub-Committee. This evaluation would limit its conclusions with regard to how well the character of the endpoint itself was assessed for and whether or not the effect reported is plausibly related to treatment or within normal variation. The latter includes how many subjects in a cohort are needed to make a meaningful statistical difference for the particular lesion in question.
      This Sub-committee will provide commentary on the consistency of characterization of the endpoint, resolution of confounds as well as any known other chemicals or conditions that affect the normal distribution of the endpoint. 
      The experts in this Sub-committee will vary depending upon the endpoint claimed by the epidemiological study and consist of experts in cancer, behavioral response, or other appropriate discipline for interpretation of the significance of the endpoint.
      3. Exposure Assessment Sub-Committee. The role of the exposure assessment Sub- committee is limited to determining that the methodology and reliability used to determine the exposure of the cohort is adequate and appropriate. The Sub- committee would provide commentary on whether or not exposure was supported by analytical chemical data, by oral history or otherwise. Also, if the actual persons exposed provided oral history by direct conversation, survey or by telephone. The committee will determine if indirect exposure information was provided by a relative (or friend or coworker) is reliable.
      4. Statistical Evaluation Sub-Committee. In the initial review of the study, the sole role of the statistical evaluation sub-committee is limited to determining if the statistical methods were/were not appropriate and adequately conducted to support the conclusion of the report. This committee would not redo the statistics at this time since the original data would be needed. Input from Sub-committees 1 (ethics), 2(3ndpoint evaluation) and 3(exposure assessment) that clarifies all issues that are the responsibility of their respective disciplines would be needed before statistical reanalysis could be conducted. Thus, receipt of the original data in a manner that satisfies Sub-committees 1, 2 and 3 and well as any requests by the Statistical Evaluation Sub-committee before any statistical reanalysis would be conducted. 
      5. Analytical Chemistry Assessment Sub-Committee. The role of the analytical chemistry committee is limited to determining if the analytical techniques/ methodology were appropriate and adequate for the study. In some cases where no analytical chemistry data were generated, the committee will comment on the need to have included such data and/or if quantitative analysis of exposure data was even possible. 
      6. Animal Toxicity and Structure Activity Relationship (SAR) Assessment Sub- Committee. In most cases with chemicals, the Agency should already have a battery of standard animal toxicity studies that were required for the registration of that chemical. There may also be studies in the open literature attempting to define the toxicity of the chemical. Thus, the person(s) responsible for evaluating the animal toxicity studies would be a member of this Sub-committee. In addition, this Sub- committee will have individuals that can address the SAR issues in relation to the possibility that the chemical could affect the endpoint in question.
      It should be noted here that a Record of Review (ROR) signed by at least two qualified toxicologist is needed for any publication supporting a mechanism of action for the chemical.
      If the endpoint occurs in animals or has SAR correlates, such information can support the epidemiological findings. If not, it does not automatically dismiss the epidemiological findings since the effect on the endpoint may be unique to humans.
      7. Council Chairperson, Council meetings and resulting product: ROESR.
      The Council Chairperson should not be in the same program that has responsibility for regulating the chemical or environmental condition. That is, if the chemical is a pesticide, the Chairperson cannot be from the Office of Pesticide Programs (OPP). Representatives of OPP may still be on the Council if an epidemiological study with a pesticide is being reviewed.
      At the initial Council meeting, the Council will discuss the conclusions of each the six Sub-committees. In some cases, the Council may determine that the association between exposure and the endpoint is otherwise strong enough to support regulatory action to protect the public health and no additional information is needed from the study authors.

During the initial meeting the Council can overrule a request for additional data made by any of the Sub-Committees. However, clear justification for overruling a Sub-Committee's decisions must be provided. 
      In other cases, the Council may determine that there is a definite need to require additional data supporting the study methodologies and conclusions are required. The Council would then notify the authors articulating of the need for the specific additional information.
      The Council will also determine if the authors of the epidemiological report should be requested to attend a closed meeting with the Council and Sub-committee members. The purpose of the meeting will be an opportunity for the study authors to address any concerns that the Sub-committees have. This will include problems with providing any additional information that the Sub-committees requested. If the study authors refuse to attend a requested meeting, the Council will determine if the study should be rejected outright.
      The Council Chairperson (or secretary of the Council) will prepare the ROESR product that contains the decision with supporting justification. The ROESR product would include the report of the Council with the Chairperson and Council members' signatures. Separate attachments for the signed Sub-Committee reports and the original epidemiological report (i.e. a publication) will also be attached. An executive summary of any meeting(s) with the study authors will also be appended.
      Purpose. The purpose of this procedure is to assure that responsible persons for each of the critical Sub-disciplines independently reviewed the study in terms of their expertise. The Council Chairperson and members of the Council and the sub-discipline committees own their decisions and are responsible for defending them. If industry or any concerned public interest groups object to the conclusions, they can address the conclusions presented in the ROESR and/or the sub-disciplines. Therefore, the decisions in the resulting ROESR should be transparent. 
      The ROESR is not a final conclusion (no documents in EPA should be considered final). It is very likely that a Science Advisory Panel will ultimately review the ROESR. Also, as new information is generated, the ROESR can be updated and the recommendations for use (or otherwise) can be adjusted as appropriate.
Response: EPA will continue conducting reviews according to the existing peer review policies and practices, including any programmatic specific processes.
14.1.1.7 Oversight of Data Storage and Sharing
Comment: Commenter (2426) suggests that, depending upon the identity and role of the governing entity [in determining exemptions], the same body or a separate third-party entity should oversee the storage of the data and grant access to qualified researchers. The commenter states that the independence of the entity that would grant access to the data also ensures that the EPA is not put in the paradox of collecting data to ensure data transparency and then possibly having to reject requested access to that data. 
Response: The final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available.
Comment: Commenter (6117) suggests that the EPA focus on creating a data sharing system for long term human subject studies where privacy concerns exist. The commenter suggests that, in order to make data available to external researchers for scientific purposes, the EPA follow previous efforts to facilitate the sharing of data. The commenter recommends: 
 An independent intermediary group should be created and should partner with the EPA to oversee data requests. By allowing an independent group to have full jurisdiction to make decisions regarding data access, potential political motivations can be avoided. 
 The independent group should ensure that data are made available for scientific research that is aimed at advancing knowledge. Data requestors should provide basic information about the principal investigators, all study personnel and funders, and submit a detailed proposal outlining specific aims and specifying study methodology. All data requests should undergo external review, to facilitate feedback from independent experts in the field to verify scientific merit. To ensure transparency, the independent group should make all of the data requests publicly available, along with the reasons for granting or denying data requests. 
 All requestors should be required to sign a Data Use Agreement (DUA), which states that access to the data will be used to enhance knowledge and that all findings will be made publicly available through publications and meetings. 
 The independent group should ensure that the scope of the analyses is limited to the specific aims set out in the proposal and that additional objectives are outlined in new submissions. 
Response: The final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available.
The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying data or models are publicly available or available through restricted access. Restricted or tiered access in this rule means that the underlying data or models are be available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
14.1.2 Incorporating Stronger Data and Model Access Requirements in Grants and Cooperative Agreements
14.1.2.1 Support Stronger Data and Model Access Requirements/Cooperative Agreement and Grant Requirements
14.1.2.1.1 Encourage and Ensure that More Data and Models be Publicly Available
Comment: Commenter (6918) supports efforts EPA can reasonably undertake to encourage and ensure, as a general matter, that more data and models underlying EPA decisions are available for the public to use and independently verify, while respecting lawful restrictions on data access. The commenter states that EPA currently underutilizes certain tools it has. The commenter states that, as an agency of the federal government, EPA currently possesses the authority to obtain data produced by federal grants. 2 C.F.R. § 200.315(d)(1) ("The Federal Government has the right to: (1) Obtain, reproduce, publish, or otherwise use the data produced under a Federal award. . . ."). The commenter further states that EPA also has the authority to authorize others to use such data. Id. § 200.315(d)(2) ("The Federal Government has the right to: ... (2) Authorize others to receive, reproduce, publish, or otherwise use such data for Federal purposes."). The commenter states that current regulations recognize the importance of public access to data generated by federal grants and that research data produced from federal grants are subject to the Freedom of Information Act (FOIA) in a specific way. The commenter notes that when a federal agency uses research findings produced from federal grants "in developing an agency action that has the force and effect of law," if a FOIA request is made for the research data relating to these findings, the agency that issued the grant "must request, and the [grantee] must provide, . . . the research data so that they can be made available to the public" through the FOIA process. Id. § 200.315(e)(1). The commenter states that these regulations also recognize that release of certain research data may be inappropriate, specifically, the definition of "research data" for FOIA purposes exempts the following from requirements for public release:
      (i) Trade secrets, commercial information, materials necessary to be held confidential by a researcher until they are published, or similar information which is protected under law; and
(ii) Personnel and medical information and similar information the disclosure of which would constitute a clearly unwarranted invasion of personal privacy, such as information that could be used to identify a particular person in a research study.
Response: EPA has implemented the public access requirements for grantees and contractors as required by the February 22, 2013, OSTP memorandum "Increasing Access to the Results of Federally Funded Scientific Research" and the 2016 EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research." Grantees and contractors are required to make their data and models underlying any peer-reviewed journal articles resulting from EPA-funded research accessible to the public. This is requirement went into effect for any funding granted as of September 26, 2019. EPA's Environmental Dataset Gateway provides metadata records stating where those data will be available. PII and CBI are still protected under the public access requirements.
The EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research" requires the data underlying a journal article resulting from research funded by EPA be made publicly accessible. However, in the final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. The step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
14.1.2.1.2 National Academy of Sciences (NAS) Guidance 
Comment: Commenter (3971) notes that the NAS has issued guidance on this matter and suggests that transparency, as well as privacy can be achieved if carefully executed. Commenter (6356) states that, with regard to Grants and Cooperative agreements, EPA should stipulate that methodologies and technologies designed to provide protected access to sensitive information on personal identities should be based on criteria adopted by the National Academies for "masking" personal identity, or similar, policies for masking personal information.
Response: The public access requirements for grants and cooperative agreements requires that data undergo quality review before they are made publicly accessible to safeguard against the release of personally identifiable or proprietary data.
14.1.2.1.3 Data Sharing Provisions
Comment: Commenters (6102, 6150, 6179, 6360, 6362, 6375, 6918) suggest EPA establish new contracting and grant requirements with data sharing provisions.
Commenters (6102, 6150, 6179) suggest that EPA can establish new contracting requirements to include data and model access requirements. 
Commenters (6102, 6150) suggest that EPA establish new contracting requirements to include data sharing provisions, create new interagency data sharing agreements, implement protocols to de-identify data with personal information, and ensure protected data sharing mechanisms where complete de-identification is not possible. Commenter (6102) states that the increased transparency will spur new research and increase the value of federal research investments. The commenter provides that data sharing will allow testing of new ideas on existing data sets. The commenter also recommends that new causal analytical techniques be used to test validity of existing data thereby increasing public confidence in results.
Response: EPA has implemented the public access requirements for EPA researchers, grantees and contractors as required by the February 22, 2013, OSTP memorandum "Increasing Access to the Results of Federally Funded Scientific Research" and the 2016 EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research." Grantees and contractors are required to make their data and models underlying any peer-reviewed journal articles resulting from EPA-funded research accessible to the public. This is requirement went into effect for any funding granted as of September 26, 2019. EPA's Environmental Dataset Gateway provides metadata records stating where those data will be available. PII and CBI are still protected under the public access requirements.
The EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research" requires the data underlying a journal article resulting from research funded by EPA be made publicly accessible. However, in the final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. The step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
14.1.2.1.4 Study Design Requirements 
Comment: Commenter (6179) suggests EPA could incorporate "transparency by design" requirements into its research grants by conditioning funding on accounting for transparency throughout studies' lifecycles. The commenter provides that, rather than employing after-the-fact de-identification protocols, it would be far easier to achieve the proposal's replicability and verification goals if transparency were incorporated into studies from the outset. The commenter suggests that EPA go beyond the proposal to develop methods that encourage transparency early in studies' development. The commenter states that EPA should borrow from the well-known privacy policy, "privacy by design," which calls for privacy to be taken into account throughout an entire developmental process. The commenter states that EPA could incorporate "transparency by design" requirements into its research grants by conditioning funding on accounting for transparency throughout studies' lifecycles. The commenter suggests that EPA could conduct further research on "transparency by design" protocols, such as anonymous coding methods, to be incorporated in these requirements. According to the commenter, such forward-looking planning can most efficiently accomplish the proposal's goal of better informing public comments in significant EPA rulemakings.
Commenter (6911) encourages EPA in the Final Rule to create "Transparency by Design" in its research requirements. Commenter states that this would be much more helpful that employing after-the-fact de-identification protocols. Commenter states that such a policy would encourage transparency early in study or a series of studies development. 
Similarly, commenter (6918) states that, when EPA carries out its own studies or other data generating activities, EPA has the ability to take care, at that time, to avoid using or generating data and models with access issues, can design data-masking procedures into the process, or can undertake other reasonable actions to enhance the public's ability to use and independently verify the underlying data and models at a later point, should the study or data be used to justify regulatory action. The commenter states that EPA's role as the generator of the data should allow for increased access as a general principle. According to the commenter, EPA should be held to a higher standard when it has the opportunity from the start to require that the study design include appropriate measures to allow for transparency without violating relevant legal and policy limits on data access. 
Response: EPA has implemented the public access requirements for EPA researchers, grantees and contractors as required by the February 22, 2013, OSTP memorandum "Increasing Access to the Results of Federally Funded Scientific Research" and the 2016 EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research." EPA researchers, grantees and contractors are required to make their data and models underlying any peer-reviewed journal articles resulting from EPA-funded research accessible to the public. This is requirement went into effect for any funding granted as of September 26, 2019. EPA's Environmental Dataset Gateway provides metadata records stating where those data will be available. As part of the grant applications, grantees are required to develop a scientific data management plan (SDMP), which includes information on where the data underlying any journal publications resulting from the project will be made publicly accessible. Contractors will also need to develop SDMPs for any projects that could result in journal articles.
The EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research" requires the data underlying a journal article resulting from research funded by EPA be made publicly accessible. However, in the final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. The step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
14.1.2.1.5 Require Level II or Level III Transparency and Openness Promotion (TOP) Guideline Requirements
Comment: Commenter (6360) recommends EPA require Level II or Level III Transparency and Openness Promotion (TOP) Guideline Requirements standards for all its research to be consistent with the best and the current standards of science, regardless of whether the researchers are EPA employees, other federal agency employees, or private researchers employed by companies, universities, or other organizations. According to the commenter, by adopting these standards in EPA's research enterprise, EPA's application of the funded science is ready to support policy judgements.
Response: EPA has implemented the public access requirements for EPA researchers, grantees and contractors as required by the February 22, 2013, OSTP memorandum "Increasing Access to the Results of Federally Funded Scientific Research" and the 2016 EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research." EPA researchers, grantees and contractors are required to make peer-reviewed journal articles and the underlying data and models resulting from EPA-funded research accessible to the public.
The EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research" requires the data underlying a journal article resulting from research funded by EPA be made publicly accessible. However, in the final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. The step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
14.1.2.1.6 Require Open-Access/Public Availability
Comment: Commenters (0671, 6362) suggest that EPA implement requirements that all EPA-funded data, models and results be open-access.
Commenter (0671) states that EPA can increase public access to EPA-funded data by requiring publications based on EPA-funded data to be open-access (or open-access after 12 months).
Commenter (6362) suggests that EPA can implement requirements that all models and results developed under EPA Cooperative Agreements and Grants be open access and not proprietary. 
Response: EPA has implemented the public access requirements for EPA researchers, grantees and contractors as required by the February 22, 2013, OSTP memorandum "Increasing Access to the Results of Federally Funded Scientific Research" and the 2016 EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research." EPA researchers, grantees and contractors will be required to make peer-reviewed journal articles and the underlying data and models resulting from EPA-funded research accessible to the public. All peer-reviewed journal articles that were funded by EPA are posted on NIH's PubMed Central within one year of publication. The data and models underlying those journal articles are to be made publicly accessible.
The EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research" requires the data underlying a journal article resulting from research funded by EPA be made publicly accessible. However, in the final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. The step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
Comment: Commenter (6375) recommends that EPA clearly stipulate in cooperative agreements and grant terms that data used by researchers receiving funding from EPA must be made publicly available to the extent possible after consultation with EPA. The commenter also suggests that EPA require as part of the cooperative agreement or grant that the data be placed into EPA's designated system to allow access to the underlying data and models.
Commenter (6449) states that, if the models and data are generated from government funds, EPA should require that they be publicly available on EPA's website or by request to EPA. Commenter states that, in some cases where compelling reasons are given and publicly available, a time delay might be justified, which should also be identified. 
Response: EPA has implemented the public access requirements for EPA researchers, grantees and contractors as required by the February 22, 2013, OSTP memorandum "Increasing Access to the Results of Federally Funded Scientific Research" and the 2016 EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research." Grantees and contractors are required to make their data and models underlying any peer-reviewed journal articles resulting from EPA-funded research accessible to the public. This is requirement went into effect for any funding granted as of September 26, 2019. EPA's Environmental Dataset Gateway provides metadata records stating where those data will be available. For intramural research, the data underlying journal publications are posted on EPA's ScienceHub and are available to the public via data.gov.
The EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research" requires the data underlying a journal article resulting from research funded by EPA be made publicly accessible. However, in the final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. The step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
14.1.2.1.7 Data Management Plan
Comment: Commenter (6362) recommends that EPA require all grant proposal applicants to include as part of any grant proposal a data management plan, similar to those required by the NIH. The commenter suggests that EPA may elect to exclude from these requirements grants/agreements of some specified annual amount, but that annual amount should be reasonable and ensure that the vast majority of models and results developed under grants/agreements is shared. 
Response: EPA has implemented the public access requirements for EPA researchers, grantees and contractors as required by the February 22, 2013, OSTP memorandum "Increasing Access to the Results of Federally Funded Scientific Research" and the 2016 EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research." Grantees and contractors will be required to make their data and models underlying any peer-reviewed journal articles resulting from EPA-funded research accessible to the public. This is requirement went into effect for grants as of September 26, 2019. As a part of grant application, applicants are required to provide a scientific data management plan (SDPM) for their proposed project. Part of the SDMP requires applicants to specify how they will make their scientific research results publicly accessible.
The EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research" requires the data underlying a journal article resulting from research funded by EPA be made publicly accessible. However, in the final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. The step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
14.1.2.1.8 Common Data Templates and Digital Platforms
Comment: Commenter (6362) states that EPA should develop common data templates and digital platforms for the most common types of research studies to be used by entities subject to Cooperative Agreements and Grants to facilitate public use and validation. 
Response: EPA has implemented the public access requirements for EPA researchers, grantees and contractors as required by the February 22, 2013, OSTP memorandum "Increasing Access to the Results of Federally Funded Scientific Research" and the 2016 EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research." Grantees and contractors are required to make their data and models underlying any peer-reviewed journal articles resulting from EPA-funded research accessible to the public. This is requirement went into effect for any funding granted as of September 26, 2019. EPA's Environmental Dataset Gateway provides metadata records stating where those data will be available.
The EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research" requires the data underlying a journal article resulting from research funded by EPA be made publicly accessible. However, in the final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. The step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
14.1.2.1.9 Anonymize or Mask Data to the Extent Necessary to Facilitate Data Sharing
Comment: Commenter (6918) states that when EPA funds research, there are opportunities to maximize both transparency and privacy. The commenter recommends that, to facilitate increased access without forcing grant or award recipients to bear unforeseen costs to achieve it, EPA consider adding a new condition to its grants and awards that requires recipients of federal funding through grants or agreements to "anonymize" or mask the data, to the extent necessary to facilitate the sharing of data with EPA and others. The commenter asserts that EPA's role as the funder allows EPA to require the grantee to consider transparency values at the beginning of the process. The commenter notes that recognition from the outset that EPA may request the underlying data will encourage the grantee to put into place, at the start of the study, appropriate data-masking or anonymizing procedures. According to the commenter, this will efficiently allow for increased data access without running afoul of legal limits on data access or policy reasons for limiting access.
Response: EPA has implemented the public access requirements for EPA researchers, grantees and contractors as required by the February 22, 2013, OSTP memorandum "Increasing Access to the Results of Federally Funded Scientific Research" and the 2016 EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research." Grantees and contractors will be required to make their data and models underlying any peer-reviewed journal articles resulting from EPA-funded research accessible to the public. This is requirement went into effect for any funding granted as of September 26, 2019. EPA Order 1000.17B "Policy for Increasing Access to Results of EPA-Funded Extramural Scientific Research" states, ". . . data should undergo quality review before they are made publicly accessible to safeguard against the release of personally identifiable or proprietary data."
The EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research" requires the data underlying a journal article resulting from research funded by EPA be made publicly accessible. However, in the final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. The step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
14.1.2.2 Do Not Support Stronger Cooperative Agreements and Grants
Comment: Commenters (0671, 1011, 4595, 5181, 6895) do not support establishment of stronger contracting requirements with data sharing provisions.
Commenter (6895) asserts that the issue of incorporating stronger data and model access into the terms and conditions of cooperative agreements and grants has been previously and adequately addressed in the OMB Final Information Quality Bulletin for Peer Review (70 FR 2664). This document states that pivotal research must include information so that agency reviewers can "ensure that scientific uncertainties are clearly identified and characterized." The commenter states that this is the task of peer reviews. The commenter notes that there is no need to make anything else available if scientific uncertainties can be quantified. (See, discussion above related to EPA's Peer Review Handbook.)
Commenter (0671) states that EPA should not incorporate stronger data and model access requirements into the terms and conditions of cooperative agreements and grants. Commenter states that such requirements are already very strong and strengthening them will add to the regulatory burden on scientists doing research of potential regulatory import without improving the reproducibility of science underlying regulation. Commenter states that, if incorporating these requirements prevents the EPA from entering into cooperative agreements or funding grants involving human subjects then the EPA will have prevented itself from using the best available science, or, even worse, prevented the highest quality science from being done in the first place, which is counter to the text and intent of the CAA and its amendments.
Commenter (1011) states that, when EPA funds research activities through grants or cooperative agreements, the agency should seek to minimize administrative burdens on researchers. 
Response: EPA has implemented the public access requirements for EPA researchers, grantees and contractors as required by the February 22, 2013, OSTP memorandum "Increasing Access to the Results of Federally Funded Scientific Research" and the 2016 EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research." Grantees and contractors will be required to make their data and models underlying any peer-reviewed journal articles resulting from EPA-funded research accessible to the public. This is requirement went into effect for any funding granted as of September 26, 2019 through EPA Order 1000.17B "Policy for Increasing Access to Results of EPA-Funded Extramural Scientific Research."
The EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research" requires the data underlying a journal article resulting from research funded by EPA be made publicly accessible. However, in the final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. The step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
Comment: Commenter (4595) states that it would be premature to introduce additional changes to the terms of research grants until the full impacts of any proposed EPA transparency policy are understood. Commenter (5181) recommends that the sufficiency of existing approaches be fully considered before making substantive changes. Commenter (5181) also recommends that adequate safeguards be considered in any proposed rules to protect personal information of relevant parties.
Response: EPA has implemented the public access requirements for EPA researchers, grantees and contractors as required by the February 22, 2013, OSTP memorandum "Increasing Access to the Results of Federally Funded Scientific Research" and the 2016 EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research." Grantees and contractors will be required to make their data and models underlying any peer-reviewed journal articles resulting from EPA-funded research accessible to the public. This is requirement went into effect for any funding granted as of September 26, 2019. EPA Order 1000.17B "Policy for Increasing Access to Results of EPA-Funded Extramural Scientific Research" states, ". . . data should undergo quality review before they are made publicly accessible to safeguard against the release of personally identifiable or proprietary data."
The EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research" requires the data underlying a journal article resulting from research funded by EPA be made publicly accessible. However, in the final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. The step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
14.1.3 Building Upon Other Federal Agencies' Policies Regarding Grantee and Cooperator Requirements
Comment: Commenter (9227) expresses concern that, while EPA claims that the proposal builds on the "experience of other federal agencies in this space," EPA simply lists other federal agencies without referring to any policies, documents or actions by those agencies, except for one particular Census Bureau database that allows federal Census data to be shared securely. The commenter states that an uncited statement that other federal agencies have "experience in this space" is far too vague to allow meaningful comment by the public on EPA's rationale for its action, much less provide any support or rationale for the proposed policy. 
Commenter (9227) states that the Census Bureau database cited in the Proposal is an example of how an agency can provide secure access to its own data, but it does nothing to explain or justify EPA's proposal to exclude third party studies with nonpublic data from consideration in rulemaking. The commenter provides that the U.S. Census Bureau operates the Federal Statistical Research Data Centers (FSRDCs), which are secure facilities providing authorized access to restricted-use microdata for statistical purposes only. The commenter states that to gain access, researchers must obtain Census Bureau Special Sworn Status -- passing a moderate risk background check and swearing to protect respondent confidentiality for life. The commenter explains that this approach meets the U.S. Census Bureau's needs by allowing access to confidential information only to researchers whose proposals meet certain criteria, who go through a vetting process, and who agree to protect the information. According to the commenter, this is a structure designed to protect data collected by the government, not third parties, and there are substantial costs to this approach, which are borne by the Census Bureau. The commenter contends that it is clearly not directly transferable to the context of the proposal. The commenter asserts that it is also unclear whether such a structure, even if it were practical (which it is not), would be sufficient to satisfy EPA's requirement to make data and models "publicly available."
Response: The Census Bureau, the NIH and the National Science Foundation are not cited in the final rule. 
Comment: Commenter (1012) states that the National Science Foundation (NSF) has long required data sharing as a condition of funding. The commenter provides that, since 2007, the NIH has required public access to publications for all of its funded research after 12 months. The commenter notes that an interagency committee set up by OSTP to develop and coordinate federal policies for long term data stewardship and dissemination of federally funded research has seen at least 17 agencies phase in such data policies. The commenter notes that all of the Agency plans exempt data that is legally protected from disclosure.
Commenter (0556) states that the Federal Government already has rules about sharing data that is developed in federally-funded research. The commenter notes that in some ways this proposed rule looks like a natural extension of the existing requirement for a Data Management Plan. The commenter states that these rules are part of what is called "Public Access," a program which began in 2013 and requires every federal science agency to have a Public Access Plan mandating that research data be shared. The commenter states that EPA's proposed rules extend the Public Access Program from research that is federally-funded to research that is federally-used. 
Response: EPA has implemented the public access requirements for EPA researchers, grantees and contractors as required by the February 22, 2013, OSTP memorandum "Increasing Access to the Results of Federally Funded Scientific Research" and the 2016 EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research." EPA requires all journal articles resulting from Agency-funded research be made available to the public within 12 months of publication. Also, data underlying those journal articles are to be made publicly available while still protecting CBI and PII according to existing requirements. 
Under 40 CFR Part 30, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent. This is for pivotal science regardless of whether or not it is funded by EPA.
This final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
The EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research" requires the data underlying a journal article resulting from research funded by EPA be made publicly accessible. However, in the final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. The step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
14.1.4 Platform for Increasing Public Access to EPA-Funded Data
14.1.4.1 Do Not Support Development of a Platform for Increasing Public Access to EPA-Funded Data
Comment: Commenters (0671, 1012, 6895) do not support development of a platform for increasing public access to EPA-funded data. 
Commenter (0671) states that if EPA is considering purchasing or developing a software platform that would magically encompass any and all data used in scientific studies potentially used in rulemaking, the EPA should not waste taxpayer money on what they believe would be "a colossal boondoggle."
Commenter (1012) states that, with a reduced number of EPA-funded research studies conducted, a platform to increase access to the data collected is likely to be irrelevant. The commenter states that, if this rule is to be implemented as written, it is likely that fewer scientists or research institutions will want to participate in EPA-funded research if they are concerned that the personal health information they collect may have to be publicly-disclosed. The commenter notes that scientists and researchers may also find it difficult to recruit participants for long-term, large-scale studies that are needed to understand air and water quality issues if the participants are concerned that their medical records may be made publicly-available. 
Commenter (6895) contends that a platform to house data from different studies is not necessary. The commenter states that corresponding authors are required to respond to public requests and should be responsible for requests for data necessary to replicate study results. According to the commenter, EPA's request for comment on possible exceptions to the proposed rule, specifically exceptions that would be made on a case-by-case basis, underscores that even EPA has doubts as to the necessity and validity of the proposed rule. 
Response: EPA has implemented the public access requirements for EPA researchers, grantees and contractors as required by the February 22, 2013, OSTP memorandum "Increasing Access to the Results of Federally Funded Scientific Research" and the 2016 EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research." Researchers are required to make their data and models underlying any peer-reviewed journal articles resulting from EPA-funded research accessible to the public. For intramural research, the data underlying journal publications are posted on EPA's ScienceHub and are available to the public via data.gov. The public access requirements under the 2013 OSTP memorandum only concern EPA-funded research and was prospective upon implementation. 
Under 40 CFR Part 30, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent. This is for pivotal science regardless of whether or not it is funded by EPA.
This final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedure and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
The EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research" requires the data underlying a journal article resulting from research funded by EPA be made publicly accessible. However, in the final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. The step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
Comment: Commenter (1012) states that OMB Circular A-110 limits the kind of data produced under a federally funded award that is required to be made public, in order to strike a balance between preserving the integrity of the research process and the need for public access to data. The commenter provides that this is accomplished by permitting such access through the procedures established under FOIA, and including appropriate exemptions such as "personnel, medical, or similar files whose release would constitute an unwarranted invasion of privacy." The commenter states that much of the data from the previously cited air pollution studies would fall under this exemption. The commenter is concerned that this proposed rule could potentially contradict these protections by requiring sensitive data that would have previously been exempted to be made "publicly available in a manner sufficient for independent validation." The commenter states that this requirement stipulated in the proposed rule is particularly concerning as "publicly available" is further defined as "information necessary for the public to understand, assess, and replicate findings."
Commenter (2427) states that there are already various means for public access to studies that EPA uses, and in some cases their underlying data, without the release of confidential information, including the FOIA, which provides an avenue to request raw data, including a process ensuring that sensitive data is protected. 
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
14.1.4.2 Support Development of a Platform for Increasing Public Access to EPA-Funded Data
Comment: Commenters (1941, 6126) suggest that EPA should work to set up data-housing capabilities and protocols at a Research Data Center (RDC). Commenter (1941) states that an RDC or another suitable data-storage/analysis site could facilitate the receipt and centralized management of pivotal study data in a manner that can streamline third-party access to data, while also providing safeguards and Agency oversight to protect confidentiality concerns.
Commenter (6126) suggests EPA establish a strong culture and practice of protecting such data with a sufficiently secure data infrastructure. The commenter asserts that EPA's data infrastructure and use of authorities to protect such data appear to be limited. The commenter recommends that EPA explore the availability of the FSRDCs as an option for making confidential data available for approved research projects and statistical activities. The commenter states that the FSRDCs have existing protocols about the de-identification of data and disclosure review processes on which EPA could reasonably build. The commenter provides that FSRDCs may also be useful for EPA to make sensitive, de-identified data available in a secure manner that reduces the risk of re-identification. 
Response: EPA has implemented the public access requirements for EPA researchers, grantees and contractors as required by the February 22, 2013, OSTP memorandum "Increasing Access to the Results of Federally Funded Scientific Research" and the 2016 EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research." Researchers are required to make their data and models underlying any peer-reviewed journal articles resulting from EPA-funded research accessible to the public. For intramural research, the data underlying journal publications are posted on EPA's ScienceHub and are available to the public via data.gov. Also, data underlying those journal articles are to be made publicly available while still protecting CBI and PII according to existing requirements. The public access requirements under the 2013 OSTP memorandum only concern EPA-funded research and was prospective upon implementation.
The EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research" requires the data underlying a journal article resulting from research funded by EPA be made publicly accessible. However, in the final rule, the EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. The step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
14.1.4.3 Consider Various Existing Data Management Platforms/Requirements
Comment: Commenters (0556, 2171, 2426, 2479, 6375, 6449, 6908) suggest EPA consider the use of various existing data management platforms/requirements. Commenter (2426) notes that other organizations already have infrastructure and policies in place to handle sensitive information, and it makes sense to use these existing capabilities, rather than attempting to re-create them at the EPA. Commenter (6375) states that EPA should study existing systems and adopt, adapt or set up new online data repositories similar to those that already exist that allow public access to data on which agency actions rely that do not contain personal or confidential information. 
Commenter (6375) notes that various government agencies, such as the Centers for Disease Control and Prevention (CDC), the Centers for Medicare & Medicaid Services (CMS) and the U.S. Census Bureau maintain online databases and data repositories to allow public data access or restricted access to identifiable data. The commenter provides that CDC and CMS both provide restricted and secured access to health data that contain potentially identifiable and privacy information, such as name, residential address, vital status, birth date, and disease diagnosis. The commenter notes that EPA could also implement additional background screening processes and security measures for access to data that may contain identifiable or proprietary information, similar to those employed by the CDC and CMS for access to research identifiable data. 
Commenter (6449) encourages EPA to cooperate with other federal agencies to generate common data access, management and storage standards. Commenter also encourages EPA to participate with the PubChem and the National Toxicology Program databases. Commenter states that the same types of legally binding documents that EPA needed to sign to obtain such data, e.g., non-disclosure agreements, would be expected to be sufficient for other parties. The commenter states that EPA could consider requiring data acquisition and analysis by parties independent of the stakeholder. The commenter suggests, for example, if CBI were requested by an industrial competitor, EPA might require that a non-affiliated consulting firm or academic be the entity that signs the NDA and receives the data for analysis. 
Commenter (2479) states that policies vary based on the type of research and the confidential information researchers gather, but the NIH maintains several online databases, like the National Cancer Institute Data Catalog, that qualified researchers can access for free. Commenter (6144) asserts that the NIH system is not publicly accessible and requires significant resources to manage and share with researchers. 
Commenter (2171) states that Minnesota agencies are at the forefront of making environmental and health surveillance data available, providing technical assistance for using data, and engaging partners across communities and research institutions around effective dissemination and data utilization. The commenter notes that their agencies host multiple platforms for accessing high-quality health surveillance and environmental monitoring data, while protecting privacy and providing essential risk communication and prevention strategies. 
Commenter (2479) states that many scientific journals, like Science, already require study authors to publish their data in online databases, but at Science, for example, the policy allows exceptions for studies that contain sensitive details like medical information. 
Commenter (6908) states that Bayer is uniquely positioned to provide information on the efforts to make data transparent. The commenter notes that Bayer has submitted nearly 50,000 volumes of research to the agency in support of regulatory actions related to their products and nearly the full set is currently available to almost everyone upon demand. The commenter provides that beginning in 2016 Bayer made the decision to make the safety data underlying its products available to interested parties and began the process of developing a platform to accomplish this. The commenter states that in late 2017 Bayer's transparency platform was launched and may accessed at https://www.cropscience-transparency.bayer.com/.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenter (9224) suggests EPA should encourage authors to submit their data to public repositories as well as considering giving preference, when possible, to studies where the data is publicly available in this manner.
Response: This rule exclusively pertains to the internal practices of the EPA and does not regulate the conduct or determine the rights or obligations of any entity outside the federal government.
14.1.4.4 Issues Related to the Development of a Data Sharing Platform
Comment: Commenters (6144, 6191, 6909, 6915) express concern that the proposed rule does not address numerous issues related to the development of a platform. The commenters assert that ensuring all this extra information is received, stored, and made publicly accessible would be no small undertaking. Commenter (6144) inquires where the raw data collected will reside, how will the resources to manage the data be obtained, and how will the agency deal with legally and ethically protecting data that is confidential or sensitive. 
Commenter (6191) states that guidelines would need to be established as to how long the data would be held and a repository would need to provide mechanisms for file format migration that ensures that data remain un-corrupted and readable. According to the commenter, aggregating sensitive health data onto one platform would unnecessarily increase risks to human subjects. The commenter asserts that data breaches at the Office of Personnel Management (OPM) (https://www.opm.gov/cybersecurity/cybersecurity-incidents/ ) and other commercial organizations (e.g., Equifax, Yahoo) demonstrate that no platform can ever be completely secure. 
Commenter (6909) states that, even where privacy issues do not exist, insufficient infrastructure can present a substantial barrier to uniform public sharing of data. The commenter asserts that this is a reason why, in seeking to increase data sharing and improve transparency in science, EPA should take a measured and nuanced approach, consulting closely with the relevant stakeholders rather than swiftly imposing a bright line rule. According to the commenter, existing infrastructure is often still insufficient for hosting and sharing large data sets. The commenter notes that, for example, mechanisms for aggregating information about associated research projects together (rather than having them distributed across numerous repositories), and for properly and usefully identifying and tracking related resources  -  all of which are central to improving openness and reproducibility of studies  -  are often lacking. The commenter provides that some groups are developing such services and integrations, but these capabilities need significant improvement. The commenter states that, in addition, sharing of data increasingly means processing large amounts of data stored not in one location but rather on a distributed database. The commenter states that existing technologies are sometimes unable to process such large-distributed datasets efficiently. 
Response: The final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedure and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
14.1.5 Methodologies and Technologies Designed to Provide Protected Access to Data
14.1.5.1 Consultation with and Consideration of Methodologies and Technologies Designed to Provide Protected Access to Data Already Developed and Used by Agencies and Organizations 
Comment: Commenter (6915) expresses concern that, while the proposal says EPA "should collaborate" with other agencies ("to identify strategies to protect confidential and private information in any circumstance in which it is making information publicly available"), there is no timeframe for this process, no explanation of what will happen until such strategies are formed, and no indication of what these strategies will be.
Response: The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science. The final rule does not mention collaborating with other agencies to identify strategies to protect confidential and private information.
Comment: Commenters (0039, 4537) recommend that EPA, as it determines how best to implement the methodologies and technologies of a "reproducibility architecture"' to facilitate the adoption of a best available science/best available reproducible science (BARS) standard, consult with representatives or the Center for Open Science, the Meta-Research Innovation Center at Stanford (METRICS), and the Laura and John Arnold Foundation Research Integrity Initiative. The commenters also recommend that the EPA consult with the American Statistical Association about how to institute standard procedures that will ensure that all scientific research used or funded by the EPA is conducted according to the highest standards of statistical practice. 
Commenter (5185) encourages work to protect and de-identify sensitive information to allow for its continued use in regulatory decision-making. The commenter suggests that coordination with other federal agencies could assist; for example, the Department of Health and Human Services (DHSS) and National Institute of Standards and Technology (NIST) have developed tools and guidance to de-identifying PII. According to the commenter, drawing on the expertise and experience of other agencies across the federal government through active and ongoing coordination will improve EPA's ability to make this effort a success. Commenter suggests that, such an effort would be undertaken in concert with the OMB Office of Information and Regulatory Affairs.
Commenters (3971, 6126, 6375) suggest EPA should seek inter-agency advice and technical support on this matter. Commenter (6126) suggests that, regarding EPA's inquiries about nondisclosure avoidance protocols, EPA should consult with the Federal Committee on Statistical Methodology and the federal statistical agencies that have developed much of the research and practice regarding de-identification approaches for confidential data. Commenter (6375) states that multiple government agencies provide restricted and secured access to data that contain privacy or confidential information. 
Commenter (6921) states that for sensitive data sets that are otherwise protected due to personal information or CBI, EPA should work with other federal agencies to develop: (1) de-identification protocols that will allow an increasing number of data sets to be fully released to the public; and (2) protected data sharing platforms that will allow the release of more sensitive data sets to researchers subject to confidentiality agreements.
Commenter (6362) states that several agencies and organizations, in addition to NIH, have successfully addressed the issue of data access while maintaining confidentiality that should be considered by EPA. For example:
 The existing rule requiring federally-funded research to be made available to other researchers. This standard could be adopted and applied to third-party funded researchers.
 Health care claims and related data are now being made available to researchers in de-identified form by some health insurance companies, such as Optum, which offers a "proprietary research database of health care and administrative data that links patient, physician, and treatment attributes from millions of geographically diverse individuals in the U.S." Optum appears to have developed methods and procedures to appropriately address confidentiality concerns.
 Medicare claims data are already available to researchers in de-identified form. Algorithms and methods developed by the Center for Medical Services should be examined by the EPA.
 Several professional societies have guidance on the protection of health data and de-identification, such as the Institute of Electrical and Electronic Engineers and the International Association of Privacy Professionals.
Commenter (9224) encourages EPA to consult with entities, including the journals that EPA references, that have implemented similar efforts in order to better inform the methods and protocols that should be in place for a rule of this nature.
Response: The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science. The final rule does not mention collaborating with other agencies to identify strategies to protect confidential and private information.
14.1.5.2 Methodology and Technology Limits to Protection of Data
14.1.5.2.1 De-identification Methods and Technologies are Limited and Can Potentially Lead to Re-identification
Comment: Commenters (0569, 2746, 5173, 6046, 6110, 6126, 6132, 6135, 6357, 6446, 6868, 6924, 6937, 9227) caution that there are limits as to how much an individual's identity can be protected based on commonly used de-identification methods and technologies. Therefore, they do not believe the proposed rule can be effectively implemented.
Commenters (0569, 6446) assert that researchers could redact and anonymize personal health data but there are limits as to how much an individual's identity can be protected, especially in small communities where basic identifiers may be enough to recognize an individual. 
Commenters (5173, 6110, 6446, 6924) assert that, while the proposal references simple data masking, coding, and de-identification techniques, these methods will not sufficiently protect patient identity in studies such as those in which personal health information is integral to the study. Commenter (6868) suggests that sharing data on animal research or controlled exposure of healthy people can be relatively straight-forward but sharing medical data about real people with diseases at work and at home is much more challenging. The commenter states that full de-identification of the study participants in studies that use dates and addresses to estimate pollution exposure would be impossible.
Commenters (5173, 6126, 6132, 6153, 6191, 6924, 9227) cite several studies to demonstrate that de-identified and anonymous data can be re-identified.
Commenter (5173) asserts that the need to protect sensitive information would make compliance with the proposed rule impossible but discarding such studies from the regulatory decision-making process would exclude relevant and important science, thereby compromising public health. Commenters (6126, 9227) assert that, as the Commission on Evidence-Based Policymaking described in its final report, simply removing direct identifiers is often insufficient to minimize the risk of re-identification. Commenter (6132) states that, in a time of "big data", it is all too easy to crack any de-identification process, especially when lots of publicly available spatial environmental data are matched to people in the study. Commenter (6153) states that, although public release of data collected on individuals has relied on anonymization to protect privacy, researchers have demonstrated the potential of "reidentification" techniques, which combine seemingly innocuous data from multiple publicly available sources, to identify individuals.,  Commenter (6191) references two studies and concludes that de-identified and anonymous data can be re-identified., Commenter (6924) states that, according to an NAS Workshop report re-identification of respondents may be increasingly possible because of high-speed computers, external data files containing names and addresses or other direct identifiers as well as information about a variety of individual characteristics, and sophisticated software for matching survey and other files. Commenter (6924) states that re-identification is not hypothetical; study participants, including children, have been identified from data sets which were thought to be sufficiently masked. 
Commenter (9227) cites additional studies that show the reality and scope of the re-identification threat.,.. The commenter states that the Commission on Evidence-Based Policymaking, which EPA also cites in the proposal, also stresses the dangers of re-identification of data that has been stripped of direct identifiers: "No existing statistical disclosure limitation method. . . is able to completely eliminate the risk of re-identification," despite increasingly complex techniques that have been developed since the 1970s. The commenter provides that the Commission also notes the threat posed by the "cumulative amount of information available about individuals and businesses that could be used for reidentification," with the threat increasing as available information grows and technology to allow re-identification improves.
Commenter (9227) states that while EPA suggests de-identification and redaction of sensitive information can be used to protect privacy when study data is made public, EPA ignores the real concerns, based in empirical evidence, about reidentification of individuals through cross linking with existing public datasets and the ensuing breach of privacy. Commenter states that experts have observed that even the disclosure of redacted or "de-identified" data sets has become more fraught as public health studies have become more rigorous, because these studies are relying upon greater quantities of ever more granular personal information. 
Commenter (2746) states that the proposed rule does not address or define the required data set granularity that must be made publicly available; that is, would data presented in aggregate be sufficient for independent validation or would only raw data be sufficient? The commenter states that depending upon the data set, inference and other methods can often be used to deanonymize data, which would result in compromised privacy for study subjects.
Commenter (9227) asserts that, while EPA states that "[o]ther federal agencies have developed tools and methods to de-identify private information," the Agency then cites to only one source, which does not address the concerns raised here. The commenter states that this guidance provides two methods for de-identifying data: (1) expert determination method, where an expert determines that, after application of statistical and scientific principals and methods, the risk is very small that the information alone or with other available information could be used to identify the subject; and (2) the safe harbor method, requiring that a number of identifiers are removed. The commenter notes that the first method requires case-by-case work, and EPA has provided no information regarding how EPA or others could potentially implement it or how much it might cost. The commenter states that there is no indication of how broadly this technique might be applicable to adequately de-identify data. i.e., EPA must provide its views on whether this technique is likely to be applicable to the majority of studies relevant to EPA with non-public data, some studies, or only a handful. The commenter provides that the second method requires removal of much information that may be necessary to be able to reanalyze or reproduce the research results, so it is unclear whether it would satisfy EPA's requirements in the proposal. According to the commenter, even the safe harbor method has been shown to provide potentially insufficient privacy protections due to the mosaic effect. 
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying data are publicly available or available through restricted access. Restricted or tiered access in this rule means that the underlying data are be available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The rulemaking does not require the EPA to collect, store, or publicly disseminate data underlying pivotal science.
Comment: Commenter (9227) takes issue with the suggestion in the proposal that the National Academies are of the opinion that simple data masking, coding, and de-identification techniques have been developed over the last half century so that data can easily be modified to address privacy concerns. The commenter notes that the National Academies in fact recognizes that complex, evolving, and yet undeveloped techniques are needed to resolve these concerns, that there are limitations of producing restricted data that sufficiently limit identifiability to allow it to be made publicly available in useful form, and that more work is needed to allow high quality public-use files that still assure inferential validity of the data while safeguarding confidentiality.,,, The commenter adds that the National Academies note, "data that are most useful to legitimate researchers typically have characteristics that pose substantial risk of disclosure" and there is increased vulnerability in "[d]ata with geographic detail, such as census block data" and longitudinal data obtained in panel surveys, which is often salient in environmental research.
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenter (9227) states that ongoing developments in data analytics make data de-identification more difficult to conduct and less likely to adequately protect privacy and confidentiality. The commenter expresses concern that EPA fails to mention, let alone address, the increasing risk of re-identification through data analysis using multiple data sets. According to the commenter, the so-called "mosaic effect" makes even very limited, redacted releases of data to the public a threat to the privacy of study subjects. The commenter provides that OMB has recognized the threat to privacy from the mosaic effect, which it describes as "when the information in an individual dataset, in isolation, may not pose a risk of identifying an individual (or threatening some other important interest such as security), but when combined with other available information, could pose such risk." The commenter notes that OMB specifically highlighted the complicated nature of this threat and the need for agencies to address it carefully, particularly as they may not possess the needed expertise. 
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenters (6191, 6357, 6446, 9227) express concern that EPA fails to recognize the risks to privacy that remain even if these methods are employed. Commenter (6191) states that government systems and public sector organizations with highly personal data have been breached and there is no reason to believe any system could be completely secure. Commenter (6357) states that making certain types of data public carries substantial privacy risks and may not always be necessary or sufficient to achieve transparency. Commenter (6357) states that, with advancing computational methods and increasing types of data collected on individuals in multiple domains, re-identification of de-identified human subjects becomes more likely without evolving protections. Commenter (6446) states that the fact that the public would be able to de-anonymize much epidemiological data, after de-identification processes have been applied, would almost certainly have a chilling effect on voluntary public participation in important research. 
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
14.1.5.2.2 De-Identification Methods and Technologies Could Potentially Make Datasets Unusable
Comment: Commenters (6135, 6924, 9227) suggest that another limitation of de-identification efforts is that they may make data sets unusable for reanalysis purposes. 
Commenter (6135) asserts that it is disingenuous that the proposed rule suggests that health datasets can be sufficiently de-identified in the public domain in a way that would simultaneously allow for reanalysis while also preventing the illegal disclosure of protected information. Since there is no way to provide de-identified data in the public domain that would allow for secondary reanalysis of both the exposure assessment and health analysis in studies assessing the health impacts of air pollution, the commenter recommends EPA should instead continue to focus on assessing results across multiple studies to evaluate the veracity and general applicability of scientific findings.
Commenter (6924) states that often, identifying information is a metric key to a study's data analysis and conclusions, such as when home addresses are used as a basis to estimate exposure measures for pesticides or other pollutants. The commenter states that, in this case, no meaningful analysis could be conducted with de-identified data where home addresses have been removed to ensure participant privacy. According to the commenter, for these types of studies, data de-identification and masking do not provide a viable solution that would both allow re-analysis and protect participant privacy.
Commenter (9227) states that work by other experts in this area suggests that de-identification can be carried out and help protect privacy, but it may produce datasets that have lost vital information needed for specific analyses. The commenter states that the Health Insurance Portability and Accountability Act (HIPAA) guidelines document states: "Of course, de-identification leads to information loss which may limit the usefulness of the resulting health information." According to the commenter, such results limit the utility of de-identified data sets and would not meet the requirements of the proposed rule which state that "EPA will ensure that the data and models underlying the science is publicly available in a manner sufficient for validation and analysis."
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Due to limits in the ability to protect data, commenters (6110, 6446, 9227) express concern that EPA may not be able to rely on the best science. Commenter (6110) expresses concern that the proposed rule has oversimplified the issue at hand, which will lead to difficulties in implementing this rule while maintaining EPA's commitment to using the best available science in its regulatory actions. Commenter (6446) states that the inability to de-anonymize much epidemiological data would also prevent EPA from relying on the "best available science" in carrying out its work.
Commenter (9227) states that some researchers might respond to the rule by choosing to work only on public administrative datasets, but this would harm rather than strengthen science quality by curtailing scientific inquiry. The commenter states that the effects of EPA's proposed approach would cause some researchers to choose not to pursue research with human subjects, stifling scientific discovery, while others would forgo compliance with EPA's regulatory requirements and have their research ignored by EPA. The commenter contends that, as a result, EPA's rule would both discourage the development of best available science as well as EPA's use of it.
Response: The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
14.1.5.2.3 Concerns About Access to Confidential or Private Information Being Addressed by Common Solutions
Comment: Commenters (6110, 6114, 6046, 6448, 6937, 9227) do not agree with the statement in the proposal (which includes footnote 16) that "EPA believes that concerns about access to confidential or private information can ... be addressed through the application of solutions commonly in use across some parts of the Federal Government." 
Commenter (9227) notes that EPA claims that there are examples of common solutions from the Department of Health and Human Services (DHHS), the NIST, the Department of Education, and the Census Bureau. Commenter states that, apart from a reference to DHHS guidance on data de-identification, EPA does not actually identify or cite to any specific examples from these agencies in the proposed rule itself, making it impossible to discern what examples EPA believes exist or to meaningfully comment upon the degree to which such examples, if they exist, might suggest that these issues are manageable. The commenter states that the additional hyperlinks added to the docket on May 25, 2018, weeks into the comment period, also link to examples that provide no further assurance that this proposal can be implemented without implicating privacy concerns, and the vaguely referenced other agencies' "solutions" are unlikely to be of much help. 
Commenters (6110, 9227) assert that the proposed rule addresses privacy issues only in a vague sense and does not state what those "solutions" would be. Commenters (6114, 6448) assert that this statement is not supported by any information in the proposal. Commenter (6046) states that, with respect to footnote 16 in the proposal, confidential data are not easily accessible and to make them so would require resources and planning not provided for or outlined in the proposed rule. Commenter (6937) states that the examples cited in footnote16 are difficult to transfer because of differences in the nature of the data involved, the sample sizes (and thus the degree to which anonymity can be guaranteed through aggregation), and the nature of incentives and preexisting trust between the parties that access the data. Commenter (6937) asserts that, while the proposal claims that science using protected/private health data will still influence public policy (because viable protections are available), this is not true. 
Commenter (9227) states that the Proposal ignores or assumes away real and significant issues, suggesting that existing mechanisms and techniques can be used to protect privacy and confidentiality while making underlying research data publicly available. According to the commenter, the evidence (including several of the sources that EPA cites) indicates that the potential mechanisms alluded to by EPA would only have the potential to address some of the barriers, have serious limitations even for those, and are actually becoming less effective as it becomes easier to combine and manipulate public data sets. The commenter provides that EPA's own SAB voiced concerns that EPA was discounting the challenges to making even limited releases of data, saying:
      The proposed rule oversimplifies the argument that "concerns about access to confidential or private information can, in many cases, be addressed through the application of solutions commonly in use across some parts of the Federal government." For studies already completed or underway, the participation of human subjects is undertaken according to terms approved by the cognizant IRB [Institutional Review Board]. These terms can vary from study to study. In some cases, the data cannot be released simply by redacting portions of it. For example, data may have been collected with an assurance to the participating individuals that their data would be kept confidential.
Commenter (9227) states that the "solutions" EPA might have in mind do not address the issues raised by the Proposal because no other agency has tried to implement a requirement such as the one EPA proposes. The commenter states that other agencies provide guidance and techniques to protect privacy during data collection and disclosure to allow more use of data collected by the government, not to mandate that data collected by academic or industry researchers be publicly available for purposes of replicating analyses. For example, the commenter provides that the Department of Education has shared techniques for institutions to provide data on students and schools to meet reporting requirements without compromising privacy. The commenter states that they recognize that each technique "requires some loss of information." According to the commenter, while de-identified information may still be useful, e.g., to show overall school progress, in the context of the Education Department, it is not clear these techniques are transferable to other contexts.
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenter (9227) states that EPA links to a document of the Privacy Technical Assistance Center, Data De-identification: An Overview of Basic Terms, which provides a high-level overview of key terms and practices to help educational agencies and institutions comply with the Family Educational Rights and Privacy Act (FERPA). The commenter provides that this document addresses concerns with data disclosure that occurs "when schools, districts, or states publish reports on student achievement or share students' data with external researchers" and not to make underlying data publicly available for independent validation, thus, it is unclear that methods used to de-identify but preserve data for those purposes would be adequate in this context.
Response: The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (9227) states that EPA links to a NIST document entitled De-Identification of Personal Information by Simson L. Garfinkel (NISTIR 8053), which discusses de-identification, but not in the context of making research data publicly available for independently validating scientific studies. The commenter states that the document instead notes that "that there is a trade-off between the amount of de-identification and the utility of the resulting data" and that "[i]t is thus the role of the data controller, standards bodies, regulators, lawmakers and courts to determine the appropriate level of security, and thereby the acceptable trade-off between de-identification and utility." The commenter adds that it further notes that "de-identification approaches based on suppressing or generalizing specific fields in a database cannot provide absolute privacy guarantees, because there is always a chance that the remaining data can be re-identified using an auxiliary dataset."
Response: The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (9227) states that EPA's reference to the U.S. Census Bureau is similarly unhelpful. The commenter notes that EPA provides a link to a website titled Data Ingest and Linkage that details the U.S. Census Bureau's approach to linking data across many records they hold. The commenter provides that the Website links to a working paper that describes the method by which the Census assigns a unique person identifier that enables it to link records together to create the final file. According to the commenter, it is totally unclear how this process on linking together records is a solution that EPA could implement to protect privacy of individuals when disclosing data as it concerns how to identify data with specific people -- not protecting privacy.
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenter (9227) states that, while other agencies are clearly grappling with the issue of how to make government-collected data available, they have also highlighted the many challenges in protecting privacy and confidentiality while doing so -- such as the ability for de-identified data to be re-identified --  and these agencies accept that there is more work to be done before these concerns are fully addressed. The commenter provides that the letter filed in this docket by the Presidents of the National Academies of Science, Engineering and Medicine underscores these difficulties, specifically noting the National Academies' previous work finding that "statistical analyses of data sets that generate highly precise results -- such as geographic specificity or other characteristics that identify respondents -- may result in privacy breaches . . . The commenter states that this presents a new challenge that federal statistical agencies are just beginning to address." The commenter contends that EPA does not even acknowledge, much less try to address, these gaps in agencies' abilities to protect sensitive data.
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenter (9227) states that EPA cursorily mentions a range of options for facilitating secure access to confidential data, including: "[r]equiring applications for access; restricting access to data for the purposes of replication, validation, and sensitivity evaluation; establishing physical controls on data storage; online training for researchers; and nondisclosure agreements." The commenter expresses concern that EPA does not indicate whether it would deem providing access with these types of controls in place sufficient to meet EPA's proposed requirement "publicly available in a manner sufficient for independent validation." 
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted or tiered access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. 
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science. 
Comment: Commenter (6875) states that, while it is reassuring that EPA believes that concerns about access to confidential information or private information issues can be addressed by common solutions, it is important that the agency outline a clear protocol that is transparent and clear. The commenter recommends that EPA solicit public comment on this protocol before implementing it agency-wide.
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science. Further details regarding implementation of the rule will be developed before the publication in the Federal Register when the final rule is effective.
14.1.5.2.4 Legal, Ethical, and Practical Barriers to Making Data Publicly Available
Comment: Commenter (6915) states that the generally accepted professional practice for the collection of human data requires IRB review and informed consent from the individuals from whom the data are collected. Although the proposal states that "concerns about access to confidential or private information can, in many cases, be addressed through the application of solutions commonly in use across some parts of the Federal government," 83 Fed. Reg. at 18,770, the commenter asserts that this will not be possible for many studies because IRBs dictate the specific terms of this informed consent, including that the conditions of collection and analysis of human data be specified before initiation of the study. The commenter notes that these a priori conditions include the types of analyses that will be performed, how the data will be used, and whether and how the data can be shared. In general, the commenter asserts that a priori conditions preclude sharing raw data with entities not included in the original IRB approval and performing analyses not specified in the original IRB approval, even if portions of the data are redacted. Furthermore, the commenter adds that clinical data collected from physicians, hospitals, clinics, etc., may also be subject to restrictions under HIPAA, over and above IRB restrictions.
Commenter (9227) states that there are legal and ethical requirements that restrict making public the data underlying studies, including rules to shield private personal information, requirements to maintain CBI, situations where obtaining the necessary permissions to release data are logistically difficult or impossible, and situations in which researchers have made significant investments in developing datasets that they intend to continue to work with for future studies. The commenter states that not all of these barriers can be overcome, nor can they be overcome in every case. The commenter contends that, while there are ways potentially to address some of them, they can be extremely costly and burdensome, and/or may harm the prospects for further research. 
Commenter (9227) states that, with respect to human subjects, there are strong legal and ethical privacy and confidentiality protections, which researchers are bound to respect. The commenter notes that, in some cases, researchers would be subject to civil or criminal penalties for violations. According to the commenter, in some cases, de-identification techniques are simply not applicable or still leave significant risk of breach of privacy. The commenter states that, for studies that had received IRB approval prior to finalization of this proposed rule, there may be no practical opportunity to make the data publicly available. The commenter contends that even for new studies going forward, it may be extremely difficult, require additional (often unavailable) funding for elaborate protective measures, or simply impossible to obtain IRB approval for protocols that would allow the data to be made publicly available. 
Commenter (9227) states that, with respect to studies that have already been completed, there are additional formidable barriers to public release of underlying data and models. The commenter notes that, with older studies, simply finding the data sets and determining ownership may be expensive or impossible. According to the commenter, for older studies with human subjects, obtaining consent to release of data may be practically impossible, and the data may have been collected in ways that would make protecting privacy with release difficult or impossible.
Commenter (9227) states that, for some studies, administrative issues related to the data could be the most difficult barrier to overcome in providing for public release. The commenter states that larger and more costly studies are often performed by groups of researchers within a university, across multiple institutions, or across multiple individual companies. The commenter provides that over time, the data itself may become lost or misplaced, or it may become unclear who actually owns and controls access to the data. The commenter adds that academics move among institutions, companies merge and spin off, and the initial agreements were not always clear in the first instance. According to the commenter, obtaining consent from multiple institutional players takes extensive time and resources, at minimum, and simply may no longer be possible in some instances. The commenter contends that these problems are exacerbated with respect to human subject studies. Additionally, the commenter states that researchers are legally and ethically obliged either to protect the privacy of the individual study subjects or attain each subject's consent to share data. The commenter provides that this can be impractical for older studies and virtually impossible for larger studies, and extremely burdensome. For example, the commenter states that the ACS CPSII extended analysis by Krewski in 2009, which is central to PM2.5 national ambient air quality standards (NAAQS), was initiated in 1979 and encompassed data from 500,000 study participants who lived in 116 metropolitan areas. The commenter asserts that, for these types of situations, tracking down participants (or where the participants have passed away, their family members) to get consent is simply not realistically possible.
Commenter (9227) states that studies conducted on behalf of industry or with industry cooperation may contain CBI, the release of which could jeopardize a company's competitiveness. The commenter provides that, in some instances, researchers cannot make their data sets public without losing much of the value to the researcher of these laboriously and meticulously collected sets of information. 
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
14.1.5.3 Support the Development of Methods and Technologies
Comment: Commenters (5165, 6141, 6150) support the development of methods and tools for making underlying confidential and private information publicly available while ensuring protection of that information. Commenter (6141) recommends that tools and methods include the following: requiring applications for access; restricting access to data for purposes of replication, validation, and sensitivity evaluation; establishing physical control on data storage; requiring the use of nondisclosure agreements.
Commenter (6877) states that a "one-size-fits-all" approach for access to regulatory data does not work and that a framework for providing varied levels of access to regulatory research data is essential. The commenter states that, as has been learned from confidential data centers providing access to data in a secure environment for some statistical agencies, such access is administratively complex, takes considerable time to implement, and is costly to both agencies and the scientists who use it. The commenter supports the use of such access to data across the federal government, but recommends it be implemented and developed independent of this proposed rule.
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenter (6918) suggests that, for data that EPA neither generates nor funds, additional measures may be necessary to reflect EPA's values and achieve good governance. The commenter supports efforts, by EPA or others, to develop data-masking measures that lawfully and responsibly increase availability of data and models. The commenter provides that, as EPA acknowledges in the Proposal, there are currently available tools that EPA underutilizes in its rulemakings. The commenter states that there may be new or modified methods that may be appropriate and should be further investigated. The commenter suggests that EPA recognize that, in certain limited cases, the most appropriate approach may be to make the data and related information available to third-party, neutral investigators. 
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
14.1.5.4 Do Not Support the Development of Methods and Technologies
Comment: Commenter (0671) does not trust any methodology and asks if EPA would retain legal liability for a data breach.
Commenter (1006) states that there is no good way to provide transparent data while still protecting the identity of those in the study. The commenter states that there are plenty of studies that show a person's identity can by revealed with a set of a few data points. The commenter asserts that it seems counter intuitive that by making the data public, it helps make better science. 
Commenter (1012) states that the methodology and technology used to provide access to the underlying data would be irrelevant if the overarching policy precludes the use of certain data sets that would provide the most comprehensive information.
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
14.1.5.5 Recommended Methodologies and Technologies to Provide Protected Access to Data
Commenters request that EPA make efforts to make underlying data available and make several recommendations for methodologies and/or technologies to provide protected access to data.
14.1.5.5.1 General
Comment: Commenter (6155) agrees with EPA's goal of making data, as well as the underlying assumptions and options used in the modeling of data, both transparent and available. Commenter (6935) states that in any final rule, EPA also should balance strict adherence to new transparency requirements against the need for regulatory certainty and efficiency.
Response: The purpose of this rule is to codify internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information.
Comment: Commenter (6375) recommends that EPA clearly define data processing/scrambling/analytical measures that are deemed sufficient for protection of privacy and confidentiality of study participants, protection of proprietary data and CBI, and other compelling interests. The commenter suggests that EPA, for example, could consider whether certain analytical processes can be compartmentalized to allow further de-identification of the participants and better protection of the privacy information.
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenter (5185) states that additional guidance explaining how agency officials should advance transparency goals while ensuring protection of sensitive information under varied circumstances would be beneficial.
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenter (4534) states that data released to the public should be published in a way that is easy for members of industry to obtain and understand. The commenter suggests EPA establish standards for how to publish data so users are able to easily understand how EPA arrived at their conclusions.
Response: This rule exclusively pertains to the internal practices of the EPA and does not regulate the conduct or determine the rights or obligations of any entity outside the federal government. This rule does not require any researcher or other outside entity to provide data to the EPA. Further, the final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenter (6155) supports a process for protecting data if release of such data may impair privacy, confidentiality, or national security. The commenter states that this can be accomplished in part by doing the following:
      (a) clearly detailing an objective and consistent process that would determine when release of data underlying pivotal studies would be deemed inappropriate in order to protect privacy, confidentiality, and national security, and
      (b) specifying conditions under which EPA could rely on studies for which the underlying data cannot appropriately be released, or released in ways that will protect privacy, confidentiality, or national security.
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. 
Comment: Commenter (6362) states that EPA might consider an approach followed under the Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA) where Data Evaluation Records of studies are made publicly available, but not full studies. 
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (0560) recommends that EPA's policy and regulations address each of the issues detailed by Noble Laureate Richard Feynman in "Cargo Cult Science" (1974). According to the commenter, the idea is to try to give all of the information to help others to judge the value of your contribution; not just the information that leads to judgment in one particular direction or another. 
Response: The rule does not categorically exclude studies, even studies where the underlying dose-response data are not available for independent validation, and therefore any impact on researchers who are developing science and deciding whether to voluntarily make the underlying dose-response data available is incidental. Instead, it governs internal agency procedures for determining the consideration to afford various studies according to factors that include data availability.
14.1.5.5.2 Privacy Data
Protection of Health Data
Comment: Commenter (6362) recommends that EPA ensure that it implements its final rule in a manner that enables it to use confidential health records that may exist with certain kinds of studies, such as long-term air pollution and workplace exposure studies that involve confidential health records. 
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenter (6362) states that, when protecting data while also promoting data access, NIH guidelines should be consulted. The commenter believes many of these guidelines could be applied in EPA's implementation of this proposed policy under each of the statutory programs EPA administers to ensure the guidelines adopted suit the specific needs of each statute. 
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (2479) recommends that, when researchers disclose their datasets, they should not black out personal health information; instead, they should group the results and the databases should then be accessed through a strict ID and password system.
Response: This rule does not regulate researchers or anyone outside of the EPA.
The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (6942) requests that EPA provide clarification and guidance for implementing the proposed regulation when human data are utilized for dose-response modeling and development of a chemical-specific cancer slope factor, reference dose, or reference concentration (e.g., risk assessments by the Integrated Risk Information System (IRIS)). 
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Institutional Review Board (IRB) Approval of Disclosure of De-identified Data 
Comment: Commenter (6942) recommends that, for human data, information available to researchers be limited to individual-level, de-identified data, to ensure that the privacy and confidentiality of the study participants is maintained. Commenters (1006, 2171) state that detailed data are similarly available for research uses, under the approval and guidance of IRBs that help regulate and evaluate studies that use important data so identify can be preserved. The commenter recommends that any disclosure of de-identified data should require approval by an IRB. 
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science. 
14.1.5.5.3 Confidential Business Information
Comment: Commenter (6362) recommends that, in circumstances where company CBI and other intellectual property may be implicated, EPA should confer with the CBI data owner to determine how to make that data available to the greatest extent possible without disclosing the CBI within that data, study, or model. The commenter states that how this is handled will likely be impacted by the type of regulatory decision and statute involved. Commenter provides the following example:
      For example, under TSCA, while the summarized study results, analysis, and final report may be publicly available, the underlying data in a health and safety study may qualify as CBI when the underlying data are not in the public domain and that data provides a commercial value to its owner. In such circumstance, it is the availability of the underlying data that determines whether or not an unpublished study can be used by a competitor to support its notification or registration of a substance overseas without obtaining ownership or citation rights to use such data, depriving the data owner of the value of its investment in the underlying data. Current EPA regulations require chemical manufacturers to submit health and safety studies under some circumstances. However, it is noteworthy that none of these regulations routinely require study submitters to submit underlying data along with a final report. This indicates that the final report likely communicates sufficient information about the potential health and environmental effects to the public when a company has submitted health and safety studies in which it has a commercial interest in protecting.
Commenter (6362) believes that making a final study report publicly available where the underlying data are CBI would, in most circumstances, be an effective way to make relevant information publicly available about studies and data EPA may rely on, but which must be protected as CBI in circumstances triggering this policy. The commenter provides that, in these situations, EPA can access the underlying data to confirm the methods, models, and approaches are based on validated procedures, accessible data, etc. According to the commenter, if necessary, when specialized expertise is needed, EPA could contract with an independent third-party science reviewer to confirm those findings, although we believe this would likely only be necessary in unusual circumstances. 
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The EPA will continue to follow existing requirements/guidance for handling CBI.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed.
Comment: Commenter (6130) states that the proposed rule establishes increased protections for CBI, diminishing the amount of information available to the public to inform policy, whether from the scientific community or from the business community. 
Commenters (0559, 6120) assert that CBI may be necessary in some cases for scientific experts to evaluate the potential for hazards due to chemical exposures e.g., in consumer products. The commenters suggest that responsible authorities, including state agencies, the EPA, and independent scientific reviewers should be able to request and have access to CBI when necessary to evaluate these hazards.
Commenter (5170) states that because CBI may be necessary to evaluate health risks from exposures to some products, EPA and independent scientific reviewers should be able to access CBI in assessing these risks. The commenter states that, given the recent apparent scientific fraud committed by the founder of Theranos, it would seem that CBI and industry-based science are due substantial scrutiny and transparency.
Commenter (6120) asserts that one significant way to improve transparency is to increase access to CBI; this would give the public confidence that this information is independently reviewed by experts in relevant fields. The commenter recommends that EPA establish a group of independent scientists who could serve as scientific reviewers of CBI data. The commenter provides that this review could be done under strict confidentiality agreements and would ensure more confidence in the scientific vigor of the data submitted. The commenter urges the EPA to follow the OMB guidelines for peer review to ensure peer reviewers are independent of industry and include academic scientists and allow scientists who receive government research grants to offer "independent scientific advice to the agency on other projects."
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science. 
Comment: Commenters (6126, 6906, 6935) support the need for CBI provisions/standards to protect data from disclosure. 
Commenter (6906) commends the Agency for acknowledging the importance of CBI and simultaneously establishing that while transparency is key, data transparency must be conducted in a way where CBI and intellectual property are retained. The commenter states that industry heavily relies upon the use of CBI provisions in various EPA programs to help retain competitive advantage between companies. According to the commenter, this could be especially pertinent when addressing industry-sponsored testing data, modeling and risk assessments. The commenter provides that, additionally, many studies (especially dose response data and models, which are the focus of this rule) have a significant amount of personal information that would need to remain confidential as well. The commenter states that, for both instances, EPA needs to determine a standard to either sanitizing certain aspects of the data, so that they can be made publicly available, or a procedure regarding selective publication so that certain aspects of these studies can also be made publicly available. 
Commenter (6935) suggests that any final rule should include a detailed, objective and consistent process for safeguarding CBI and that fits seamlessly with the Administrator's potential exemption process after it is expanded to provide additional information on criteria to be considered and procedures for exemptions.
Commenter (6126) urges EPA to incorporate Recommendation 3-4 of the U.S. Commission on Evidence-Based Policymaking which encourages the adoption of activities that protect public trust for confidential data.
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed.
Comment: Commenters (6112, 6115, 6855) support the Agency's proposal and state that its policy and guidance documents sufficiently address concerns about unauthorized access to confidential personal and business information. Commenter (6162) states that, to allay concerns regarding the disclosure of CBI, the Alliance supports the proposed regulatory text of §30.5:
      Where the Agency is making data or models publicly available, it shall do so in a fashion that is consistent with law, protects privacy, confidentiality, CBI, and is sensitive to national and homeland security. 
Commenters (6448, 6918) supports EPA's efforts to balance appropriate protection for copyrighted or CBI, including where protected by law, with requirements for increased transparency of pivotal regulatory science. Commenter (6448) notes that there will be situations in which CBI cannot be shared publicly but this should not preclude its inclusion in regulatory decisions. Commenter (6918) states that EPA must, and should, continue to recognize limits on data access and modify, as appropriate, efforts to enhance transparency to reflect these lawful, reasonable limits. 
Response: Section 30.5 in the final rules states, "The Agency shall also give greater consideration to pivotal science based on dose-response data that include confidential business information, proprietary information or PII if these data are available through restricted access in a manner sufficient for independent validation." It also states, "For pivotal science where there is no access to dose-response data, or access is limited, the Agency may still consider these studies but will give them lesser consideration unless the Administrator grants an exemption under 40 CFR 30.9."
Comment: Commenter (6379) urges EPA to establish a process to maintain confidentiality through a peer review process to ensure EPA can continue to rely on CBI even though it is not publicly disclosed while also ensuring that scientific rigor is applied to consideration and use of this information. 
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed.
14.1.5.5.4 Proprietary Data
Comment: Commenters (6150, 6191, 6375) provide recommendations on how to balance protection and transparency with regards to proprietary data.
Commenter (6150) asserts that the protection of proprietary data, models and methodologies should not apply to research that is funded by EPA or was developed under EPA contract. The commenter states that utilization of private models or failure to disclose methodologies should be discouraged, if not deemed a disqualifier, as such actions fail to meet minimum transparency requirements. 
Commenter (6191) requests that intellectual property considerations be noted and that data with intellectual property or confidentiality concerns should not be shared. The commenter states that the sharing of data containing patentable information or trade secrets should not be required due to the potential negative economic implications of such a requirement. 
Commenter (6375) suggests EPA solicit input from EPA Offices (such as the Office of Pesticide Programs) or other federal agencies (such as FDA) that rely heavily on copyrighted information, proprietary studies, and CBI for strategies to balance these concerns with transparency. The commenter states that, for example, EPA Offices (e.g. Pesticides) that rely heavily on proprietary studies/CBI apparently have already devised mechanisms that balance protection with transparency of pivotal regulatory science. The commenter provides that one mechanism is Data Evaluation Records (DERs) that summarize proprietary studies. The commenter provides that another is human health risk assessments that integrate findings in DERs such that stakeholders can assess the impact of a particular endpoint (e.g. health effect) in a particular study (pivotal regulatory science) on the overall risk assessment. The commenter states that both DERs and risk assessments have been made available in the docket or by other means. According to the commenter, while not as transparent as making the underlying data and models fully available, sufficiently detailed DERs and risk assessments are arguably as detailed and transparent as many health studies and risk assessments published in the peer-reviewed scientific literature. The commenter states that, regarding copyright protection, a potential alternative for copyrighted works that use data that are federally funded may be to solicit from the authors the raw data (which often isn't copyrighted) that were used to produce the copyrighted work. The commenter notes that these raw data, along with EPA's analysis of these raw data, could then be made publicly available.
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed.
Section 30.5 in the final rules states, "The Agency shall also give greater consideration to pivotal science based on dose-response data that include confidential business information, proprietary information or personally identifiable information if these data are available through restricted access in a manner sufficient for independent validation." It also states, "For pivotal science where there is no access to dose-response data, or access is limited, the Agency may still consider these studies but will give them lesser consideration unless the Administrator grants an exemption under 40 CFR 30.9."
In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule.
14.1.5.5.5 Non-CBI Animal Studies
Comment: Commenter (6942) recommends that the Hexavalent Chromium Mode of Action Research study serve as a model for other research programs that desire to foster collaboration in the research community and promote transparency and validity of the scientific data used in regulatory risk assessment. The commenter provides that similar approaches, using the internet and web-based repositories, could be implemented for both government-funded and stakeholder-funded research. The commenter states that additional resources may be needed to accommodate the proposed rule, but with regard to animal research data, they do not anticipate that significant hurdles would arise, unless the pivotal study is highly dated, or the best available science. The commenter provides the following information regarding their experience:
      For recent research and risk assessment projects, we have used confidential, personal identifying information, with Institutional Review Board (IRB) approval, including data collected at the federal level (e.g., Social Security and National Death Index records), and we have also used de-identified personal information generated by the National Institute for Occupational Safety and Health (NIOSH) and the Institute of Medicine (IOM) for research projects related to the development of pivotal data for risk assessment. Therefore, we believe we offer highly informed comments to the proposed regulation based on specifically-relevant experience.
      In that regard, we offer the example of our Hexavalent Chromium Mode of Action (MOA) Research study. During this nine-year project, ToxStrategies created a vast repository of in vitro and experimental animal data, including original toxicology data, modeling code, and toxicogenomics, and that body of data has been made freely available for use by other scientists and regulators seeking to analyze the raw data (https://cr6study.info). Both U.S. EPA and California EPA have accessed and downloaded the raw data sets. The Hexavalent Chromium Mode of Action Research study was developed to better understand how hexavalent chromium caused cancer in rodents from drinking-water exposures. Key objectives of the research study were to (i) evaluate the MOA, (ii) develop and make assessible physiologically based pharmacokinetic (PBPK) data and models allowing for inter- and intra-species extrapolations in risk assessment, and (iii) ultimately to provide data that will improve upon the science used for risk assessment and setting drinking-water standards for Cr(VI). This research study has been conducted as a collaborative effort among multiple organizations, universities, and government researchers, and full disclosure of the original data has always been our objective.
Response: The final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science. 
14.1.5.6 Definition, Guidance, Standards and Training are Needed
Comment: Commenters (1008, 1011, 6126, 6362) recommend that EPA provide training/guidance to researchers to support making research data publicly-accessible. Commenter (1008) states that, should this proposed rule place transparency requirements on EPA-funded researchers, these researchers must be provided appropriate training and support to make research data created in future research studies publicly accessible. Commenter (6126) states that, if the intent of EPA is to rely solely on de-identified data, some researchers may lack the appropriate expertise to sufficiently de-identify data without additional training, support, and guidance on techniques and approaches for doing so. Commenter (6362) suggests EPA develop guidance on protecting privacy, de-identifying data, and settling disputes should a breach occur. Commenter (1011) suggests that EPA provide researchers guidance regarding proper channels for data dissemination. 
Response: The final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA or any other entity to collect, store, or publicly disseminate dose-response data underlying pivotal science.
14.1.6 Other Implementation Issues that May Require Special Consideration
14.1.6.1 Consideration of Differing Regulations and Requirements
Comment: Commenter (6191) states that, as research is increasingly the product of international collaboration, they recommend that differing regulations and requirements worldwide be considered. 
Response: In section 30.9 of the rule it states, "The Administrator may grant an exemption to this part for a study on a case-by -case basis if he or she determines that greater consideration is warranted because: (1)technological or other barriers render sharing of the dose-response data infeasible; . . ."
14.1.6.2 Consideration of Secondary Data Analysis Data Use Agreements
Comment: Commenter (6191) states that where secondary data analysis is being conducted, data use agreements prohibit broader dissemination of data, which would also prohibit data from being released publicly.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
14.1.6.3 Regulatory Determinations Should be Based on More than One Study
Comment: Commenter (6875) encourages EPA to not base any regulatory determination on a single study, regardless of whether or not the data is publicly available. The commenter suggests that relying on a single study, no matter how robust it may be, can bring bias to a model or regulatory decision. If EPA must base a decision off of a single study, the commenter recommends that the data be publicly available as well as the reasoning and methodology behind why the study was chosen, how it is being used and why no other studies were deemed sufficient to be included.
Response: The rule will apply to the pivotal science underlying significant regulatory actions and influential scientific information. Section 30.2 states, "Pivotal science means the dose-response studies or analyses that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information." Pivotal science is not necessarily a single study or analysis. 
14.1.7 Timing of Implementation 
14.1.7.1 Do Not Delay Implementation of Proposal
Comment: Commenter (6150) encourages EPA to not delay implementing core components of this proposal until the Final Rule is issued. The commenter asserts that it is appropriate and consistent with federal statutes for EPA to immediately begin to identify key studies that are likely to be pivotal in significant upcoming rulemakings immediately. The commenter provides that EPA could seek public comment on a list of these potential studies/models, allowing the public to nominate other studies or models. The commenter states that identification now would allow time for EPA to contact researchers and develop appropriate data sharing approaches. The commenter adds that such early action could also assist in identifying research used by EPA in the past that may need additional disclosures. 
Commenter (6867) suggests that the EPA should not delay implementation of this rule unless required to do so by statute. The commenter states that certain commenters claim the EPA has not complied with specific statutory obligations. The commenter notes that they do not have a position on the EPA's statutory obligations, but they encourage the EPA to carefully comply with applicable procedural requirements so as to minimize the risk of dilatory litigation. 
Response: The final rule is effective on the date of publication in the Federal Register. The final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Also, the rule does not require any researcher or other outside entity to provide data to the EPA. Researchers may choose to make more data available, but the EPA recognizes that these parties will weigh their own benefits and costs and make choices that they deem appropriate.
30.7 of the final rule states, "To assist the Administrator in determining whether to grant an exemption, the EPA program or Region responsible for the significant regulatory action or influential scientific information and public commenters can provide input when the Administrator is considering an exemption." Forthcoming guidance/substantive rulemakings subject to notice and comment could be further opportunity for input from the public. 
14.1.7.2 Request for Implementation to be Delayed Until After Proper Procedures and Infrastructure are Established
Comment: Commenter (6908) suggests that EPA not implement all or portions of the regulations until the proper procedure and infrastructure, such as IT systems and databases, are in place and functional.
Response: The final rule is effective on the date of publication in the Federal Register. However, the Agency plans to have implementation guidance available by the date of publication in the Federal Register. However, as outlined in 40 CFR 30.3, the provisions of the rule apply to significant regulatory actions for which a proposed rule was published in the Federal Register after the effective date of the rule and influential scientific information submitted for peer review after the effective date of the rule. Further, EPA intends to issue implementation guidelines and statute-specific rulemakings that will provide further clarity. 
14.1.7.3 Ensure Data Access Early in the Rulemaking Process
Comment: Commenter (6375) suggests EPA should ensure that data access occurs early enough in the rulemaking process so that the public has sufficient time to analyze the data.
Response: The rule requires the EPA to identify and make publicly available the science that serves as the basis for informing a significant regulatory action at the proposed or draft stage to the extent practicable. The definition of publicly available in the final rule includes, "The public must be able to access the information on the date of publication of the proposed rule (or, as appropriate, a supplemental notice of proposed rulemaking, or notice of availability) for the significant regulatory action or on the date of dissemination of the draft influential scientific information for public review and comment.
14.1.8 Impacts on other Federal Agencies, States and Tribes
14.1.8.1 State and Tribal Implementation
Comment: Commenter (6125) expresses concern that the proposal does not describe how it will be implemented in regard to states and tribes. The commenter states that federal standards may govern the level of environmental protection within states and on tribal lands. According to the commenter, those levels of protection would be reduced under a new regulation that ignores evidence because of the new rule. The commenter asks the following related questions:
 Would states or tribes have any recourse when this happened, or could they keep the old standards? 
 Under what circumstances would or could EPA disapprove a program for failing to adhere to the new federal law requirement? 
 Does EPA believe that a state with an authorized program based on actual science would still meet federal standards after the program becomes regulated under censored science providing for reduced federal standards? 
 Would EPA disapprove a state program if the state failed to relax its standards to reflect new federal requirements? 
 What kind of oversight would there be when states make permitting or enforcement decisions, actions that are not covered by the proposal but possibly implementing regulations that are covered. 
 Has EPA considered whether it would be consistent with the federal trust responsibility to tribes to relax requirements or standards, and thus reduce protection for tribal lands solely because of a new regulation based on ignoring reliable and relevant science? 
 Would such a reduction in protection on tribal lands be consistent with principles of environmental justice?
Response: EPA plans to issue implementation guidelines and statute-specific transparency regulations or programmatic regulations. The final rule states, "The action imposes no enforceable duty on any state, local or tribal governments or the private sector." The EPA does not agree that this rulemaking would weaken human health protections. The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety.
14.1.8.2 Relationship Between EPA-Funded Research and Research Funded by Other Agencies
Comment: Commenter (6126) suggests that EPA has ill-considered the relationship between EPA funded research relevant for regulatory actions, and the vast amount of research funded by other agencies (e.g., NSF and NIH) or through non-federal funding sources (e.g., many universities, states, foundations, private firms). The commenter asks: 
 Would EPA request data when needed or would researchers be expected to provide it unsolicited or on-demand? 
 What obligations would researchers have to provide such information? 
 What would be the consequences for not doing so? 
 If EPA cannot obtain data from studies and thus excludes those studies from consideration, how can EPA ensure use of the "best available science?" 
Response: This final rule does not regulate any entity outside the EPA. Rather, the requirements modify the EPA's internal procedures regarding the transparency of pivotal science underlying significant regulatory actions and influential scientific information.
14.1.9 Communications 
Comment: Commenter (6126) suggests that government agencies develop a "learning agenda," which would be operationalized as a strategic plan for evidence-building activities. The commenter states that, to the extent EPA is planning regulatory activities well in advance, the learning agenda is intended to signal to researchers inside and outside government what the priorities are and could also be used to indicate data priorities. 
Response: The final rule does not regulate any party outside of the EPA but rather exclusively governs the EPA's internal process for determining the consideration to afford pivotal science with respect to certain actions. This rule does not require any researcher or other outside entity to provide data or models to the EPA. Nor does the rule categorically exclude studies -- even studies where the underlying dose-response data are not available for independent validation -- and therefore any incidental impact on researchers who are developing science and deciding whether to make the underlying dose-response data available is negligible. Instead, it governs internal agency procedures for determining the consideration to afford various studies according to factors that include data availability. In doing so, the final rule provides greater transparency on the consideration the EPA will give pivotal science where the underlying dose-response data are or are not available for independent validation.
If researchers want to increase the likelihood that their studies receive greater consideration by the EPA, they may take steps to ensure that the underlying dose-response data are available to the greatest extent possible. But any such response to this final rule would be purely voluntary. It is not required by this rule.
Comment: Commenter (5185) suggests that EPA ensure that relevant transparency information is incorporated into public communications and marketing materials associated with regulatory initiatives. For example, if the claimed benefits of a proposed action are based on non-transparent information, the commenter recommends that the Agency adopt a practice of disclosing this in press releases and fact sheets. The commenter states that, paired with an enhanced focus on risk communication, this practice will lead to greater public understanding of the scientific and other inputs that drive important EPA policies.
Response: Section 30.4 of the final rule states, "The EPA shall clearly identify all science that serves as the basis for informing a final significant regulatory action. The EPA shall make all such science that serves as the basis for informing a significant regulatory action publicly available to the extent permitted by law"
14.1.10 Implementation Costs
14.1.10.1 Inclusion of Cost Analyses in the Proposal
14.1.10.1.1 More Robust Analyses of Implementation Costs Are Need
Proposal Fails to Adequately Consider the Costs of Implementation 
Comment: Commenters (1012, 1281, 1346, 1388, 2024, 2787, 3135, 5021, 5022, 5165, 5579, 6109, 6111, 6122, 6126, 6133, 6185, 6233, 6168, 6373, 6375, 6925, 8272, 9227) generally assert that EPA has failed to adequately consider the costs of implementation. 
Commenter (1388) states that in its proposal, the EPA has not fully accounted for the cost and consequences of excluding valuable information. Commenter (5579) provides that data transparency comes at costs which the EPA does not elaborate on in this rule and which leads the current rule as it stands to be incomplete. Commenter (6109) states that EPA does not explain how it will pay for compliance with the proposed rule or where the personnel hours will come from. Commenter (6375) suggests EPA should assign designated personnel and allocate sufficient funds to facilitate data access. 
Commenter (6126) states that, in the absence of incentives to provide information as requested by EPA for regulatory actions, researchers that are not funded or mandated by EPA appear to have no direct incentive to comply with EPA's regulation. Commenter (6133) states that EPA does not confront the financial dimensions or the need for financial incentives to support the unprecedented data release requirements in the rule. 
Commenter (9227) contends that the proposed rule fails to acknowledge or provide for costs of the rule related to data masking, de-identification, deposition in public data repositories, and controlled access in federal RDCs. The commenter states that there are various costs to the agency of administering the regulation, which include contacting researchers, gathering data, and ensuring that patient confidentiality and CBI are not disclosed. The commenter states that additional costs could also be incurred because of the need to conduct additional peer reviews required by proposed §30.7 and any additional analyses imposed by proposed §30.6's requirement that "EPA shall clearly explain the scientific basis for each model assumption used and present analyses showing the sensitivity of the modeled results to alternative assumptions." 
Response: The EPA agrees that neither the 2018 proposed rule nor the 2020 SNPRM included a characterization of costs or benefits to the Agency. The final rule acknowledges that the EPA will incur some administrative costs associated with the rule, but the EPA finds that these costs are likely to be small given that the additional requirements are limited to pivotal science, and that EPA already conducts extensive review of the science when promulgating regulations or publishing influential scientific information. See III.J of the final rule preamble for further discussion of benefits and costs of the rule.
The rule does not require any action by any party other than the EPA and imposes no costs on other parties. The rule also does not require EPA or any party to disseminate scientific data and incur associated costs. The final rule does not require additional peer review, though the EPA will evaluate whether or not to conduct additional peer review of individual studies identified as pivotal science if the studies have already undergone peer review.
Comment: Commenter (6915) expresses concern that, despite the existing peer review process, EPA apparently proposes to require that EPA itself conduct an additional "independent" review. See 83 Fed. Reg. at 18774. Yet, the commenter notes that the proposal nowhere discusses how EPA would vet reviewers to identify persons who are purportedly more competent than those already used in past or current peer review processes, or the level of EPA staffing and associated costs that would be needed for additional review -- only stating that EPA will implement the proposed rule in a manner "that minimizes costs." Id. at 18774. The commenter adds that the EPA fails to acknowledge the rigor of existing processes in statutes, policies and federal procedures, or to explain how its proposal would provide any added value and minimize costs. 
Response: The EPA is clarifying that 40 CFR 30.6 does not require de novo peer review on studies that have already undergone adequate peer review. The EPA will evaluate whether or not to conduct additional peer review of individual studies identified as pivotal science if the studies have already undergone peer review. The EPA's determination of what constitutes adequate peer review is made consistent with the EPA's 2015 Peer Review Handbook.
Paperwork Reduction Act, Executive Orders, and OMB Guidance
Comment: Commenter (2024) states that EPA should do a burden estimate under the Paperwork Reduction Act (PRA). 
Commenter (6125) states that the proposal mentions cost-effective strategies for providing public or restricted access to shared data and methods in repositories that might involve collaboration with other federal agencies, but never commits to any path, nor indicates that the agency would bear the cost in time and effort for researchers who would be willing to use them. The commenter states that, instead, EPA claims that the proposal "does not contain any information collection activities and therefore does not impose an information collection burden under the PRA." According to the commenter, EPA cannot have it both ways: either researchers are compelled by a new requirement to expend significant resources on arrangements to ensure the implications of their research for policy are considered or they have no burden because EPA will pay for it. 
Response: The rule does not require any action by any party other than the EPA and imposes no paperwork collection burdens or other costs on other parties. The rule also does not require EPA or any party to disseminate scientific data and incur associated costs. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
Response: In the final rule the EPA has identified and described some incremental costs that the Agency may incur as a result of the rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
Comment: Commenter (6137) states that EPA's proposal will have serious implications on public health, privacy, the environment, and the judicial review process, and thus the costs will be significant. The commenter asserts that there is no evidence that EPA evaluated any of these adverse impacts, quantitatively or qualitatively. According to the commenter, the EPA cannot ignore these harms in its cost-benefit analysis, even if the costs are not readily quantifiable. See, e.g., Michigan v. EPA, 135 S. Ct. 2699, 2707 (2015) (costs include "harms that regulation might do to human health or the environment"); Executive Order 12866, 58 Fed. Reg. 51,735, 51,735 (Oct. 4, 1993) (it is "essential to consider" the "qualitative measures of costs and benefits that are difficult to quantify"); see also Food Labeling: Nutrition Labeling of Standard Menu Items in restaurants and Similar Retail Food Establishments; Extension of Compliance Date; Request for Comments, 82 Fed. Reg. 20,825, 20,828 (May 4, 2017) (acknowledging that delaying a nutrition labeling requirement would lead to millions of dollars in lost health benefits). 
Response: The EPA disagrees that the proposal will have the serious implications described by the commenter. As described in the rule, the EPA is maintaining language in 40 CFR 30.3 stating that the statutes EPA administers, or their implementing regulations, will control in the event of any conflicts with the requirements of this rule. The Agency will continue to comply with and abide by the requirements in those statutes and implementing regulations.
Comment: Commenters (6125, 9227) assert that Agencies are encouraged to weigh the costs and benefits of developing higher information quality in OMB's Information Quality Guidelines. The commenters note that costs that the Guidelines encourage agencies to consider include costs attributable to agency processing effort, respondent burden, maintenance of needed privacy, and assurances of suitable confidentiality. Commenter (9227) states that EPA's existing information quality guidelines track the OMB Guidelines closely. Commenter (6125) states that the proposal contains no evidence that the EPA has conducted such an assessment; instead the proposal simply states the belief that the benefits of the rule will be greater than the monetary costs (and potential disbenefits), without providing any quantitative evidence to support that belief. Commenter (9227) argues that EPA's disregard of the Guidelines' recommended weighing of costs and benefits further contributes to the arbitrariness of EPA's failure to consider the costs of the proposal.
Response: The EPA agrees that the benefits and costs of the rule were not fully characterized in the 2018 proposed rule or the 2020 SNPRM. However, in the final rule the EPA has identified and described the benefits of the rule and some of the incremental costs that the Agency may incur as a result of the rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
Mercatus Paper
Comment: Commenters (6111, 6916, 9227) express concern that the proposal inappropriately relies on the Mercatus Paper. 
Commenter (6111) argues that EPA cannot abdicate its responsibility to conduct its own analysis of the costs and benefits of this regulation by relying on the paper by Randall Lutter and David Zorn for the Mercatus Center cited in the proposal, which arrives at a lower cost estimate than the Congressional Budget Office (CBO), to support its conclusion that "the benefits of this proposed rule justify the costs."
Commenters (6916, 9227) assert that the only thing EPA relied upon in discussing the costs and benefits of the proposal was the industry-funded Mercatus working paper which rejects a CBO projection that proposed legislation, with terms like the Proposed Rule here, would cost the country $250 million a year. Commenter (6916) states that the Mercatus Paper does not find the new policy to be cost-free; instead it reports that compliance with the proposed legislation would add $2,558 to the costs of each study. 
Commenter (9227) states that the working paper of the Mercatus Center asserts that improvements in reproducibility "can be thought of as increasing the net benefits of regulation because they would avoid situations in which costs or benefits are wrongly estimated to occur or in which regulatory costs are imposed without corresponding benefits." The commenter asserts that, setting aside the lack of substantiation for this assertion, it entirely omits situations in which costs and benefits are wrongly estimated because the relevant science is not used -- and the costs that would be imposed on society if EPA inadequately protects communities from harmful pollution or toxic exposures.
Commenter (9227) states that even the Mercatus working paper notes that "[t]he cost of providing access to data has been one of the primary concerns about requiring access to data used by the federal government." The commenter states that, far from concluding, as the proposal suggests, an increase in net benefits from greater reproducibility, the Mercatus working paper simply explained a figure the authors were suggesting could be calculated (the point where net benefits would be positive); the authors do not themselves calculate the benefits, and admit that their "estimates of the benefits of public access to data supporting federal regulatory decisions fall short of proving that the benefits outweigh the associated costs." The commenter asserts that, while the Mercatus working paper disagrees with CBO's cost estimates, it does not argue that requiring access to data is cost-less; as it discusses the "costly activities and services that need to be performed," including activities related to "data collection and data accessibility." The commenter states that, according to that working paper, data collection requires "correspond[ing] with researchers and publishers to obtain the data, review[ing] the data for confidentiality concerns, format[ting] the data for public access, publicly post[ing] the computer code and models used in each study's analysis, and provid[ing] descriptions and documentation on how to obtain the date." The commenter states that data accessibility requires "computer processing services to construct and maintain data bases to store study-related information." According to the commenter, while the actual calculations put forward by the Mercatus working paper appear faulty (for example, it entirely omits the cost to researchers to compile and make their data public, does not include the costs of ensuring patient privacy is protected, and makes assumptions about the similarity of a chemical manufacturer collecting its own studies and EPA collecting and disseminating information of other researchers), the working paper acknowledges that there are costs, which the commenter states the EPA's proposal ignores.
Response: The final rule does not draw upon or characterize the findings of the Mercatus working paper. The EPA acknowledges that the benefits and costs of the rule were not fully characterized in the 2018 proposed rule or the 2020 SNPRM. However, in the final rule the EPA has identified and described the benefits of the rule and some of the incremental costs that the Agency may incur as a result of the rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
Costs Related to Deidentifying/Redacting Data
Comment: Commenters (1008, 2477, 2479, 4591, 5176, 6103, 6114, 6116, 6158, 6361, 6772, 6915, 9227) assert that proposal fails to identify the substantial costs related to the time, effort and resources required to make research data involving PII publicly accessible.
Commenters (1395, 2171, 6133, 6915) state that the CBO estimated the cost of redacting data at $100 million per year. Commenter (6915) states that these costs would encompass obtaining the underlying data, review of the data to address confidentiality concerns, formatting the data for public access, providing computer codes and models used, and providing directions for accessibility of the data, but would not include the additional costs related to the potential need for contractors due to EPA staffing issues to assist with this work. 
Commenters (1433, 2249, 6464) state that the process for removing confidential personal and business information before making it public would be virtually impossible, especially considering smaller Agency budgets. Commenter (4878) states that, even if there was an acceptable way to mask personal data while maintaining enough information to comply with the rule, costs of such anonymizing are prohibitively expensive. 
Commenter (1008) states that the deidentification process is even more time consuming for data gathered in the past, where research subjects did not consent to public access of information gathered about them. Commenter (2477) states that one cannot ignore that the cost to protect privacy by anonymizing all data is substantial. Commenter (4591) states that de-identification to maintain privacy is not simply a matter of redacting names from documents, but a complex and resource-intensive process. Commenters (6114, 6116, 6361) state that it is hard to imagine that these extra costs could be borne in an era where EPA funding has been cut, or at best has remained stagnant. Commenter (6915) states that redacting confidential data from large studies is a huge job that can take thousands of hours, at commensurately high cost. 
Commenters (5176, 9227) assert that, in proposing to redact private information, the proposed rule fails to acknowledge the enormously burdensome and costly complexities involved in redacting such information in a way that both protects individuals' privacy and ensures that the data is still useful for future analysis. Commenter (9227) states that while EPA mentions a range of options for facilitating secure access to confidential data, EPA fails to recognize the significant costs associated with implementing most of these options or the risks to privacy that remain even if these methods are employed. 
Response: The rule does not require any action by any party other than the EPA and imposes no costs on other parties. The rule also does not require EPA or any party to disseminate scientific data and incur associated costs. The analysis by CBO noted by the commenter is not applicable to this rule because the analyses were not for this rule and the requirements analyzed are not the same as those in this rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
Comment: Commenter (2006) expresses concern that, while the proposal suggests that access to private data may be addressed through a RDC, RDCs are not for casual users or the general public and have serious limitations and costs, even for researchers. The commenter states that the proposal in footnote 16 refers, in part, to the U.S. DHHS, which is responsible for the National Health and Nutrition Examination Survey (NHANES). The commenter states that confidential data are available only through an RDC. According to the commenter, public access to confidential NHANES data at RDCs requires payment of "management fees" (i.e. fees to have the data made available at the RDC, ranging from $1,800 to $15,000 per project, skills with specific statistical software (usually SAS), and statistical expertise. The commenter states that access is contingent upon approval of a research proposal, which takes time for the approval as well as to schedule a date to visit the RDC. The commenter states that additional costs would also include transportation and lodging costs for RDCs which are not in one's home town. According to the commenter, no hard copy materials are to be brought into or removed from the RDC which means that analyses must be thoroughly designed before the visit and one must be able to interactively analyze the data as findings are made during the analysis. 
Commenter (9227) states that deidentification is complicated and costly. The commenter asserts that, while EPA states that "[o]ther federal agencies have developed tools and methods to deidentify private information," the EPA cites only one source, the U.S. DHHS' Guidance Regarding Methods for De-identification of Protected Health Information in Accordance with the HIPAA Privacy Rule. According to the commenter, this guidance provides two methods for de-identifying data: (1) expert determination method, where an expert determines that, after application of statistical and scientific principals and methods, the risk is very small that the information alone or with other available information could be used to identify the subject; and (2) the safe harbor method, requiring that a number of identifiers are removed. The commenter states that the first method requires case-by-case work, and EPA has provided no information regarding how EPA or others could potentially implement it or how much it might cost. The commenter states that the second method is also costly (which EPA disregards) and requires removal of much information that may be necessary to be able to reanalyze or reproduce the research results, so it is unclear whether it would satisfy EPA's requirements in the proposal. 
Response: The EPA recognizes that there may be costs associated with accessing data through RDCs and conducting research. However, the rule does not require any action by any party other than the EPA and imposes no costs on other parties. The rule also does not require EPA or any party to disseminate scientific data and incur associated costs. See III.J of the preamble for further discussion of benefits and costs of the rule.
Data Management and Acquisition Costs Should Be Addressed
Comment: Commenters (1008, 1012, 2006, 2472,3135, 5419, 6046, 6126, 6127, 6144, 6449, 9227) express concern that the proposed rule offers no information on where the raw data collected would reside, how the resources to manage the data would be obtained, or how the agency would deal with legally and ethically protecting data that is confidential or sensitive. Commenter (1008) states that EPA should allocate resources to develop the digital infrastructure necessary to publicly share research data, especially sensitive data.
Commenters (6046, 6449, 9227) express concern that the rule needs to take into consideration the cost of establishing and maintaining the electronic infrastructure necessary for obtaining and storing data and models. Commenter (6046) recommends that, while the proposed rule does not estimate EPA's cost for an RDC (including space, costs for storage, ongoing maintenance costs to assure security, and staff to manage the data), nor assign responsibility for such a center to any EPA division, the EPA should investigate all options, including the options' feasibility, costs, etc, for keeping data confidential if the data are to be made public. Commenter (6449) states that the cost of safeguarding integrity of the data also needs to be considered. 
Commenters (2006, 6046, 6127, 9227) state that the expense of data storage in a public repository can be significant. Commenter (9227) provides that even smaller costs could be prohibitive for many researchers.
Commenters (1011, 2006, 6126, 9227) suggest EPA needs to provide funding for data infrastructure. Commenter (1011) states that EPA should provide sufficient funding to cover the indirect costs associated with preparing and disseminating data generated by researchers funded through grants or cooperative agreements. Commenter (2006) suggests EPA should set aside money to grant to scientists for public data storage. Commenter (6126) states that EPA must ensure resources are available to enable a data infrastructure and responsible use of evidence, consistent with the principles for evidence-based policymaking. 
Commenters (0039, 4537) provide the following recommendations regarding EPA funding: 
 EPA prioritize its funding toward upgrading existing research data so that it may meet the standard of best available science; e.g., by anonymizing research data so as to preserve privacy, confidentiality, etc.
 EPA should provide substantial funding for a "reproducibility architecture" of hardware and software to facilitate the production of reproducible science by all American scientists whose research informs the EPA's decision-making.
 EPA should provide substantial funding so as to make it possible and then required for all EPA notices, proposed rules, regulations, etc. to include easily accessible links to all scientific materials used to justify these EPA actions. These links should include all relevant scientific research that meets the BARS standard. but that does not support the proposed EPA action.
 EPA's granting program should establish a funding category, with funding priority over all other categories, for meta-analysis and research into publication bias.
Commenter (6144) states that, while EPA's proposed rule references a system used by the NIH to collect health data, there are barriers and questions to how such a system could apply to EPA. The commenter states that, while the administration asserts that the proposal is about transparency, the NIH system is not publicly accessible and requires significant resources to manage and share with researchers. The commenter expresses concern that, at a time when EPA's budget and capacity continues to decrease, it seems unreasonable for the Agency to initiate such an undertaking. 
Response: The EPA recognizes that there are costs associated with hosting data for public access. However, this rule does not require EPA or any party to disseminate scientific data and incur associated costs. The rule does not require any action by any party other than the EPA and imposes no costs on other parties. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
Comment: Commenters (1941, 5419) recommend that EPA take the steps necessary to ensure that future efforts to gain access to the data underlying pivotal study data are streamlined and cost-effective. 
Commenter (1941) describes the Diesel Exhaust in Miners Study (DEMS) study conducted by National Institute for Occupational Safety and Health (NIOSH) and National Cancer Institute (NCI) published in 2012 and the burdensome processes and expenses incurred by the Truck & Engine Manufacturers Association (EMA) to obtain access to full DEMS data sets. The commenter states that EMA's efforts to help facilitate independent investigators' access to the full DEMS data sets in a manner allowing for replication and reanalysis of the DEMS results took well over three years and well over $200,000.
Commenter (5419) encourages EPA to work with the Centers for Medicare and Medicaid Services (CMS) to streamline and simplify the data acquisition process and make the data available to the public free of charge. The commenter states that, otherwise, the cost and security requirements of obtaining these data may place an undue burden on stakeholders seeking to replicate analyses. The commenter provides that data accessed for an Electric Power Research Institute (EPRI) project were acquired at a significant cost (> $100,000) and a comprehensive data management plan that outlined rigorous security protocols was required. The commenter contends that these requirements would make replication of the work untenable for many researchers. The commenter expresses concern that the challenges of making these data available in an equitable fashion may result in the data and insights from this study being removed from consideration in standard‐setting activities for PM2.5, ozone, and NO2.
Response: The EPA first notes this rule does not require EPA or any party to disseminate scientific data and incur associated costs. However, the EPA may opt to undertake activities to gain access to data or to disseminate data when it is in the public interest to do so and will take the points from the commenters into consideration when applicable.
Costs to Researchers Need to Be Addressed - Overview
Comment: Commenters (0671, 1011, 2024, 4542, 4840, 4933, 6127, 6133, 6181, 6373, 6877, 6894, 6924, 6937, 6942, 9227) express concern that the rule will increase costs for researchers and thus have a burdensome effect on them. Many of these commenters contend that this will decrease the resources available for new research. 
Commenters (0671, 2024, 1011) assert that EPA's proposed regulation will require a lot of work by researchers, especially the labor necessary to post all their data in a publicly accessible manner no matter what form the data take and to certify to EPA that their work is properly available, such that EPA can use it. One commenter (6133) notes that scientists do not typically receive funding to make the data underlying peer-reviewed studies available for public inspection (6133). Other commenters (6181, 6894, 9227) contend that the cost of sanitizing such data would be immense and would be borne by researchers who do not necessarily have the resources or institutional support to complete the process. Thus, as commenter (6937) notes highly‐trained researchers and public health officials would need to spend time complying with the regulation rather than doing research. Commenter (6373) asserts that the level of effort required to prepare a public version of confidential data, including in the manner described in section 30.5 of the proposed rule, can be enormous. The commenter further states that the onerous requirements and expensive processes that the proposed rule would require most likely would put smaller laboratories conducting academic research at a disadvantage. Another commenter (4542) contends that even for purposes of reanalysis, simply making data available is insufficient; the participation of the original researchers is often required because of their unique expertise in the design, collection and analysis, especially for large or complex datasets. This is burdensome. 
Several of these commenters contend that unless the EPA pays the compliance costs of researchers, this action will have a significant economic impact on small entities such as researchers. Several of these commenters also suggest that EPA should increase Federal grant funding to off-set this increased burden. 
Response: The rule does not require any action by any party other than the EPA and imposes no costs on other parties. The rule also does not require EPA or any party to disseminate scientific data and incur associated costs. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
Comment: Commenter (0844) states that the rule "does not directly regulate any entity outside the federal government" and would be of interest "to entities that conduct research and other science activity relevant to EPA's regulatory activity." The commenter notes that researchers should be prepared to release their underlying data if they want their research to be considered by the EPA to promulgate regulations.
Response: The rule does not require any action by any party other than the EPA and imposes no costs on other parties. The rule also does not require EPA or any party to disseminate scientific data and incur associated costs. As described in the rule EPA will continue to consider all relevant and appropriate science regardless of data availability but will give greater consideration to studies where dose-response data are available in a manner sufficient for independent validation.
Costs to EPA
Comment: Commenters (0061, 1012, 4845, 5011, 6125, 6915, 8281-PH84, 9227) express concern that this rule would place a substantial new burden on the Agency and further hamstring the regulatory decision-making process to protect public health and the environment. 
Commenter (6125) states that EPA officials told the CBO that the Agency would likely greatly reduce the number of studies it relied on and would not take on the cost of disseminating the underlying data. 
The commenter (6915) suggests that any requirement for EPA to conduct additional review would entail additional significant costs, contrary to the proposal's assertion. 83 Fed. Reg at 18772. The commenter states that the practical outcome of the proposal is that EPA may end up relying on a much smaller number of studies and/or on a less robust subset of relevant available studies, thus undermining the regulatory decision-making process.
Commenter (9227) expresses concern that the proposal includes no discussion of whether funds would be made available to implement the rule, whether other EPA activities would be sacrificed, how these tradeoffs would be justified, and what the overall impacts might be on public health and the environment. The commenter states that with respect to the potentially very large costs that would accrue to EPA, EPA's proposal provides no indication that any funding to support such activities would be available. The commenter asserts that EPA funding is at its lowest level since the 1980s and, absent a significant change in Congressional priorities, any EPA expenditures for the purposes of supporting making data publicly available would necessarily require cutbacks in other critical areas of environmental protection, which might include supporting additional research, conducting inspections, issuing permits, setting standards, or many other activities. 
Commenter (6125) believes that EPA is not staffed to reanalyze the number of pivotal studies that could reasonably arise for all of its significant rules under all statutes, and EPA provides no reason to believe that staffing levels would be adequate. Therefore, the commenter suggests that the agency might need to enlist external analysts. The commenter contends that this would take substantial time and money. As a point of comparison, the commenter provides that the reanalysis of the American Cancer Society (ACS) and Six-City Studies, which included analysis of alternative model specification, cost the Health Effects Institute (HEI) approximately one million dollar. If just half of recent cohort studies were to submit their data through some limited use agreement, it could cost EPA on the order of $10 million to hire outside analysts to do the work. The commenter notes that if this level of analysis were required for even one or two studies in every major regulation, it would likely impair significantly the agency's ability to meet regulatory deadlines for the NAAQS or other programs, among other things precluding the agency from meeting its avowed goal of completing NAAQS reviews for particulate matter (PM) and Ozone by 2020.
Commenter (6144) suggests that it is clear that the current proposal would create a tremendous time and resource burden on EPA and on scientists outside of the agency, as they would scramble to track down, collect, compile, and post (including any statistical methods needed to conceal private information) all the data, models, code, etc. for each of the hundreds of studies EPA cites on any given decision. 
Response: The rule does not require EPA nor any other party to reanalyze data, only that EPA determine whether dose-response data for pivotal science are publicly available in a manner sufficient for independent validation. The final rule acknowledges that the EPA will incur some administrative costs associated with the rule, but the EPA anticipates that these costs are likely to be small given that the additional requirements are limited to pivotal science, and that EPA already conducts extensive review of the science when promulgating regulations or publishing influential scientific information. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule. The analysis by CBO noted by the commenter is not applicable to this rule because the analyses were not for this rule and the requirements analyzed are not the same as those in this rule.
Comment: Commenter (2561) expresses concern that this rule will open the EPA to numerous state and locality lawsuits, at the expense of the American taxpayer. 
Commenters (4839, 4845, 6196) express concern that the proposed regulation includes a case-by-case exemption process for data in significant regulatory decisions. The commenters contend that the result will be more cost, more bureaucracy, and may result in litigation over whether exemptions are justified.
Response: The EPA does not agree that this rule will lead to more and more costly litigation. The EPA does acknowledge the Agency will incur some administrative costs associated with the rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
Who Is Responsible for Covering Implementation Costs? 
Comment: Commenters express concern that EPA has not identified who will pay for rule implementation. Commenters (5419, 6142, 6125, 6446) state that EPA should develop a plan and either provide the resources to compensate researchers for the burdens imposed by this rule or identify who will pay specifically for which aspect of the rule, including, (1) who is responsible for the collection, de-identification, storage, and dissemination of sensitive data (2429, 2746, 2906, 6158, 6447), (2) who is going to compensate researchers for the increased compliance costs (0671, 6909, 6915, 6924), (3) how will EPA fund its activities (6125, 6126, 8274, 9227), and (4) who will be responsible for implementation of the rule and associated costs within the agency (6868).
Commenter (6909) states that many improvements to existing technologies and infrastructure are necessary before uniform data sharing can become a universal reality. Commenter states that it is not at all clear who will pay for these improvements. Commenter states that this proposed rule implicitly requires all these associated costly investments in improved technology and infrastructure without providing any funding for them. Commenter states that this is a huge unfunded mandate that researchers will not be able to cope with quickly, and the inevitable result will be sound science being ignored by EPA regulators who should be paying close attention to it.
Commenter (6125) states that the proposal mentions "cost-effective" strategies for providing public or restricted access to shared data and methods in repositories that might involve collaboration with other federal agencies, but never commits to any particular path, nor indicates that the agency would bear the cost in time and effort for researchers who would be willing to use them. The commenter states that, by not offering any support for the process, the proposal betrays its real motive namely to reduce the availability of scientific information that the agency now sees as inconvenient in view of its policy goal to retreat from environmental rules that have provided significant public health benefits. 
Response: The rule does not require any action by any party other than the EPA and imposes no costs on other parties. The rule also does not require EPA or any party to disseminate scientific data and incur associated costs. The EPA acknowledges that the Agency will incur some administrative costs associated with the rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
Costs will Result in Delays in Necessary Environmental/Public Health Standards
Comment: Commenter (5120) expresses concern that the agency will not be able to support the proposal's requirements within a timely manner, given that the Agency has been targeted for a twenty-three percent decrease in its budget.
Commenter (9227) states that, if EPA complies with the regulation not by spending the money to make data publicly available, and if the research community does not bear those costs itself, then it appears that EPA would simply ignore studies that do not comply with the regulation (EPA is proposing to exercise its discretionary authority to establish a policy that would preclude it from using such data in future regulatory actions). The commenter states that course of action has its own significant costs, and EPA provides no analysis in the proposal of the magnitude of studies that it has previously relied upon that it could no longer rely upon in regulating. 
Commenter (6872) states that it is critical that the impacts of the proposed rule on EPA resources and capacities to complete toxicity assessments, and their downstream impacts on the states, be evaluated and addressed. The commenter asserts that EPA completing assessments of chemical toxicity and health effects are critically needed by the states to protect public health and the environment. The commenter states that a delay in EPA action on completing toxicity assessments and standards as a result of the proposed rule will necessitate state resources to complete these toxicity assessments and standards in order to protect public health and the environment. 
Response: The final rule EPA will consider all appropriate and relevant science consistent with the Agency's statutory obligations. Greater consideration will be given to pivotal science where dose-response data are available in a manner sufficient for independent validation. EPA acknowledges that the Agency will incur some administrative costs associated with the rule, but finds that these costs are likely to be small given that the additional requirements are limited to pivotal science, and that EPA already conducts extensive review of the science when promulgating regulations or publishing influential scientific information. The Agency believes the incremental burden of assessing the availability of data and models for pivotal science is unlikely to result in significant delays in promulgating regulations or publishing influential scientific information. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
Comment: Commenters (6163, 6358, 8272) suggest that EPA should assess the cost of eliminating science from consideration. 
Commenters (6358, 8272) state that the proposal should include an examination of the studies that have previously informed regulation and the extent to which they will meet the proposed criteria; an analysis of regulations that would not have been promulgated, would have included less-protective limits, or that could be weakened in the future under the proposed criteria; an analysis of the time and costs required to bring all relevant studies into compliance with the criteria; and a calculation of the foregone public-health benefits and associated costs to the U.S. economy when regulations are delayed or are less protective due to this rule.
Response: The final rule states that the EPA will consider all appropriate and relevant science consistent with the Agency's statutory obligations. Greater consideration will be given to pivotal science where dose-response data are available in a manner sufficient for independent validation. The Agency does not believe it is feasible or necessary to attempt an analysis of how past regulatory decisions may have been made differently. The rule does not require EPA nor any other party to change relevant studies or bring them into compliance with any standard. The EPA disagrees with commenters that requirements of this rule will result in any meaningful delay in promulgating regulations. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
14.1.10.2 Proposal Fails to Adequately Consider Public Health-Related Costs
Several comments express concerns that there will be an increase in public health-related costs as a result of the proposed rule due to: (1) the delay of necessary regulations to protect public health, (2) the inability to use the best available science when making regulatory decisions, and (3) because persons will choose not to participate in studies because of confidentiality concerns. The commenters state that these public health costs were not adequately considered in the development of the proposed rule. An overview of these comments is provided below.
14.1.10.2.1 Public Health-Related Costs as a Result of the Delay of Regulations Needed to Protect Public Health
Comment: Several commenters (3135, 5579, 6358, 6373, 6872, 6915, 6920, 8274) express concern that the added costs and efforts to comply with this rule may significantly delay the development of EPA regulations that improve public health. The commenters assert that the result of delaying the development of EPA regulations that improve public health will be the imposition of significant healthcare costs on American citizens. 
Commenters (0671, 6122, 6358, 6373, 6872, 6920, 8272, 8274) assert that this proposed rule does not include the foregone public health benefits that would result when regulations are delayed. Commenters (6358, 8272) state that the proposal should include a calculation of the foregone public-health benefits and associated costs to the U.S. economy when regulations are delayed due to this rule. 
Response: The EPA disagrees with commenters that requirements of this rule will result in any meaningful delay in promulgating regulations. Further, with this final rule, the EPA is maintaining language in 40 CFR 30.3 stating that the statutes EPA administers, or their implementing regulations, will control in the event of any conflicts with the requirements of this rule. The Agency will continue to comply with and abide by the requirements in those statutes and implementing regulations, including regulatory deadlines.
14.1.10.2.2 Health Costs Because of the Inability to Use Studies that Represent the Best Available Science
Comment: Commenters (2336, 6125, 6137, 8281-PH63) express concern that the costs to public health from implementation of this rule could be significant due to the inability of the EPA to use the "best available science" under the proposed rule which could result in a weakening of regulations. Commenter (2336) explains, for example, if the "six cities" and "150 cities" studies had been rejected, and, consequently, the NAAQS for PM2.5 had not been tightened in 1997, the harm to public health would have amounted to several hundred billion dollars. 
Commenter (6125) states that the CBO analysis does not tell the whole story as it did not assess the disbenefits to the regulatory process and to public health of being unable to base regulations on numerous influential studies for which data could not be made available. The commenter asserts that considering only the science policy value of the information in multiple original studies that are likely to be lost (e.g. cohort epidemiology studies, which must limit access to protect privacy), it is reasonable to conclude that their loss would almost certainly outweigh the value of any information gained by subsequent re-analyses of a more limited set of studies, which rely on publicly available databases that are often inferior to those that contain more relevant information.
Commenter (6137) states that, in a recent report on rulemakings related to health, EPA and OMB quantified some of the benefits of various regulations that would be dramatically weakened under the new rule. The commenter states that these recent studies show health rulemakings, particularly air rulemakings, create significant benefits (including health and protection of life, reductions in the need for health care and health care costs, as well as job creation, and other economic values in avoiding days lost at work and school) that are quantitatively larger and more qualitatively valuable than the costs of pollution controls or other economic costs of the regulations.
Commenters (5021, 6163, 6185, 6188, 6916, 8281-PH63, 9227) express concern that the EPA offers no accounting of the public health costs that would inevitably result if EPA is constrained in its ability to use the best research to support its future decision-making.
Response: The EPA disagrees with commenters that requirements of this rule will result in any meaningful delay in promulgating regulations. The final rule states that the EPA will consider all appropriate and relevant science consistent with the Agency's statutory obligations. Greater consideration will be given to pivotal science where dose-response data are available in a manner sufficient for independent validation. The EPA is maintaining language in 40 CFR 30.3 stating that the statutes EPA administers, or their implementing regulations, will control in the event of any conflicts with the requirements of this rule. The Agency will continue to comply with and abide by the requirements in those statutes and implementing regulations, including regulatory deadlines.
14.1.10.2.3 Public Health-Related Costs Associated with Reduced Participation of Persons in Studies Due to Confidentiality Concerns
Comment: Commenter (9227) states that the proposed rule does not disclose the cost that would be associated with the resulting reduced study participation due to perceived risks to privacy and confidentiality. The commenter states that NAS explains that this "threatens the research enterprise itself, because concerns about privacy and confidentiality are among the reasons often given by potential respondents for refusing to participate in surveys, and those concerns have been shown to affect behavior as well. , The commenter provides that the NAS panel emphasized: "Any confidentiality breach that became known would be likely to heighten such concerns and, correspondingly, reduce survey response rates." According to the commenter, efforts to increase researchers' access to data must, therefore, take into account the need to avoid increasing the actual and perceived risks of confidentiality breaches. The commenter states that the proposal does not take into account this potential cost and that this confidentiality risk has a further cost: it affects the quality of the data collected, as the NAS explained: 
      The reason for confidentiality pledges and for stringent procedures to prevent disclosure is that they improve the quality of data collected from individuals, households, and firms. It is essential that respondents believe they can provide accurate, complete information without any fear that the information will be disclosed inappropriately. Indeed, if the information was disclosed, harm might come to an individual respondent.
Commenter (9227) asserts that the proposal's only acknowledgment of this complex problem and cost is its statement that "EPA believes that concerns about access to confidential or private information can, in many cases, be addressed through the application of solutions commonly in use across some parts of the Federal government." The commenter states that EPA does not cite a single example of these common solutions, nor does it include an analysis of these alleged solutions and their costs and benefits, it does not even explain what the solutions are that EPA believes address this concern.
Response: The rule does not require EPA or any party to disseminate scientific data, nor does the rule does not require any action by any party other than the EPA. EPA anticipates that researchers will abide by any agreements regarding confidentiality that they have made with participants or other parties. The EPA recognizes that researchers and other third parties may voluntarily consider the EPA's position on data and model availability, as described in this rule, as they make their own decisions about how to conduct research and the extent to which they make dose-response data available, weighing their own benefits and costs and make choices that they deem appropriate. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
14.1.10.3 The Rule May Be Prohibitively Expensive
Comment: Commenters (1433, 2249, 6111, 6126, 6464) express concern that the proposal would be unrealistically expensive.
Commenter (6126) states that EPA will not likely have the resources or capability to assess the entire suite of regulations and the evidence-base retrospectively. The commenter states that, for example, EPA is technically required to evaluate every regulation promulgated under the Resource Conservation and Recovery Act (RCRA) on a triennial basis; recent research suggests that statutory requirement has been largely unfulfilled due to a lack of resources to comply with the decades-old mandate. The commenter states that, in the absence of a major influx of new resources, EPA would be unlikely to engage in such meaningful and transparent reviews while also complying with statutory obligations. The commenter recommends that, if implemented as stated, the regulation should be applied prospectively. 
Several commenters (0061, 1102, 1395, 1433, 2171, 2249, 2336, 2787, 2906, 2907, 4840, 4845, 4878, 5022, 6103, 6114, 6116, 6122, 6125, 6132, 6133, 6137, 6144, 6158, 6185, 6188, 6196, 6450, 6464, 6937, 6915, 8274, 9227) express concern over the potential cost to implement the proposal. The commenters point out the HONEST Act CBO ,  and EPA estimates of similar measures introduced in Congress indicate that the rule could cost millions of dollars a year to implement. 
Commenters (0061, 1012, 1433, 2336, 2787, 2906, 2907, 4840, 4845, 5022, 6114, 6116, 6122, 6132, 6133, 6137, 6158, 6196, 6464, 6937, 8274) state the HONEST Act CBO indicates that the proposed rule could cost $250 million initially to implement, and $1 million to $100 million (depending on the approach taken by EPA in assessing studies) per year thereafter; at an estimated cost of $10,000-$30,000 per study, but some studies may cost closer to $1 million. Commenters (2907, 6133) state that when EPA staffers in 2017 considered the potential effects of the HONEST Act that mirrors the approach of the proposal, they calculated  that efforts to anonymize health data and CBI could top $1 million per study. 
Commenter (9227) states that the CBO concluded, just with respect to the costs to EPA, that "based on information from the EPA and other federal agencies, as well as organizations and researchers in the scientific community that publish in peer-reviewed journals," EPA "could spend between a few million dollars per year to more than one hundred million dollars per year ... to ensure that data and other information underlying studies are publicly available in a format sufficient to allow others to substantially reproduce the results of studies." The commenter states that, in the 2017 estimate, the CBO concluded that "[i]f the EPA continued to rely on as many scientific studies as it has used in recent years ... then the CBO estimates that the agency would need to spend at least $100 million dollars per year to upgrade the format and availability of those studies' data," "on average, $10,000 per scientific study." According to the commenter, such costs would cover the costs of "obtaining all the underlying data used in a study, reviewing the data to address any confidentiality concerns, formatting the data for public access, providing access to the computer codes and models used in the study's analysis, and providing descriptions and documentation on how to access the data." The commenter states that, as Deputy Assistant Administrator Nancy Beck noted, during the development of the proposal, requiring "a huge amount of data to be submitted to the agency" would "be incredibly burdensome" and "not practical."
Response: The cost analyses referred to by the commenter are not analyses of this rule and assume costs of disseminating information. The rule does not require EPA or any party to disseminate scientific data and incur associated costs. The rule does require EPA to determine if dose-response data for pivotal science are publicly available in a manner sufficient for independent validation and the EPA acknowledges that the Agency will incur some administrative costs associated with the rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
14.1.10.4 Costs of Replicating, Reproducing, or Reanalyzing a Study (§30.5)
Comment: Commenters (1012,4848, 4933, 5015, 6196, 6373, 6925, 9227) express concern that replicating, reproducing, or reanalyzing a study (especially epidemiological studies) is a highly cost-intensive and time-consuming endeavor and resources spent trying to recreate a study identically expends time and resources that might otherwise be used to conduct new research.
Commenter (4848) states that timing and the cuts in EPA funding makes replicating studies (as a condition of promulgating regulations) an impossibility. Commenter (4933) states that the very nature of longitudinal public health studies where health and toxins intersect are by design, large, expensive and require years or decades before results are found. Commenter (4933) states that because these studies are so big, they are often too expensive to repeat. 
Commenter (9227) asserts that it is critically important to note that reanalysis projects are not simple or inexpensive. The commenter states that the reanalysis of just the Harvard Six Cities Study and the ACS CPSII took three years to complete and cost $899,046 in direct expenditures, without accounting for costs incurred by Health Effects Institute for oversight and review as well as staff compensation.
Response: The rule does not require EPA nor any other party to perform or conduct any reanalysis or replication of a study. The rule does require EPA to determine if dose-response data for pivotal science are publicly available in a manner sufficient for independent validation and the EPA acknowledges that the Agency will incur some administrative costs associated with the rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
Comment: Commenter (6941) expresses concern that, under the proposed rule, chemical manufacturers or importers could be forced to incur the cost of unnecessarily repeating the use of vertebrate animals in the testing of chemical substances. The commenter states that the Toxic Substances Control Act (TSCA) directs EPA to reduce and replace the use of vertebrate animals in the testing of chemical substances to the extent practicable, scientifically justified, and consistent with its policies. According to the commenter, TSCA identifies the formation of industry consortia to develop information as one means to advance this goal by reducing duplicative animal testing. The commenter states that, for some chemical substances, testing may already have been conducted under other regulatory initiatives, such as the European Union's Registration, Evaluation, Authorization and Restriction of Chemicals program. The commenter asserts that, while full study data may not be publicly available, robust summaries of the objectives, methods, results and conclusions of such testing can provide sufficient information to independently assess its value. 
Response: The rule does not require EPA nor any other party to perform or conduct any reanalysis or replication of a study. Further, with this final rule, the EPA is maintaining language in 40 CFR 30.3 stating that the statutes EPA administers, or their implementing regulations, will control in the event of any conflicts with the requirements of this rule. The Agency will continue to comply with and abide by the requirements in those statutes and implementing regulations.
14.1.10.5 Costs to Describe Assumptions, Methods, Variability and Uncertainty (§30.6)
Comment: Commenter (6125) states that the most puzzling, and perhaps costly requirement in the rule (§30.6) appears only in the wording of a specific requirement in the actual proposed rule regarding use of dose-response data and models from pivotal studies: "EPA shall clearly explain the scientific basis for each model assumption used and present analyses showing the sensitivity of the modeled results to alternative assumptions." The commenter expresses that this appears to require the agency not only to explain the basis of each model-related assumption in the original study but also to provide an analysis of the original data using alternative assumptions. The commenter states that EPA is not staffed to reanalyze the number of pivotal studies that could reasonably arise for all its significant rules under all statutes, and EPA provides no reason to believe that staffing levels would be adequate. Therefore, the commenter suggests that the Agency might need to enlist external analysts, which would take substantial time and money. The commenter states that, as a point of comparison, the reanalysis of the ACS and The Harvard Six Cities Study, which included analysis of alternative model specification, cost HEI approximately one million dollars. According to the commenter, if just half of recent cohort studies were to submit their data through some limited use agreement, it could cost EPA on the order of $10 million to hire outside analysts to do the work. The commenter states that, if this level of analysis were required for even one or two studies in every major regulation, it would likely impair significantly the Agency's ability to meet regulatory deadlines for the NAAQS or other programs, among other things precluding the Agency from meeting its avowed goal of completing NAAQS reviews for PM and Ozone by 2020. 
Response: The EPA has removed proposed 40 CFR 30.6 and transferred some of the requirements to 40 CFR 30.5(e). The EPA has revised requirements and limited their application to dose-response assessments. The final rule requires EPA to describe and document only critical assumptions and methods and to characterize variability and uncertainty of the assessment. It requires an explanation of the scientific basis for critical assumptions used in the dose-response assessment. These changes from the proposed requirements result in only small incremental costs for the EPA. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
14.1.10.6 Costs of Independent Peer Review (§ 30.7)
Comment: Commenters (1012, 4839, 4931, 6125, 6142, 6866, 6868, 6915) express concern that having the Agency conduct an "independent peer review on all pivotal regulatory science used to justify regulatory decisions" would be time-consuming, duplicative of current practice and costly to implement, without adding commensurate environmental and public health value. Commenter (6125) states that independent peer review is a cost and impact which the Agency has failed to acknowledge or account for. 
Commenter (1012) states that having the Agency conduct an "independent peer review on all pivotal regulatory science used to justify regulatory decisions" would likely hamstring the Agency if additional resources and staff are not provided for this express purpose. Commenter (6866) asserts that implementing independent Agency peer review of all such studies would significantly slow down rulemaking and may invite lawsuits that further stymie regulatory processes. 
Commenter (6142) states that there is no value in this second peer review without guidelines ensuring the review is rigorous and unbiased in nature and conducted by subject matter experts. The commenter states that this rule proposes adding this costly step to the process for developing EPA policy without setting aside resources for its execution. According to the commenter, the rule is likely to hinder the application of potentially important scientific findings to public and environmental health without improving the quality of data used to guide EPA regulatory policy.
Commenter (6915) states the proposal nowhere discusses how EPA would vet reviewers to identify persons who are purportedly more competent than those already used in past or current peer review processes, or the level of EPA staffing and associated costs that would be needed for additional review -- only stating that EPA will implement the proposed rule in a manner "that minimizes costs." The commenter states that the practical outcome of the proposal is that EPA may end up relying on a much smaller number of studies and/or on a less robust subset of relevant available studies, thus undermining the regulatory decision-making process. According to the commenter, the EPA fails to acknowledge the rigor of existing processes in statutes, policies and federal procedures, or to explain how its proposal would provide any added value and minimize costs. 
Response: This rule requires that the EPA shall conduct independent peer review consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review and the exemptions described therein. The EPA will evaluate whether or not to conduct additional peer review of individual studies identified as pivotal science if the studies have already undergone peer review. The EPA anticipates that the incremental peer review costs to the Agency will be small. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
14.1.10.7 Do Benefits Justify the Costs of the Rule?
14.1.10.7.1 EPA Needs to Quantify the Costs and Benefits of the Rule
Comment: Commenters (1012, 2171, 2787, 5022, 6046, 6125, 6137, 6143, 6168, 6196, 6358, 6874, 6925, 8272, 9227) assert that, while EPA claims benefits outweigh costs, there is no published record or analysis to demonstrate that this claim is valid and that the EPA should commit to quantify these costs and benefits (including foregone health benefits) and implement the rule in a manner that optimizes the costs and benefits. 
Commenter (2787) states that EPA only stated that the Agency will implement the provisions "in a manner that minimizes cost." Commenter (6046) states that it is unclear who bears the burden of the cost and who benefits, by how much and how. Commenter (6137) states that, despite the lack of evidence in the regulatory docket, the costs of this Proposed Rule are far-reaching and substantial, while EPA has failed to identify any alleged benefits at all, other than vague and unsupported references to improved transparency. Commenter (6874) states that the theoretical  -  but undemonstrated  -  benefits of EPA's proposal must be weighed against the extensive and unequally distributed costs of such an approach. 
Commenters (6916, 9227) argue that EPA makes unsupported statements about its "belie[f that] the benefits of the proposed rule justify the costs." Commenters (6916, 9227) assert that EPA's finding is not consistent with the conclusions of the National Academies, as the proposal suggests. Commenter (9227) states that the NAS report highlighted both the risks and benefits of making data publicly available and nowhere concluded that there were benefits to excluding data from the agency's regulatory decisions simply because the underlying data was not publicly available. Commenter (6916) states that the NRC explicitly states that its work does not "weigh the potential harm posed by disclosure against the benefits potentially foregone."
Commenter (9227) states that EPA has included only a cursory mention of the expected qualitative benefits of the proposal, with no discussion of the anticipated likelihood, scope, or impact of the suggested benefits, let alone any effort to quantify them, much less monetize them. Commenter states that EPA simply assumes that the proposal will "improve the data and scientific quality of the Agency's actions and facilitate expanded data sharing an exploration of key data sets" without any analysis or evidence. 
Commenter (6137) notes that, in performing certain CAA and other rulemakings, EPA may not consider the economic implications of considering or excluding certain science. The commenter asserts that EPA's failure to provide any lawful or rational justification for its costs/benefits statement for the proposed rule suggests that economic costs may have impermissibly played a role in its analysis. According to the commenter, had EPA focused on health rather than cost, EPA could not possibly find that the benefits of ignoring health science outweigh the costs for public health rulemakings. 
Commenter (9227) states that there is no reason to think that excluding relevant science merely because the underlying data is not publicly available would increase the net benefits of a regulation. According to the commenter, for example, it appears that under the proposed rule EPA would exclude a peer-reviewed, published study whose conclusion had been reproduced based upon numerous different datasets (and whose underlying data, though not publicly available, had been reevaluated by outside experts), while including a study that had had no peer review, was not published, had no corroborating studies, and had not actually been replicated or reproduced, merely because the underlying data was made publicly available. The commenter asserts that is simply not a recipe for more accurate decision making.
Response: The EPA agrees that the benefits and costs of the rule were not fully characterized in the 2018 proposed rule or the 2020 SNPRM and has described these briefly in the final rule. The EPA acknowledges that the Agency will incur some administrative costs associated with the rule, but the EPA finds that these costs are likely to be small given that the additional requirements are limited to pivotal science, and that EPA already conducts extensive review of the science when promulgating regulations or publishing influential scientific information. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
Comment: Commenter (0671) states that the proposed rule will impose compliance costs without providing any meaningful benefit to the EPA, to the regulatory process, or to the American people. Commenters (6450, 8272) argue that, considering there is no evidence that current data transparency policies are deficient, this added cost to the American taxpayer is neither necessary nor appropriate.
Response: The EPA acknowledges that the Agency will incur some administrative costs associated with the rule, but the EPA finds that these costs are likely to be small given that the additional requirements are limited to pivotal science, and that EPA already conducts extensive review of the science when promulgating regulations or publishing influential scientific information. The EPA believes that greater transparency is inherently valuable as a principle of good government and provides benefits to the public at large, including reducing the risk of errors in EPA analyses and in the science these analyses rely upon. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
Comment: Commenter (1346) states that the rule talks about the costs of compliance but it does not talk about the benefits of compliance appropriately. The commenter asserts that, for example, the carbon rules have a cost to power companies but a clearly defined benefit to the health of human beings living near power plants. According to the commenter, it is more ethical to weigh the benefits to the lives and well-being of people higher than the benefits to a company.
Response: The Agency agrees that there are important and valuable health benefits for many regulations. The EPA follows its own peer-reviewed guidance, as well as government-wide guidance from OMB, when estimating both benefits and costs of regulations.
14.1.10.7.2 Benefits of the Strengthening Transparency Proposal
Comment: Commenter (3971) states that, if rigorous and transparent analyses underlie future proposals, there should be fewer legal challenges, and passing this regulation should permit the Agency to implement rules more quickly, not less expeditiously. 
Commenter (4534) states that increasing transparency by using peer reviewed data and publishing data for public review may increase the time and cost to bring a new or modified rule package into practice. The commenter states that the additional time and cost to develop a new regulation is small compared to the time and cost for regulated parties to comply with the regulation. The commenter asserts that many of the rule packages cost industry more than $100 million to comply and require several years to develop engineering solutions. According to the commenter, the EPA and industry both have incentive for the rule package to be right the first time. 
Response: The EPA agrees that rigorous and transparent analysis has significant value and improves the regulatory process. EPA also agrees that the incremental time and costs associated with implementing the requirements of this rule are small. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
14.1.10.7.3 EPA Must Re-Propose After Analyzing its Costs
Comment: Commenter (9227) states that the proposal's failure to analyze and disclose costs and benefits cannot be cured in a final regulation. The commenter states that, should EPA not abandon this misguided proposal, it must re-propose it after first analyzing its costs (both to public health, to researchers, and to the agency itself) and benefits, and providing the requisite opportunity for public comment on its analysis. According to the commenter, the public cannot meaningfully comment on the proposed rule without understanding the actual costs and benefits of the proposal, the alternatives EPA considered, and the analyses underlying EPA's assessments.
Response: The EPA agrees that the benefits and costs of the rule were not fully characterized in the 2018 proposed rule or the 2020 SNPRM. However, in the final rule the EPA has identified and described the benefits of the rule and some of the incremental costs that the Agency may incur as a result of the rule. The EPA disagrees that it must re-propose this rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
14.1.10.8 Minimization of Subpart Implementation Costs (§30.8)
14.1.10.8.1 What Costs Will Be Minimized and How?
Comment: In order to achieve the goals of the proposal in a streamlined and cost-effective manner, commenter (1941) recommends the following:
 EPA should develop standardized data-sharing and data-security agreements (including non-disclosure agreements) to utilize with the authors/investigators of pivotal studies to facilitate and streamline the receipt and transfer of study data to the RDC or to EPA's other approved datahousing site(s);
 EPA should develop standardized data-access and confidentiality agreements that third-party investigators can use to gain authorized access to the pivotal-study data that will be housed at the RDC or at other EPA-sanctioned data-housing site(s); and 
 EPA should work to ensure that any fees for gaining authorized access to pivotal study data at EPA's authorized data-housing site(s) are kept to a minimum to enhance the prospects that qualified third-party investigators will be able to conduct their reviews and reanalyses in a timely and cost-effective manner.
Commenters (6125, 6373) express concern that, while the proposal states that costs will be low because EPA will "implement" the rule "in a manner that minimizes costs", there is no explanation of what specific types of costs may be incurred, much less how they will be minimized. Commenter (6125) states that this type of vague and unsupported pronouncement does not provide an adequate basis for public comment. 
Commenter (6158) states that, in suggesting that the potential costs of the rule would be minimal, the EPA ignores the costs to researchers (who would have to pay to set up and maintain data sharing for their previously published studies to be considered), to the EPA (for presumably conducting the multiple re-analyses required in section 30.6 of the rule), and to the public's health (for the very real costs in dollars and human lives of undermining existing regulations by limiting access to science).  
Response: The EPA agrees that neither the 2018 proposed rule nor the 2020 SNPRM included a characterization of costs or benefits. In the final rule the EPA has identified and described the benefits of the rule and some of the incremental costs that the Agency may incur as a result of the rule. The rule does not require any action by any party other than the EPA and imposes no costs on other parties. The rule also does not require EPA or any party to disseminate scientific data and incur associated costs. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
14.1.10.8.2 Will EPA Consider Fewer Studies?
Comment: Commenters (2336, 5021, 6125, 6137, 6158, 6163, 6185, 6358, 6874, 6916, 8272, 9227) generally express concern that the cost of this rule will be less science  --  not better science  --  and weaker public and environmental health decision making. 
Commenters (2336, 4840, 4845, 6109, 6114, 6116, 6122, 6137, 6181, 6188, 6196, 6915) express concern that the proposal does not address the possibility that EPA would reduce the number of studies used in developing policies and regulations because of the cost of complying with this rule's requirements. Commenter (6181) states that, if the EPA restricts the science it relies upon to only those studies that publish open datasets, the outcomes and data from key areas of scientific inquiry related to EPA's decision-making will be put beyond the reach of Agency staff. 
Commenters (2336, 4840, 6109, 6133) assert that CBO analysis suggests that EPA may rely on significantly fewer studies each year in support of its mission. Commenters (4840, 9227) state that the cost estimate made by the current administration for a substantially similar law dropped to $1 million annually because EPA officials explained to the CBO that the Agency would implement the legislation with minimal funding and generally would not disseminate information for the scientific studies that it uses to support covered actions. 
Commenters (6114, 6116) state that an EPA staff study predicted that this policy would reduce by half the number of studies the EPA would rely upon in developing policies and regulations because of the cost of complying with its requirements. 
Commenter (6137) states that the exclusionary intent of the Proposed Rule is demonstrated by proposed 40 CFR §30.8, which requires that the rule be implemented to "minimize costs." The commenter suggests that, if EPA must minimize costs, the exclusion of science, rather than taking complicated and expensive steps to hide confidential medical data, is likely to be the approach followed.
Response: The final rule EPA will consider all appropriate and relevant science consistent with the Agency's statutory obligations. Greater consideration will be given to pivotal science where dose-response data are available in a manner sufficient for independent validation. The EPA is maintaining language in 40 CFR 30.3 stating that the statutes EPA administers, or their implementing regulations, will control in the event of any conflicts with the requirements of this rule. The Agency will continue to comply with and abide by the requirements in those statutes and implementing regulations, including regulatory deadlines.
Comment: Commenter (2478) requests that the EPA quantify how much this proposed rulemaking would result in a reduction in sponsored research in climate science. The commenter inquires how the EPA defines climate science, how much funding is allocated from the federal government to climate science, and how much will this proposed rule impact funding allocated from the federal government to climate science?
Response: Funding for climate science varies and EPA does not have at its disposal total funding across the federal government. The EPA does not foresee any meaningful connection between this rule and funding allocated from the federal government to climate science.
14.1.10.8.3 Regulatory Decisions on Cost Inconsistent with Statutory Mandates
Comment: Commenter (6137) states that proposed Section 30.8, which would require EPA to implement the provisions of the Rule "in a manner that minimizes cost," is unlawful and arbitrary, for the following reasons.
      First, because EPA has only the authority conferred to it by statute, the Agency must identify the statutory authority upon which it bases its regulatory decisions about what data and models to consider on the minimization of cost. Clean Air Council v. Pruitt, 862 F.3d 1, 9 (D.C. Cir. 2017) (quoting Verizon v. FCC, 740 F.3d 623, 632 (D.C. Cir. 2014) ("It is `axiomatic' that `administrative agencies may act only pursuant to authority delegated to them by Congress.'") (alteration and citations omitted)). The Agency has failed to do so here. 
       Second, EPA's proposal to base these decisions on cost is inconsistent with statutory mandates requiring the Agency to base regulatory decisions on the best available science. See, e.g., 42 U.S.C. § 300g-1(b)(1)(B)(ii)(II) (directing EPA to base its determination about whether to regulate any contaminant "on the best available public health information"); 15 U.S.C. § 2625(h) (requiring EPA to act "consistent with the best available science"). Plain meaning and common usage confirm that "best available science" does not refer to whatever science is least costly. See also supra, Section IV.A.
      Third, EPA's proposal to base its data quality decisions on cost is inconsistent with applicable statutory mandates requiring EPA to make regulatory decisions based on considerations of public health. Whitman v. Am. Trucking Ass'ns, 531 U.S. 457, 469 (2001) (cost "is both so indirectly related to public health and so full of potential for canceling the conclusions drawn from direct health effects that it would surely have been expressly mentioned . . . had Congress meant it to be considered."). If anything, EPA's proposal to base decisions on consideration of cost here is even more irrational than it was in Whitman, because in this case, decision-making about the science used to inform EPA's regulatory decisions will result in a proliferation of costs and benefits that resist quantification, monetization, and comparison. These include substantial and important non-market values, including privacy, transparency, health and environmental considerations, and even the value of scientific knowledge itself. 
      Fourth, section 30.8 is also unlawfully vague. As the Supreme Court has explained, "`cost' includes more than the expense of complying with regulations; any disadvantage could be termed a cost." Michigan v. EPA, 135 S. Ct. at 2707. Yet the provision does not specify to what types of costs it refers. And even though various types of costs would necessarily need to be weighed against one another when applying this provision, the Proposed Rule is notably silent as to how that will be done. As proposed, it appears to confer nearly limitless discretion on EPA to decline to collect and utilize relevant science on grounds of cost, in contravention of applicable statutory mandates and reasoned decision-making. See U.S. Sugar v. EPA, 830 F.3d 579, 644 (D.C. Cir. 2016) ("in light of the unambiguous statutory command [to regulate toxic pollution sources] . . . [t]he Agency was obligated to collect the data it needed"). 
      Finally, even if EPA had authority to base its data quality decisions on cost, it would be irrational and arbitrary for EPA to ignore all the costs and benefits to the public  -  including public health and environmental costs and benefits and privacy-related costs and benefits  - that may flow from its decisions. EPA must treat costs and benefits alike and may not "put a thumb on the scale by undervaluing the benefits and overvaluing the costs of more stringent standards." Ctr. for Biological Diversity v. Nat'l Highway Traffic Safety Admin., 538 F.3d 1172, 1198 (9th Cir. 2008). And it may not ignore the public health, environmental, and privacy-related costs of its action or inaction of its decision just because they are not quantified. See, e.g., Exec. Order No. 13,563 § 1, 76 Fed. Reg. at 3821; Exec. Order No. 12,866 § 1, 58 Fed. Reg. at 51,735 (it is "essential to consider" the "qualitative measures of costs and benefits that are difficult to quantify"). OMB Circular A-4 cautions agencies against ignoring the potential magnitude of unquantified benefits, because the approach with the "largest quantified and monetized . . . estimate" is not necessarily the most cost-justified. Under all these authorities, a "full accounting" of the costs and benefits of a rule requires that indirect benefits be counted "equivalently" with other costs and benefits. 
Response: The EPA acknowledges that the Agency will incur some administrative costs associated with the rule and will implement the rule in a manner that minimizes costs while fully complying with the rule EPA's other statutory obligations. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule. The EPA will continue its current practice of conducting extensive review of scientific studies during the development of significant regulatory actions and influential scientific information. The final rule EPA will consider all appropriate and relevant science consistent with the Agency's statutory obligations. Greater consideration will be given to pivotal science where dose-response data are available in a manner sufficient for independent validation. The EPA is maintaining language in 40 CFR 30.3 stating that the statutes EPA administers, or their implementing regulations, will control in the event of any conflicts with the requirements of this rule.
14.1.10.9 Costs May Lead to Study Selection Biases
Comment: Commenters (6168, 6446, 6924, 9227) are concerned that the implementation costs of this rule may lead to selection biases regarding which studies EPA will include in its analyses. 
Commenter (9227) states that, at a time when federal funding for research in environmental and public health-related fields has largely flat-lined, academic researchers are likely to have few additional funds available to undertake these activities. Commenters (6168, 6446, 6924) assert that, the lack of any commitment by EPA to fund implementation of procedures to protect privacy, confidentiality, security and other interests would privilege industry science, where funds would be available to comply with the proposed rule's requirements and could result in the exclusion of robust academic and other science from EPA's consideration only because of insufficient funds to comply with the proposed rule's onerous and unnecessary requirements. Commenter (6924) states that this would create a situation in which data EPA considers in decisions would be heavily skewed towards industry studies  -  and it is well documented that financial sponsorship (i.e. source of funding) introduces a risk of bias in the results and conclusions in favor of the regulated industry's interests., , ,  The commenter asserts that an influx of industry-funded studies with financial conflicts of interest would likely bias the final results of EPA's analysis, leading to less stringent regulations and policies than with consideration of unbiased studies -- with the ultimate result of reducing protections for the health of families. The commenter expresses concern that EPA's proposed approach could institutionalize a dangerous bias in the source of studies that EPA can use for regulatory activity.
Response: Under the final rule EPA will consider all appropriate and relevant science consistent with the Agency's statutory obligations. All such studies will be subject to extensive review for scientific quality. Though greater consideration will be given to pivotal science where dose-response data are available in a manner sufficient for independent validation, the Agency does not agree that this necessarily institutionalizes biases in the source of studies or would reduce protections for the health of families. The EPA notes that it is maintaining language in 40 CFR 30.3 stating that the statutes EPA administers, or their implementing regulations, will control in the event of any conflicts with the requirements of this rule. The Agency will continue to comply with and abide by
14.2 SNPRM Implementation
 14.2.1 Inter-organizational Efforts
Comment: Commenter (11479) points out that other federal and state agencies are involved in EPA's decision-making, yet there is little detail about how this rule would impact that coordination. Consequently, the commenter suggests that there is the possibility that this rule would hamper EPA's ability to quickly and effectively coordinate with other agencies during a disease outbreak, such as COVID-19. For example, the commenter notes that key studies which rely on confidential medical information may be excluded, "derailing any interagency effort to incorporate the study's findings into a federal pandemic response. In other words, the commenter contends that the EPA's proposed rule could, in times of crisis, slow or stifle life-saving discoveries in public health."
Commenter (12727) states that the SNPRM expansion of the 2018 proposal, has inadvertently and irrationally hampered its ability to cooperate with other agencies in conducting joint or coordinated rulemakings, beyond the examples noted previously. The commenter references the finalized emission standards for greenhouse gases from light-duty vehicles, which EPA intended to harmonize with the National Highway Traffic Safety Administration's (NHTSA) corporate average fuel economy standards (CAFE). The commenter notes that both sets of standards are based on scientific analysis for which underlying data or models are not available, and that EPA could not have relied on this analysis in setting greenhouse gas standards had the proposal's requirements been in place, even if the analysis did not alter the level of the standards. The commenter conceives that there would be instances in which discrepancies in the rulemaking records considered by EPA and NHTSA, respectively, would result in different standards and that it would be bizarre if, going forward, EPA and NHTSA were to attempt another harmonized rulemaking and be prevented from doing so by the fact that NHTSA relied on studies without publicly available underlying data or models while EPA would not.
Relatedly, the commenter (12727) contends that the proposal could impede any EPA rulemaking, or joint rulemaking, for which the National Environmental Policy Act (NEPA) requires an environmental impact statement (EIS). The commenter contends that the Safer Affordable Fuel-Efficient (SAFE) Vehicles Rule is a case in point. The commenter asserts that there, NHTSA prepared an EIS examining the emissions, air quality, and health impacts of weakening fuel economy standards, with EPA as a cooperating Agency in the preparation of the EIS, where NHTSA averred that it prepared the EIS "[t]o inform its development of the final CAFE standards." Thus, to the extent that EPA's Greenhouse Gas (GHG) standards in the SAFE Vehicles Rule conform to NHTSA's CAFE standards in the same joint rule, the commenter notes that the studies considered in the EIS also qualify as "pivotal regulatory science" that "drive[s] the requirements and/or quantitative analysis of [an] EPA final significant regulatory decision." The commenter states that NEPA requires the Agency to take a "hard look" at the relevant data and, in some cases, gather more data, and at the very least, the Agency must explain why data it omitted "would not alter its conclusions in the EIS or the approval of [the action]." According to the commenter, the SNPRM here would summarily rule out data that NEPA requires the Agency to include and consider in an EIS.
Response: The EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. Some of these subsequent actions will be substantive rules issued under the associated environmental statutes and will be subject to any applicable procedural and analytical requirements under those statutes and judicial review.
14.2.2 The EPA's Readiness to Implement the Rule
Comment: Commenter (10974) assert that there is no evidence that EPA is prepared to implement this rule. The commenter states that the SNPRM continues to leave a void around the process for implementation, thereby leaving their comments original 2018 unaddressed.
Response: This final rule is effective upon publication in the Federal Register. As outlined in 40 CFR 30.3, the provisions of the rule apply to significant regulatory actions for which a proposed rule was published in the Federal Register after the effective date of the rule and influential scientific information submitted for peer review after the effective date of the rule. The implementation guidelines are forthcoming. The comments on the 2018 were considered in the development of the SNPRM and all the comments were considered in the development of the final rule.
14.2.3 Implementation Timing/Process 
Comment: Commenter (12399) contends that Section 30.3's exclusions of drafts and preliminary analyses are too vague and may exclude exceptionally important research. The commenter provides that it might make sense to exclude preliminary analyses that are not publicly available, however, anything the Agency releases or discusses publicly as a possible justification for a regulatory standard or for educational purposes should meet the transparency rule. The commenter notes that draft risk assessments, for example, may have profound market impacts, including impacts on litigation, and hence, should not be exempt from the rule.
Response: EPA notes that this final rule does not categorically exclude studies. Section 30.3 of the final rule states, "The provisions of this part do not apply to physical objects (like laboratory samples), drafts, and preliminary analyses, and influential scientific information or pivotal science that meet one or more of the exemptions identified in Section IX of the OMB Final Information Quality Bulletin for Peer Review. In the event the procedures outlined in this part conflict with statutes the EPA administers, or their implementing regulations, the statutes and regulations will control."
Implementation guidance and statute-specific transparency regulations or programmatic regulations are forthcoming.
Comment: Commenter (11499, 12397) requests EPA clarify that the statement "at the time [a rule] ... is developed" means that EPA will apply the transparency rule to the proposed rulemaking stage, if not sooner, that is, data and models underlying any pivotal regulatory science or pivotal science that EPA might use in promulgating a significant regulatory decision or making influential scientific information final will be made available for analysis and comment in connection with the notice of proposed rulemaking. The commenter asserts that if this interpretation is incorrect, EPA should revise the rule to make clear that the proposed 40 C.F.R. Section 30.5 applies at the proposed rule stage as well as to the final Agency action.
The commenter (11499) states that, as noted in their comments on the 2018 proposal, the proposed rulemaking stage is when public stakeholders will need to weigh in on potential concerns related to the scientific basis for EPA's proposal and when efforts at independent validation of the relevant data might occur. The commenter contends that deferring public access to the "pivotal regulatory science" or "pivotal science" until the final rulemaking is issued is inappropriate, as it will be too late for active public engagement. The commenter also provides their concern that the statement could be interpreted as excluding subsequent reanalysis of the data used to develop a rule during the period between the proposed rule and the final rule. Therefore, the commenter requests that EPA include language explicitly addressing data reanalysis in its final rule. As currently proposed, the commenter provides that EPA has not specified how or when the public will be able to engage in the reanalysis of data or independent validation. The commenter notes that obtaining and reanalyzing data that are made public through this rule could take more time than has traditionally been provided to submit comments on a proposed rule and EPA must account for this step in the public comment process to allow meaningful engagement.
Response: The final rule states that the EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
Also, to assist the Administrator in determining whether to grant an exemption, the EPA program or Region responsible for the significant regulatory action or influential scientific information and public commenters can provide input when the Administrator is considering an exemption. The EPA will document the rationale for the Administrator's exemption in the significant regulatory action or influential scientific information.
This final rule applies prospectively to significant regulatory actions and influential scientific information and has no retrospective effect on existing significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data.
14.2.4 Implementation Costs and Benefits
Comment: Commenters (11192, 11354) contend that there are no analyses regarding costs, the types of research affected, or the existing rules that may be called into question when their underlying scientific basis is dismissed.
Response: The EPA agrees that neither the 2018 proposed rule nor the 2020 SNPRM included a characterization of costs to the Agency. The EPA emphasizes that this is a rule of internal procedure promulgated under the EPA's housekeeping authority. However, the EPA has identified some incremental costs that the Agency may incur as a result of this final rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
This internal procedural rule does not interpret or apply provisions of a particular statute or statutes that EPA administers. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. The final rule provides that if implementing the rule results in any conflict between this rule and the environmental statutes that the EPA administers, and their implementing regulations, this rule will yield and the statutes and regulations will be controlling.
14.2.4.1 Assessing Costs and Benefits
Comment: Commenters (11176, 11894) state that this rule should not be finalized without a full cost-benefit analysis. 
Comment (9372) notes that they do not yet know all of the costs of pollution and accompanying benefits of regulation. The commenter states that pollution is actually far more costly in lost years of life, health-related costs, human capital costs, lost wages (Borgschulte, Molitor and Zou, 2017; Isen, Rossin-Slater and Walker, 2017), lost tax revenue, and crime (see Feigenbaum and Muller, 2016; Aizer and Currie, 2017; Billings and Schnepel, 2017) than they ever guessed. The commenter contends that ignoring the indirect costs and benefits means that new research on the costs and benefits of pollution would not be taken into consideration, which could have disastrous effects on public policy. 
Commenter (11911) notes that purporting to comply with executive orders that require economic impact analysis of regulations, EPA claimed that "the benefits of [the Strengthening Transparency in Regulatory Science (STRS) Rule] justify the costs." The commenter contends that in reaching this conclusion, however, the Agency both drastically underestimated the costs of compliance with the Rule, and more importantly, neglects to consider how the "[e]xclusion of relevant studies... will adversely affect decision-making processes," imposing serious costs on the health and well-being of the American people.
The commenter (11911) states that for support of its conclusion, EPA cites to a single working paper issued by the Mercatus Center at George Mason University. The commenter notes that this paper estimates the economic impacts of a policy with a very different scope: it assumes that while 80 percent of studies will fail to meet transparency criteria, these studies will nevertheless continue to be relied upon in Agency rulemaking. According to the commenter, under the proposed STRS Rule, however, such studies would be excluded entirely from Agency consideration. According to the commenter, the Mercatus paper, therefore, fails to capture the STRS Rule's far-reaching costs.
The commenter (12430) provides that the purpose of regulatory impacts analyses -- to help the public and the Agency itself understand the costs and benefits of a significant rule -- is thwarted when impacts are not critically assessed, yet EPA did not estimate the increased resources necessary for the Agency to implement the SNPRM, even though, complying with the rule would considerably burden Agency resources. The commenter notes that EPA did not attempt to quantify the purported benefits of providing tiered access to restricted data and models or the profound impacts to Agency protection of public health and the environment. The commenter contends that not only does this failure violate Executive Order 12866, it precludes fully-informed public comment on this highly-consequential and far-reaching set of proposals.
Commenter (12727) asserts that the SNPRM would effect a major change in how EPA establishes public health protections without any consideration of the impacts or burdens that the policy would inflict upon the Agency, the research community, vulnerable populations, or the public at large. The commenter states that in comments on the 2018 proposal, they and others demonstrated that EPA failed to provide any assessment of the costs and benefits -- either quantitatively or qualitatively -- and thus fell gravely short of basic requirements for reasoned decision-making. The commenter contends that far from correcting that fatal shortcoming, the Supplemental Notice exacerbates the problem by conjuring an even more expansive policy from the same deficient record; the word "cost" appears nowhere in the Supplemental Notice, except in titles of two documents cited for other purposes.
According to the commenter (12727), it is arbitrary and capricious to "`entirely fai[l] to consider an important aspect of the problem' when deciding whether regulation is appropriate." The commenter provides that as in Michigan, EPA's present failure to consider the costs and benefits of a regulation where there is no statutory bar to doing so is arbitrary and capricious.
Furthermore, the commenter (12727) states that EPA's failure to characterize costs and benefits necessarily precludes the Agency from assessing whether the costs are disproportionate to the benefits, a finding that could render the proposal arbitrary and capricious. The commenter notes that this failure extends to harms that the proposal would inflict upon the public at large, vulnerable populations, the research community, and the Agency. The commenter asserts that EPA does not identify any statutory authority that could justify its failure to assess the costs and benefits of the proposed rule, and it is certain that the Agency could not rely on a radical use of a generally applicable statute like the Housekeeping Act in order to evade well-established requirements of administrative law.
Commenter (12731) states that an underlying assumption of the tiered access approach is that researchers involved in the development of pivotal science or pivotal regulatory science will participate. The commenter states that, however, the EPA has provided no analysis indicating the extent to which the Agency believes compliance could or would occur. The commenter states that, instead, it is reasonable to assume that many researchers would not engage in a tiered access approach for a variety of reasons, the significant anticipated costs being just one for which EPA has utterly failed to provide any analysis.
Response: The EPA agrees that neither the 2018 proposed rule nor the 2020 SNPRM included a characterization of costs to the Agency. The EPA emphasizes that this is a rule of internal procedure promulgated under the EPA's housekeeping authority. However, the EPA has identified some incremental costs that the Agency may incur as a result of this final rule. See III.J of the preamble for further discussion of benefits and costs of the rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
This rule exclusively pertains to the internal practices of the EPA and does not regulate the conduct or determine the rights or obligations of any entity outside the federal government.
14.2.4.1.1 Conducting Assessment at a Later Date
Comment: Commenter (12727) provides that EPA might analyze costs and benefits of future rulemakings that implement this policy is no substitute for analyzing costs now. The commenter contends that moreover, in future rulemakings, the methodology of this policy would already be baked into the process, and disentangling its specific impacts would be virtually impossible. The commenter notes that to do so, EPA would have to identify which studies it would have relied on (or weighted differently) but for this rule, the difference in the standards or other outcome that would have resulted, and the costs and benefits the counterfactual standards would have yielded, and even then, this rule would have been long since finalized, so any assessment of costs and benefits would come far too late to influence EPA's decision-making. The commenter also asserts that analyzing the costs of this rule only as-applied in future rulemakings would be unlikely to capture costs imposed on the research community or resulting from the public's reluctance to participate in studies as a result of this policy. The commenter provides that, as explained in their previous comments, the influential scientific information to which this proposal would apply may be utilized in processes other than EPA rulemakings -- such as to inform decisions by state and local health agencies -- that would not provide even a theoretical opportunity for EPA to assess the costs and benefits of this proposal.
Response: This is a rule of internal procedure promulgated under the EPA's housekeeping authority. The EPA has identified some incremental costs that the Agency may incur as a result of this final rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
14.2.4.1.2 Assessment is Necessary for Meaningful Public Participation
Comment: Commenter (11480) states that the SNPRM does not include any feasibility or cost analysis for either approach to data or the implementation of any portion of the proposal. The commenter provides that the proposal does not specify the means by which it will establish a system for tiered access, instead citing only the Research Data Center at the National Center for Health Statistics as a model.
According to the commenter (11480), the policies contained in this proposal, particularly if they include making massive amounts of raw data publicly available, may be very costly to the American people via the Agency's budget. The commenter contends that EPA may require those costs to be borne by researchers who (1) would be forced to account for these costs in budgeting for future research projects or (2) would be forced to find funding for this disclosure for current or completed studies. The commenter provides that EPA cannot and should not move forward with implementing a broad reaching policy in which roles and responsibilities are unclear and the costs and benefits have not been analyzed, disclosed, and available for public comment.
Response: This is a rule of internal procedure promulgated under the EPA's housekeeping authority. The EPA has identified some incremental costs that the Agency may incur as a result of this final rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
This rule exclusively pertains to the internal practices of the EPA and does not regulate the conduct or determine the rights or obligations of any entity outside the federal government.
Comment: Commenter (11894) states that EPA fails to provide any information on such basic matters as potential impacts of the expanded proposal under each of EPA's substantive statutes; the potential social costs in the form of decreased protection to public health; the economic costs (the tiered access option could be very costly); and any benefits of the supplemental proposal. The commenter contends that EPA does not discuss the need for and implications of fundamental changes in position from such things as EPA's own peer review guidance. According to the commenter, commenters can only guess as to each of these matters; it is not their responsibility to have to guess. The commenter maintains that EPA thus has failed to provide any notice of issues of fundamental import, and failed to provide adequate notice and opportunity for comment. According to the commenter, the Agency has also completely failed to meet the rulemaking standards required under FCC v. Fox Television, 556 U.S. 502 (2009). The commenter notes that these errors deprive commenters of the most basic information needed to formulate reasoned comments and so are prejudicial.
Response: The EPA agrees that neither the 2018 proposed rule nor the 2020 SNPRM included a characterization of costs to the Agency. The EPA emphasizes that this is a rule of internal procedure promulgated under the EPA's housekeeping authority. However, the EPA has identified some incremental costs that the Agency may incur as a result of this final rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
This rule exclusively pertains to the internal practices of the EPA and does not regulate the conduct or determine the rights or obligations of any entity outside the federal government.
Comment: Commenter (14397) contends that EPA does not appear to have conducted a cost-benefit analysis on either the original proposal or the supplement, nor has a Regulatory Impact Assessment been done. According to the commenter, the lack of proper analysis leaves commenters, and the EPA itself, in the dark as to what the results of this action will be.
Response: The EPA agrees that neither the 2018 proposed rule nor the 2020 SNPRM included a characterization of costs to the Agency. The EPA emphasizes that this is a rule of internal procedure promulgated under the EPA's housekeeping authority. However, the EPA has identified some incremental costs that the Agency may incur as a result of this final rule. See Section III.J of the final rule preamble for further discussion of benefits and costs of the rule.
This rule exclusively pertains to the internal practices of the EPA and does not regulate the conduct or determine the rights or obligations of any entity outside the federal government.
14.2.4.2 Costs to the Agency
Comment: Several commenters (11350, 11395, 11408, 11478, 11484, 11492, 12715, 12405, 12422, 12720, 12727) state that analyzing similar house legislation, notably the 2015 Secret Science Reform Act and the 2017 HONEST Act, that would impose similar obligations on EPA reveals that the cost of implementation would be between $100-250 million annually, resulting in a significant burden on the Agency's current activities.,,,,,,,  Specific comments on the overall cost of the rulemaking include:
 Commenter (11408) contends that expanding the scope to include "influential scientific information" will only further direct resources toward unnecessarily scrutinizing nearly every study the Agency considers due to the extensive review process this SNPRM would entail.
 Commenter (11484) states that the SNPRM comes at a potentially significant cost without contributing to any serious effort to improve the transparency and validity of scientific information.
 Commenter (11492) states that the expansion will impede both the speed and accuracy of EPA decision-making, something that is particularly important during a pandemic.
 Commenter (12720) notes that it is not practical for EPA to arbitrarily screen specific studies, and EPA has not articulated in the supplemental proposal how such requirements will be implemented without significant interference in the Agency's work. According to the commenter, the supplemental proposal does not engage with the complexity and cascading ramifications of the requirements it spells out and should be withdrawn.
 Commenter (11479) contends it is clear that the current proposal would create a tremendous time and resource burden on EPA and on scientists outside of the Agency, who would be forced to track down, collect, compile, and post (including any statistical methods needed to conceal private information) all the data, models, code, and other analytical tools for each of the hundreds of studies EPA cites on any given decision. 
 Commenter (12720) maintains that despite this significant cost estimate, EPA does not confront the financial dimensions or the need for financial incentives to support the unprecedented data release requirements in the rule. The commenter asserts that it also does not consider the fact that scientists do not typically receive funding to make the data underlying peer-reviewed studies available for public inspection.
 Commenter (12727) notes that alternatively to the $100-250 million cost estimate, Congressional Budget Office (CBO) estimated that EPA might reduce those costs to $5 million over a five-year period, but only by "significantly reduc[ing] the number of studies that the Agency relies on." The commenter asserts that EPA failed to address these costs -- including the effects of potentially relying on fewer studies -- in the 2018 proposal, and it has failed to do so again in the Supplemental Notice, even though commenters provided these figures in their 2018 comments. The commenter contends that EPA's present failure to address the CBO estimates is even more arbitrary than in 2018 -- and not only because the estimates are now clearly in the record. The commenter provides that in addition, the Supplemental Notice more closely parallels the bills that CBO reviewed due to the expanded scope of Agency actions and studies it would encompass.
Response: While this final rule requires the Agency to evaluate the availability of dose-response data for pivotal science, the incremental burden to the Agency to carry out these requirements is expected to be small given (1) the extensive scientific review the EPA already conducts regularly and (2) that the requirement is limited to pivotal science (i.e., typically a small, though highly important, subset of the studies the EPA would review).
Comment: Commenter (11408) provides that the rulemaking would do little, if anything, to improve the quality of EPA's decision-making and enhance the scientific integrity of its program implementation, as documented in detail above. The commenter notes  -  without necessarily endorsing  -  that the Supreme Court has suggested that a hallmark of rational regulatory decision-making is that the costs created by a regulation not grossly outweigh the benefits it produces.
Response: EPA disagrees that this rule would do little to improve the quality of its decision-making and the scientific integrity of its programs. The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Comment: Commenter (11403) states that a study using epidemiological data would require independent validation by EPA scientists even though it has already undergone the peer-review process. The commenter states that this is duplicative and wastes Agency resources. 
Commenter (14397) states that this expanded proposal would mean that the EPA would either have to spend hundreds of millions of dollars to have private information redacted or move ahead without the benefit of the dozens of valid studies that cannot legally, ethically or efficiently publicly release all of their data. 
Commenter (12715) states that, although data may technically be "available," it is likely to be costly to obtain. 
Response: As was noted in the SNPRM, "Although the ability to independently validate pivotal regulatory science or pivotal science is a key component of this rulemaking, EPA would like to clarify that neither the proposed nor the alternative 40 CFR 30.5 would require that EPA, a member of the public or other entity must independently validate a study before it can be considered to be pivotal regulatory science or pivotal science."
The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA to disclose, redact, or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenter (12720) asserts that if the final rule applies only prospectively to new Safe Drinking Water Act (SDWA) advisories and benchmarks and toxicity values used in Superfund cleanup, EPA will still need to replace the current methodologies for these recommended values and revise the hierarchy of human health toxicity values in order to incorporate the new requirements, incurring significant costs and time lost to replacing these methodologies.
Response: The EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. Some of these subsequent actions will be substantive rules issued under the associated environmental statutes and will be subject to judicial review.
Comment: Commenter (12715) asserts that EPA ignores the large costs that would be associated with the complex process and fails to identify who would pay for these procedures. The commenter notes that as scientists from the Union of Concerned Scientists has discussed, redacting confidential data from large studies "`isn't just blocking out a line' It's a huge job that can occupy entire offices for thousands of hours," at commensurately high cost. Furthermore, the commenter notes that in a 2018 article discussing this same issue with respect to the HONEST Act, the Union of Concerned Scientists also said that:
      EPA career staff argue that the HONEST act would incur additional costs and time to implement, would limit research that gets conducted, and would deter industry and academics from working with the agency. In their comments to the CBO, [EPA] staff said that, "In addition to spending dollars and staff time on requesting and getting data from study authors, creating [information technology] infrastructure and a data management system to manage, store, and archive large volumes of data, and making the data available in a format that is useful and accessible to the public, EPA would also have to spend dollars and staff time combing through these extensive datasets to find and redact Personally Identifiable Information and Confidential Business Information.
Response: While this final rule requires the Agency to evaluate the availability of dose-response data for pivotal science, the incremental burden to the Agency to carry out these requirements is expected to be small given (1) the extensive scientific review the EPA already conducts regularly and (2) that the requirement is limited to pivotal science (i.e., typically a small, though highly important, subset of the studies the EPA would review).
The final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenter (12727) states that EPA has not estimated the proportion of valid, relevant research that it would disregard under the supplemental proposal. The SAB, according to the commenter also noted that "[i]t is not clear: (1) how many of the studies EPA currently relies upon to take important regulatory actions would meet the public disclosure standards stated in the Proposed Rule." According to the commenter, the CBO has estimated the number of studies that legislative proposals similar to EPA's rule would affect, both overall and at the level of individual actions, such as reviews of the NAAQS. The commenter asserts that EPA presumably has better information than CBO does about the volume of scientific information it relies on across its programs as CBO relied on information from EPA on how the Agency would implement the legislation, as well as the cost of complying with it, in developing the CBO estimate. Using updated information, the commenter contends that EPA must conservatively estimate both the absolute numbers and the proportion of studies it would rule out through this SNPRM -- overall and in each foreseeable individual action -- reasonably assuming that tiered access and de-identification are not feasible. The commenter states that, similarly, the Agency must conservatively estimate the number of studies that would be given lesser consideration under the alternative approach described in the supplementary proposal. The commenter provides that the omission of any such analyses renders the proposed rule arbitrary.
Response: The final rule is limited to dose-response data and, as no studies are categorically excluded from consideration, the EPA will continue to rely on the full body of the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Consistent with the requirements of this rule, the EPA will identify a subset of those studies based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints that would inform a dose-response assessment, and will give greater consideration to pivotal science for which the underlying dose-response data are available.
The EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. Some of these subsequent actions will be substantive rules issued under the associated environmental statutes and will be subject to judicial review.
14.2.4.3 Costs to the Research Community
Comment: Commenter (11361) states that publicly available datasets, especially those created under federal research grants, enable more researchers to utilize data that has already been collected and broadly increases research efficiency. The commenter provides that primary data collection is often extremely expensive and time-consuming, while secondary data analysis can often be accomplished more rapidly and on a much leaner budget. 
Commenter (11395) provides that the proposed regulation would impose significant costs on research scientists. The commenter notes that formatting, preparing and de-identifying datasets and associated materials to be publicly available or to conform with EPA's proposed tiers of data availability would be extremely time and resource intensive. Further, the commenter states that according to a 2002 report from the National Academy of Sciences (NAS), re-analyses of studies almost always require the participation of the original researchers to provide additional information and support, at an additional cost of personnel time and resources.
Commenter (11482) notes that EPA fails to define or consider the financial costs required to implement the proposed rule and queries who will fund the collection, de-identification, storage, and dissemination of such an enormous amount of sensitive data? The commenter asserts that the Agency has not formally announced who would pay for rule implementation, and no new federal money for the Agency or outside scientists has been proposed. The commenter maintains that in suggesting that the potential costs of the rule would be minimal, the EPA ignores the costs to researchers (who would have to pay to set up and maintain data sharing for their previously published studies to be considered), to the EPA (for presumably conducting the multiple re-analyses required in section 30.6 of the rule), and to public's health (for the very real costs in dollars and human lives of undermining existing regulations by limiting access to science).
The commenter (11482) contends that had EPA adequately considered costs, it would have determined that the collection, deidentification, storage, and dissemination of sensitive data in a manner sufficient for independent validation are likely to cost millions of dollars. The commenter states that the proposed rule fails to offer any guidance on how these strategies can be achieved with zero funding and upon whose shoulders this cost burden should fall.
The commenter (11482) provides that commenters must rely on the only applicable estimates that currently exist  --  two CBO reports on (failed) federal legislation that was substantially similar to the proposed rule. The commenter notes that in 2015 and 2017, CBO estimated that a similar science program would cost $1 million dollars per year to implement; however, a 2017 cost estimate from the EPA itself placed that number at as much as $250 million. The commenter asserts that one thing is certain: the "public availability" requirement in this proposed rule would cost a great deal of money to carry out. The commenter contends that if EPA is not covering this cost, scientists and scientific institutions will bear the lion's share of this burden at a time of shrinking budgets, resulting in less science  --  not better science  --  and weaker public and environmental health decision making.
Response: The final rule does not regulate any party outside of the EPA but rather exclusively governs the EPA's internal process for determining the consideration to afford pivotal science with respect to certain actions. This rule does not require any researcher or other outside entity to provide data or models to the EPA. Nor does the rule categorically exclude studies, even studies where the underlying dose-response data are not available for independent validation, and therefore any incidental impact on researchers who are developing science and deciding whether to make the underlying dose-response data available is negligible. Instead, it governs internal agency procedures for determining the consideration to afford various studies according to factors that include data availability. In doing so, the final rule provides greater transparency on the consideration the EPA will give pivotal science where the underlying dose-response data are or are not available for independent validation.
This is a rule of internal procedure that does not impose requirements on any party other than the EPA. This rule imposes no costs on researchers or their institutions, and the EPA will consider and evaluate all relevant and appropriate science in its significant regulatory actions and influential scientific information. The EPA recognizes that researchers and other third parties may voluntarily consider the EPA's position on data availability, as described in this rule, as they make their own decisions about how to conduct research and the extent to which they make data and models available. Researchers may choose to make more data and models available, but the EPA recognizes that these parties will weigh their own benefits and costs and make choices that they deem appropriate. 
If researchers want to increase the likelihood that their studies receive greater consideration by the EPA, they may take steps to ensure that the underlying dose-response data are available to the greatest extent possible. But any such response to this final rule would be purely voluntary. It is not required by this rule.
Comment: Commenter (11487) provides that because Research Data Center (RDC) contains data collected by Centers for Disease Control and Prevention (CDC) and other federal agencies, not by independent academic researchers, a great deal of additional work and infrastructure would be necessary before it could host the full range of data that should be informing EPA work. The commenter notes that fact that RDC charges researchers $3,000 to access a single year's data hints at the substantial costs associated with the enterprise. The commenter states that EPA does not appear to have calculated the costs that the Agency and outside researchers would incur in order to create a system that evaluates researchers' proposals and provides them with access to the appropriate level of data. The commenter contends that EPA also has not explained who would be in charge of making those determinations -- a crucial issue given that communities disproportionately harmed by pollution might not trust EPA to make them.
Response: The final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenter (12546) notes that the CBO's analyses were underestimates, as they did not include the costs to research/ academic scientists. The commenter provides that making datasets publicly available along with "associated protocols...and detailed descriptions of how to access and use such information" would entail significant time and costs to format, prepare and, in the case of human data, attempt to de-identify individual results from the men, women and children who participated in the study. The commenter notes that such re-analyses almost always require the participation of the original researchers to provide additional information and support, which costs personnel time and resources. The commenter provides that a 2013 memo from the Office of Science Technology and Policy on increasing data access acknowledges these costs and directs agencies to "Allow the inclusion of appropriate costs for data management and access in proposals for Federal funding for scientific research."
The commenter (12546) asserts that in contrast, the supplemental proposal contains no provisions that address the funding needed for academic scientists to make their datasets publicly available and support re-analyses --  and if the data are not publicly available, EPA may downgrade or fail to incorporate it in relevant policymaking.
Response: This is a rule of internal procedure that does not impose requirements on any party other than the EPA. This rule imposes no costs on researchers or their institutions, and the EPA will consider and evaluate all relevant and appropriate science in its significant regulatory actions and influential scientific information. The EPA recognizes that researchers and other third parties may voluntarily consider the EPA's position on data availability, as described in this rule, as they make their own decisions about how to conduct research and the extent to which they make data and models available. Researchers may choose to make more data and models available, but the EPA recognizes that these parties will weigh their own benefits and costs and make choices that they deem appropriate.
Comment: Commenter (12291) states that creation of both the workflow implemented to create the analytical data set used and the statistical software used to analyze the data set in a manner that enables even highly-qualified investigators to reproduce the same results is difficult and time consuming. The commenter notes that that this is likely to be costly to the researchers and their institutions  -  both in the creation of the data and methods hubs, and in the continuous requests for further explication from those seeking the data.
Response: This is a rule of internal procedure that does not impose requirements on any party other than the EPA. This rule imposes no costs on researchers or their institutions, and the EPA will consider and evaluate all relevant and appropriate science in its significant regulatory actions and influential scientific information. The EPA recognizes that researchers and other third parties may voluntarily consider the EPA's position on data availability, as described in this rule, as they make their own decisions about how to conduct research and the extent to which they make data and models available. Researchers may choose to make more data and models available, but the EPA recognizes that these parties will weigh their own benefits and costs and make choices that they deem appropriate.
14.2.5 Implementation Bias
 14.2.5.1 Rule Favors Certain Stakeholders
Comment: Commenters (10055, 11330, 12522, 12727) contend that the SNPRM provides for implementation in favor of industry stakeholders and/or special interests. These concerns include:
Commenter (10055) asserts that the SNPRM would open up the process of standard-setting to lengthy debates arising from the "reanalyses" and "alternative" models funded by the regulated industries. Further, the commenter states that the proposed supplement to the Transparency Rule gives immense power to regulated industries in the development of policy and regulation. According to the commenter, there are no provisions for controlling conflicts of interest in the "reanalysis" of data, the "alternative models", or the "independent validation".
Commenter (11330) contends that the rulemaking is not about transparency, nor about protecting human health or protecting our environment. The commenter states that this emperor simply has no clothes and must again ask EPA to put science and public interest ahead of political and special interests and withdraw this rule based on its negative impacts on science, and its negative impacts on our public health.
Commenter (12422) notes that polluting corporations, attempting to defend their products and emissions, will no doubt be able to provide whatever data and materials supporting their positions in any format the EPA might demand. The commenter states that this rule could impose an onerous and burdensome requirement on independent researchers so that much environmental science, particularly epidemiologic studies, would be excluded from the evidence base and become irrelevant to efforts by EPA to protect the public and the environment.
Commenter (12727) provides that not all funders would be equally disinclined to support research meeting EPA's requirements. The commenter notes that for instance, research backed by industry that profits from pollutants or toxicants might be adequately resourced to adapt to EPA's new requirements, since such funders would have a financial interest in which studies the Agency considers. The commenter states that the proposal could thereby skew the research landscape in favor of studies whose funders hope to reap a monetary benefit from the outcome of the research.
Conversely, commenter (12692) states that conflicts of interest in the rulemaking process are ubiquitous. The commenter notes that although participants may be honest people, few are honest brokers -- persons without an interest or stake in the outcome. The commenter provides that stakeholders include not only the industries subject to the Agency's regulations but also the advocacy groups that gain influence and contributions by supporting or opposing the Agency's agenda, the researchers who receive Agency grants to study issues of regulatory interest, and the Agency itself, whose scientific assessments drive the scale and scope of its regulatory activity.
According to the commenter (12692), although every stakeholder is an interested party, disclosure requirements that bring conflicts of interest to light can help make the process more competitive, nothing in the Transparency Rule would change any conflict of interest disclosure regulations or any part of EPA's scientific integrity policy.
Response: The EPA does not agree that its approach will lead to systematic bias towards certain types of stakeholder goals.
The final rule does not regulate any party outside of the EPA but rather exclusively governs the EPA's internal process for determining the consideration to afford pivotal science with respect to certain actions. This rule does not require any researcher or other outside entity to provide data or models to the EPA. Nor does the rule categorically exclude studies, even studies where the underlying dose-response data are not available for independent validation, and therefore any incidental impact on researchers who are developing science and deciding whether to make the underlying dose-response data available is negligible. Instead, it governs internal agency procedures for determining the consideration to afford various studies according to factors that include data availability. In doing so, the final rule provides greater transparency on the consideration the EPA will give pivotal science where the underlying dose-response data are or are not available for independent validation.
14.2.6 Rule Includes Insufficient Implementation Detail/Clarity
 14.2.6.1 Need for Implementation Clarity
Comment: Commenter (10042) provides that there are two distinct ways transparency is used, and understanding the distinction between the two conditions of transparency gives clarity for the EPA in their direction and guidance for the respondents to make substantive and meaningful comments to the EPA' s proposed rulemaking.
The commenter (10042) states that the first way transparency is used is to understand that it is woven into the fabric of science and how scientific processes are carried out. The commenter asserts that the key attribute of this transparency is there is no choice. According to the commenter, one cannot choose to eliminate transparency from a scientific endeavor and still have a scientific activity; if there is no transparency, there is no science.
Although, the commenter (10042) provides that there are conditions and circumstances where transparency is a choice. The commenter notes that situations where individuals or groups are allowed to see a decision-making process across a spectrum of levels from non-participative observation to active and participative contribution in the final decision; this is transparency by choice.
Response: This rulemaking is one incremental step to increased transparency. The EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Comment: Commenter (11495) indicates several specific areas where they assert there is a lack of clarity on the proposed balance of the new shared responsibilities, including:
 How will an investigator know if their study is being considered as part of an Agency action that would fall under this rule? Will the EPA directly contact individual researchers or is it expected that individuals must preemptively be aware of these requirements and act accordingly?
 Who will determine what comprises a complete data set sufficient to meet the requirements of this new rule?
 Who will determine what is adequate with regards to the tiered access proposed in the supplement?
 Who will be responsible for hosting the data, and providing funding for data hosting, so that it can be made available to the public?
 Who is responsible for screening those that request the data to determine which tiered level of data should be made available to them?
 Who is responsible for interactions between those requesting data and those hosting the data in order to ensure those requesting the data can access only those tiers that they are eligible to receive?
 Who is responsible for mitigating any dispute between those requesting data and those hosting the data with regards to data access based on the proposed tiered system as well as the sufficiency of the provided data to enable the proposed reanalysis in the new rule?
 When will data be required to be available in order to be included in informing EPA actions?
The commenter (10042) contends that the reality is that the answer to all these questions is the same: the EPA proposes to wield capricious authority to determine if researchers have complied with these new requirements without providing any resources, clear guidance, or accountability for how this new rule will be applied as part of future Agency actions.
Response: This procedural rule does not regulate the conduct or determine the rights or obligations of any entity outside the federal government. Rather, it exclusively pertains to the internal practices of the EPA. The rule does not require the EPA to disclose or host data, but to determine if data and models are available and to give greater consideration to those studies for which data and models are available. Hence, this rule does not impose costs on the EPA or any other party to make data and models available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the rule.
The EPA will not identify a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. As noted by commenters, the step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying data or models are publicly available or available through restricted access. Restricted or tiered access in this rule means that the underlying data or models are be available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
This rulemaking applies prospectively to final significant regulatory actions and influential scientific information. This rulemaking has no retrospective effect on either final significant regulatory actions or influential scientific information. For future final significant regulatory actions and influential scientific information, it applies equally to all the data and models underlying studies used as pivotal science.
Implementation guidance and statute-specific transparency regulations or programmatic regulations are forthcoming.
Comment: Commenter (11576) states that EPA has failed to articulate clearly what the proposed rule would even require, obscuring the implications of the regulation with vague language and broad grants of discretion. According to the commenter, the SAB repeatedly emphasized this problem in its comments on the proposal, noting, for example, that:
 "[C]ertain key terms and implementation issues have not been adequately defined or described" in the proposed rule;
 "Given the relatively skeletal nature of the Proposed Rule, it is not possible to define the implications of the rule with confidence";
 "To ensure that the rule is evidence-based EPA must provide greater clarity regarding details of the rule and how it will be implemented, as well as example analyses of how it would be deployed";
 "The lack of specific criteria for what might satisfy the [proposal's availability] requirement makes it difficult for the SAB to understand the implications";
 "Given the lack of clarity in the Proposed Rule, it is difficult to understand how this regulatory action could be accomplished in a standardized and consistent manner";
 "[T]he lack of criteria for what data might satisfy the requirements of the Proposed Rule makes it difficult to understand the implications for protection of [personally identifiable information]";"[G]reater clarity is needed in definitions of `data and models' and `pivotal regulatory
science[,]'" as "[t]he definitions provided in the Proposed Rule are not adequate";
 "Case-by-case exceptions [to the proposal's requirements] without criteria may create public concerns about inappropriate exclusion of scientifically important studies";
 "The requirement in the Proposed Rule that `data' be made publicly available is
vague and, as a result, can be interpreted in different ways"; and
 "More clarity is ... needed to define the nature of the `data' that must be publicly available."
The commenter maintains that under the Administrative Procedures Act (APA), a rulemaking proposal "must describe the range of alternatives being considered with reasonable specificity. The commenter states that otherwise, interested parties will not know what to comment on, and notice will not lead to better-informed Agency decision-making."
Response: The definitions in section 30.2 of the final rule are data, dose-response data, independent validation, influential scientific information, pivotal science, publicly available, reanalyze, science that serves as the basis for informing a significant regulatory action, and significant regulatory actions.
"Data" is defined as, "the set of recorded factual material commonly accepted in the scientific community as necessary to validate research findings in which obvious errors, such as keystroke or coding errors, have been removed and that is capable of being analyzed by either the original researcher or an independent party." 
"Pivotal regulatory science" has been replaced with "pivotal science" in the final rule and is defined as, "the specific dose-response studies or analyses that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information."
Section 30.7 of the final rule states, "The Administrator may grant an exemption to this part for a study on a case-by-case basis if he or she determines that greater consideration is warranted because:
       technological or other barriers render sharing of the dose-response data infeasible; 
       the development of the dose-response data was completed or updated before the effective date of the final rule; 
       making the dose-response data available would conflict with laws and regulations governing privacy, confidentiality, confidential business information, or national security;
       a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis; or
       the factors used in determining the consideration to afford to the pivotal science indicate full consideration is justified."
"Publicly available" is defined as, "lawfully available to the general public from federal, state, or local government records; the internet; widely distributed media; or disclosures to the general public that are required to be made by federal, state, or local law. The public must be able to access the information on the date of publication of the proposed rule (or, as appropriate, a supplemental notice of proposed rulemaking, or notice of availability) for the significant regulatory action or on the date of dissemination of the draft influential scientific information for public review and comment."
The EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. As noted by commenters, the step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
Implementation guidance and statute-specific transparency regulations or programmatic regulations are forthcoming.
Comment: Commenter (12464) notes that the SAB highlighted the multiple ambiguities in the Proposal in its recent report. The commenter provides that, as the SAB put it, "key considerations that could inform the Proposed Rule are not present in the proposal, or presented without analysis, and certain key terms and implementation issues have not been adequately defined or described." The commenter maintains that moreover, the SAB concluded that, "[g]iven the relatively skeletal nature of the Proposed Rule, it is not possible to define the implications of the rule with confidence." According to the commenter, if even EPA's expert science advisors cannot determine the implications of the Proposal, it is impossible for the public to understand and provide meaningful comment on the Supplemental Notice.
Response: Section III. A. Purpose and Effect of the Action in the preamble of the final rule attempts to address these concerns. This final rule is effective at publication in the Federal Register. The implementation guidelines are forthcoming. 
Comment: Commenter (11496) contends that the rule and SNPRM do not address how the rule may work in practice. In addition, the commenter objects that the SNPRM deems comments that are not directly "addressed, altered, or replaced by the SNPRM" as out of scope when the rule has been issued in pieces and without a full analysis of its impacts. The commenter provides that the SNPRM significantly modifies the effect of the rule and creates an untenable standard where research scientists, clinicians, and peer reviewers must collect and store protected personal data so their work can be utilized by EPA.
More broadly, the commenter (11496) notes that EPA's own SAB described the rule as "skeletal" and stated that the full implications of the rule remain undefined. The commenter concurs with the SAB's conclusions. The commenter adds that EPA's supplement provides additional definitions and exemptions that only serve to add ambiguity about how the rule will be implemented.
Additionally, the commenter (11496) provides that the proposed changes in the SNPRM revise many critically important definitions such as "reanalyze," "independent validation," and even such basic terms as "data" and "models," along with a new use and interpretation of the term "substantially reproduced." The commenter asserts that these new definitions do not resolve the numerous conflicts between this rule and other existing regulations. The commenter is also concerned that the rule language places the financial costs of action above human and environmental health costs. The commenter states for example, the definition of a "highly influential scientific assessment" puts a significant amount of weight on whether an impact of regulation is greater than $500 million in any one year, but does not describe how those costs may be outweighed by health benefits to people and the environment through regulatory action.
Response: Section III. A. Purpose and Effect of the Action in the final rule preamble of the final rule attempts to address these concerns. This final rule is effective at publication in the Federal Register. The implementation guidelines are forthcoming.
The definitions in section 30.2 of the final rule are data, dose-response data, independent validation, influential scientific information, pivotal science, publicly available, reanalyze, science that serves as the basis for informing a significant regulatory action, and significant regulatory actions.
Comment: Commenter (11894) notes that EPA sponsors some studies through grants, some through contracts for various purposes, conducts some studies itself, and yet other studies are submitted to the Agency in response to regulatory requirements. The commenter provides that all these activities take place on a decentralized basis in many different offices. The commenter queries: At what point in the process would studies be selected for the data bar? At what stage in their preparation would the bar apply? Who would enforce it, and under what standards? What rights of appeal would exist? And what would happen to disapproved studies? Would they be withdrawn from the public and burned? If not, what would be their fate?
The commenter (11894) states that the proposal's only approach to addressing these questions states that the data bar would apply before "finalizing" of affected studies by EPA, an undefined term that is too general to be any answer at all.
Response: The Agency will not be removing studies funded by EPA through grants, contracts, and intramural research. 
This rule governs internal agency procedures for determining the consideration to afford various studies according to factors that include data availability. In doing so, the final rule provides greater transparency on the consideration the EPA will give pivotal science where the underlying dose-response data are or are not available for independent validation.
Comment: Commenter (12457) contends that EPA should not exclude studies from policy considerations, but rather include all available literature to obtain a better understanding of relevant science. The commenter continues in stating that while transparency and data sharing are important in science, it should be considered whether the proposed rule is the best method for advancing these initiatives. The commenter notes that other methods for increasing transparency and the availability of data are already underway in the scientific community, including work currently being done by academic institutions, local government, and federal institutes including the CDC, the Committee on National Statistics, and the National Research Council. According to the commenter, EPA should consider the proposed rule in light of these efforts and decide whether there may be better ways to encourage transparency in the science considered in policy decisions. The commenter maintains that if the EPA believes this proposed rule does represent the best methods to accomplish increased transparency, any proposed revisions to the rule should include a detailed explanation of how the EPA reached that conclusion, as well as a description of how this rule will operate in the context of ongoing transparency and privacy protection efforts.
Response: EPA notes that the final rule does not categorically exclude studies. The purpose of this rule is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Comment: Commenter (12467) urges the Agency to clarify that the proposed rule applies to all stages of the notice and comment rulemaking process. According to the commenter, this would ensure the consistency, transparency and predictability called for by the proposal extends to those stages of the regulatory process at which impacted stakeholders are afforded the opportunity to evaluate and respond to not only the data presented, but also the regulatory and policy actions they influence.
The commenter (12467) states that when impacted stakeholders and the public are provided an opportunity to comment on federal proposals, they must have access to all relevant data to ensure they fully understand the Agency's justification for the proposed rule, and their comments are clearly in response to the selected regulatory option(s) being proposed. The commenter contends that if the Agency does not make clear a full range of options and the data that informed them, including cost benefit data, those seeking to offer public comment cannot hope to be able to respond to the range of actions being considered. The commenter notes that if EPA only applies the provisions of this proposal to final regulatory actions, it is possible that impacted stakeholders will be cut off from responding to critical data that may significantly alter the outcome of regulatory policy decisions.
Response: Section 30.3 of the final rules states, "The provisions of this part apply to science that serves as the basis for informing a significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science."
The final rule states that the EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
Comment: Commenter (12546) notes that EPA's SAB issued a report earlier this year outlining that it was: "concerned that key considerations that should inform the rule appear to be omitted or presented with no analysis - for example, it is not clear how many of the studies EPA currently relies upon to take important regulatory actions would meet the public disclosure standards in the proposal, whether EPA has assessed how many of those studies would be feasible to provide underlying information, or what the impact of precluding those studies would be on EPA's decision making and its ability to protect public health / environment."
The commenter (12546) also states that the SAB had concerns about the scientific and technical challenges and feasibility of implementing some of the rule's requirements, recommending that the Agency provide more detail on how it can meet the requirement to clearly identify and make publicly available all studies relied upon when it takes "significant final agency action," among other recommendations. According to the commenter, the SAB has also not been additionally consulted, even though they have identified important flaws in the proposal.
Response: The preamble of the final rule addresses some of the concerns from the SAB report on the proposed rule. See sections III.A.1., III.C.6. and III.G. of the preamble of the final rule.
The EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
There may be pivotal science for which the underlying dose-response data are not publicly available due to technological feasibility or cannot be made available in a secure environment that still allows for independent analysis. For example, dose-response data underlying older pivotal science may no longer be available or may not exist in a currently usable format. In these cases, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator.
Comment: Commenter (12645) provides that the increasing number of co-authors in published studies raises the question of unanimous consent to data sharing. The commenter questions would all authors need to agree to data-sharing and only one refusal would prevent the study's data from being provided? The commenter states that EPA could find out by sampling the scientific community represented in cited publications underlying its regulatory decisions. According to the commenter, like so many other aspects of the initial transparency rule and this Supplement, EPA has not bothered to do any preliminary analyses or to have such analyses done by the NAS or the SAB.
Response: The final rule does not regulate any party outside of the EPA but rather exclusively governs the EPA's internal process for determining the consideration to afford pivotal science with respect to certain actions. This rule does not require any researcher or other outside entity to provide data or models to the EPA. Nor does the rule categorically exclude studies, even studies where the underlying dose-response data are not available for independent validation, and therefore any incidental impact on researchers who are developing science and deciding whether to make the underlying dose-response data available is negligible. Instead, it governs internal agency procedures for determining the consideration to afford various studies according to factors that include data availability. In doing so, the final rule provides greater transparency on the consideration the EPA will give pivotal science where the underlying dose-response data are or are not available for independent validation.
Comment: Commenter (12720) contends that Agency staff would need to conduct large information gathering assessments to determine the degree to which any study previously considered by the Agency in science reviews met the new requirements-- itself a huge task, given that each study would have to be assessed individually. The commenter states that it would be unreasonable at this stage in the process for the Agency not to contact the study authors and inform them of the new data requirements in place at EPA.
According to the commenter (12720), such communications will need to be comprehensively documented and coordinated for Agency use and public reference. The commenter notes that in many cases, it will be difficult to contact the primary study author, since in some cases years (or decades) have elapsed since study publication. The commenter queries: Would additional authors be contacted in this case? What if, upon communication of the new requirements, there arises disagreement amongst the study authors, study funders, or academic institutions about how and whether to make all underlying data available for public inspection?
Response: While the final rule requires the Agency to evaluate the availability of dose-response data for pivotal science, the incremental burden to the Agency to carry out these requirements is expected to be small given (1) the extensive scientific review the EPA already conducts regularly and (2) that the requirement is limited to pivotal science (i.e., typically a small, though highly important, subset of the studies the EPA would review).
The EPA would like to emphasize that this final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenter (12720) states that EPA has not explained in the proposal how the requirement to make the studies publicly available could be practically achieved. The commenter asserts that EPA does not identify any phasing-in period for the requirement, which would significantly complicate EPA's science review process. Additionally, the commenter provides no criteria have even been established by EPA to understand. The commenter notes that the lack of criteria at this stage in the process is problematic because it does not allow researchers--whose work can take many years to plan, fund, conduct, and publish-- adequate opportunity to understand the proposed requirements and submit comments to EPA.
Response: The final rule does not regulate any party outside of the EPA but rather exclusively governs the EPA's internal process for determining the consideration to afford pivotal science with respect to certain actions. This rule does not require any researcher or other outside entity to provide data or models to the EPA. Nor does the rule categorically exclude studies, even studies where the underlying dose-response data are not available for independent validation, and therefore any incidental impact on researchers who are developing science and deciding whether to make the underlying dose-response data available is negligible. Instead, it governs internal agency procedures for determining the consideration to afford various studies according to factors that include data availability. In doing so, the final rule provides greater transparency on the consideration the EPA will give pivotal science where the underlying dose-response data are or are not available for independent validation.
Implementation guidelines and statute-specific transparency regulations or programmatic regulations are forthcoming.
Comment: Commenter (12720) states that in EPA's memo to the House Committee on Science, Space, and Technology, "EPA's envisioned implementation  -  absent from the text of the Supplemental Proposal  -  would be as follows:
 The researchers would be responsible for managing the logistics of making the data and models publicly available in a manner that complies with the rule, in consultation with EPA staff.
 The researchers would be responsible for judging the sensitivity of the study's data and models and what information can or cannot be made publicly available through tiered access.
 The researchers would decide what tier of access should be designated for different types of information.
First, the commenter (12720) maintains that EPA burdens researchers with the obligation for managing complicated and costly logistics of making underlying data and models available to the public. The commenter notes that it is not clear how EPA would assure that these researchers actually carried out these logistical details, and no hint of this major burden to researchers is mentioned in the proposal; the logistics themselves are nowhere spelled out in the Supplemental Proposal.
Second, the commenter (12720) states that EPA evidently expects researchers to assess the sensitivity of study data but provides no criteria for doing so. The commenter provides that the expectation that researchers evaluate their own data and models conflicts with the Agency's plan to independently assess whether the research has indeed been made publicly available or whether EPA should grant a waiver from the requirements identified in the Supplemental Proposal. Similarly, the commenter notes that in the third bullet, EPA now says that it expects researchers to ascertain what data access tier is important for the underlying data and models used in their studies. The commenter states that no details are provided to assist researchers in making such important determinations. Further, the commenter contends that EPA's inability "to say how conflicts between the researcher's judgment and the Agency's judgment would be handled" is further indication that the rule is arbitrary and capricious and fails to provide fair notice or any reasoned explanation about this key implementation concern.
Commenter (12720) provides that, under the current iteration of the Supplemental Proposal, the public is left with gaping questions about how the Supplemental Proposal will be implemented. The commenter queries: How would EPA handle a reanalysis conducted by a third party? What procedures will EPA follow for assessing a reanalysis during a rulemaking? How might the absence of peer-review for a reanalysis influence the incorporation of those findings into influential scientific information or a rulemaking? How would EPA balance the timeline of a rulemaking with the timeline of a reanalysis, given that a reanalysis of a major study can take years and may not be completed before a regulatory deadline? The commenter contends that all of these critical questions have not be raised, let alone addressed, by the Agency.
The commenter (12720) notes that EPA seemed to acknowledge as much in its meeting with House staff, where the Agency "was largely unable to address these issues and frequently declined to answer questions, admitting that the Agency had simply not considered the issue this far into the rulemaking process." The commenter contends that not only are the substantive aspects of the rulemaking arbitrary and capricious, but also EPA's glaring failure to notify parties and include information on the content of the rulemaking in the Supplemental Proposal text renders the Proposal arbitrary and capricious and a truly egregious abuse of discretion. The commenter provides that as the memo notes, "the public comment period is no substitute for a well-reasoned, deliberative policymaking process. EPA's inability to answer basic questions about the rule's operation and implementation reflects the ill-conceived nature" of the Supplemental Proposal.
Commenter (12720) considers the situation where a single study examining several distinct health endpoints is consulted in different ways. The commenter questions whether EPA would make a single determination on such a study, regardless of its application in specific regulatory decisions. If so, the commenter contends the supplemental proposal would arbitrarily prevent a study from being considered in different ways for different contexts and regulatory decisions (e.g., in the case where a study examines both commonly and less-commonly studied health endpoints).
Response: The final rule does not regulate any party outside of the EPA but rather exclusively governs the EPA's internal process for determining the consideration to afford pivotal science with respect to certain actions. This rule does not require any researcher or other outside entity to provide data or models to the EPA. Nor does the rule categorically exclude studies, even studies where the underlying dose-response data are not available for independent validation, and therefore any incidental impact on researchers who are developing science and deciding whether to make the underlying dose-response data available is negligible. Instead, it governs internal agency procedures for determining the consideration to afford various studies according to factors that include data availability. In doing so, the final rule provides greater transparency on the consideration the EPA will give pivotal science where the underlying dose-response data are or are not available for independent validation.
This rule imposes no costs on researchers or their institutions, and the EPA will consider and evaluate all relevant and appropriate science in its significant regulatory actions and influential scientific information. The EPA recognizes that researchers and other third parties may voluntarily consider the EPA's position on data availability, as described in this rule, as they make their own decisions about how to conduct research and the extent to which they make data and models available. Researchers may choose to make more data and models available, but the EPA recognizes that these parties will weigh their own benefits and costs and make choices that they deem appropriate.
As was noted in the SNPRM, "Although the ability to independently validate pivotal regulatory science or pivotal science is a key component of this rulemaking, EPA would like to clarify that neither the proposed nor the alternative 40 CFR 30.5 would require that EPA, a member of the public or other entity must independently validate a study before it can be considered to be pivotal regulatory science or pivotal science."
Comment: Commenter (12727) argues that the vagueness of the proposed definition creates problems regardless of how widely or narrowly the rule is applied. Attempting to apply the rule to all studies involved in the development of influential scientific information "would be entirely infeasible, effectively paralyzing the development of influential scientific information and with it the work of the agency." On the other hand, a narrow interpretation "introduces incredible risk for selective and biased application with no accountability."
Response: The definition of "influential scientific information" at proposed 40 CFR 30.2 in the 2020 SNPRM is the same definition as in OMB Bulletin for Peer Review. The EPA proposed to adopt this definition because it intended the scope to be consistent with how that term has been interpreted and applied in the context of peer review. Given that the definition is both established and has been routinely applied by the EPA, the EPA disagrees that the term is inherently too narrow or too broad. Rather than modify the proposed 40 CFR 30.2 definition of "influential scientific information," the EPA is modifying 40 CFR 30.3 in the final rule to clarify the Agency's intent that the requirements in 40 CFR 30.3 apply to influential scientific information, unless the influential scientific information is exempted from peer review requirements as described in Section IX of the OMB Final Information Quality Bulletin for Peer Review.
Comment: Commenter (13788) notes that the SAB warned EPA that changing transparency requirements could have negative consequences:
      It is plausible that in some situations, the Proposed Rule will decrease efficiency and reduce scientific integrity, determining if in fact that will be the case requires a thorough and thoughtful examination that is currently absent in the Proposed Rule. Moving forward with altered transparency requirements beyond those already in use, in the absence of such a robust analysis, risks serious and perverse outcomes.
According to the commenter (13788), the SNPRM does not include the additional "robust analysis" of the need for the changed policy called for by the SAB, and instead continues to leave important details regarding implementation of the rule undecided. The commenter maintains that at the very least, EPA must provide a response to the problems raised by the SAB, including some explanation and justification for this rule.
Additionally, the commenter (13788) provides that EPA in the SNPRM proposes to apply its data release requirements across the board, without distinguishing between types of studies and the data they rely on, which may or may not include confidential information. Additionally, the commenter notes that while the 2018 Proposal was limited to dose-response data and models, the SNPRM expands that to all data and models broadly and without justification. The commenter references the SAB's report on the 2018 Proposal that stated:
      the requirement to make data available for public inspection will be more easily implemented for some datasets than others (e.g., [certain] studies include individual sample data as part of standard reporting). The expectation that data and methods will be available for all endpoints may be unrealistic.
The commenter (13788) contends that the SAB was clearly concerned that the rule's requirements could not even be put into practice, never mind whether they are lawful or rational or adequately justified. Additionally, the commenter states that although the SNPRM allows the Administrator to grant an exception to the requirements, the SNPRM provides no reason to believe that provision will sufficiently address the concern identified by the SAB, nor is there any safeguard included to ensure that the EPA Administrator, in exercising his discretion, will not bring extra-statutory political considerations into the decision-making process.
Response: Section III. A Purpose and Effect of the Action in the preamble of the final rule attempts to address these concerns. This final rule is effective at publication in the Federal Register. The implementation guidelines are forthcoming.
Comment: Commenter (11176) states that the supplement introduces a new process that lacks description and that uses language requiring judgement calls for which no protocols are provided. The commenter provides that the supplement calls on EPA staff to determine, for example, what studies are "sufficient for independent validation" and what studies should be given "greater consideration."
The commenter (11176) notes that, starting with the first phrase, "sufficient for independent validation", the EPA's supplement does introduce a definition for the term, independent evaluation: "the reanalysis of study data by subject matter experts who have not contributed to the development of the original study to demonstrate that the same analytic results are capable of being substantially reproduced." The commenter contends that the concern is that there is insufficient guidance or standards for what the terms, "sufficient" and "substantially", mean or who will make the determination (e.g., scientific staff with oversight of an EPA scientific advisory panel). The commenter states that to its credit, EPA is using the 2002 OMB definition for "Capable of being substantially reproduced". The commenter notes that the definition however uses the words, "similar" and "acceptable," which are subject to the same concerns as "sufficient" and "substantially". The commenter asserts that standards for these terms and determination should be included before this proposed rule is finalized and should include sufficient guidance to, for example, take into account how unlikely it is for two studies to exactly replicate. The commenter provides that in 2015 Ralph Cicerone, then president of the NAS addressed this point in a speech, noting that a NAS panel had grappled with the very question: "Because variability across studies is to be expected, how can we assess the acceptable degree of variability and when should we be concerned about reproducibility?" According to the commenter, there should also be guidance to require explanation of whether research has or has not met the sufficient-for-independent-validation criterion.
Additionally, the commenter (11176) highlights the importance of meta-analyses as a research standard and states the fact that the supplement does not reference this well-known and valued scientific practice, which EPA staff regularly uses, reinforces the point that EPA has much work to do in laying the foundation for accomplishing the stated goal of the proposed rule.
According to the commenter (11176), these two issues -- ambiguity of terms and lack of process and transparency -- risk the introduction of bias, if not improper influence, into the EPA process. The commenter states that specifically, the two issues risk "arbitrary and capricious decisions as to what evidence to include or exclude" and "would also often result in regulatory analyses that suffer from biased selection and inadequate statistical power," among other issues.
The commenter (11176) contends that before the EPA implements the rule, it should design and explain its process for considering different studies. Additionally, the commenter provides EPA should also acknowledge the risk of bias in the process and explain how it will work to minimize such risk. The commenter notes that a highly regarded site is that of the Cochrane Collaboration's revised risk of bias framework available at the following url: https://www.riskofbias.info/.
Response: The EPA is not identifying a specific step in a multi-step analysis as the stage of data that would be sufficient for independent validation through reanalysis because this would be overly prescriptive and not informative. As noted by commenters, the step at which the final clean data set will be generated will vary from study to study. The level of detail required would be that needed for a separate party to reanalyze the study. The appropriate step is where the data are ready to be analyzed to extract relevant information.
Section 30.5 of the final rule states, "The EPA will use the following factors to assess the quality of studies identified in the systematic review: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. . . . The Agency will identify the pivotal science that was given lesser consideration and provide a short description of why lesser consideration was given."
The EPA does not agree that its approach will lead to systematic bias towards certain types of stakeholder goals.
Comment: Commenter (11894) asserts that EPA's alternative proposal on study and model acceptability does not set out any truly binding standards for making decisions. The commenter provides that the factors it mentions are so numerous and poorly articulated that the Agency would remain free to do whatever it wants.
Response: Section 30.5 of the final rule states, "The EPA will use the following factors to assess the quality of studies identified in the systematic review: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. . . . The Agency will identify the pivotal science that was given lesser consideration and provide a short description of why lesser consideration was given."
14.2.6.2 Added Definitions
Comment: Commenter (12715) The commenter provides that the supplemental proposal both fails to address many of the technical problems identified in the 2018 Comments on the initial proposal and creates several new problems. The commenter states that for example, the clarifications, modifications, and additions to the proposed rule raise significant additional concerns regarding the intent, applicability, and implementation of the rule, as discussed above. The commenter notes that the expansion of the scope of studies that would now be excluded or given less weight is particularly problematic not only because the manner in which EPA proposes to evaluate data and studies for consideration defies basic scientific principles, but also because there is a complete lack of transparency regarding how the Agency will apply the rule.
Additionally, the commenter (12715) notes that the narrative section of the supplemental proposal states that EPA is "modifying, deleting and proposing new regulatory text" in the definitions section, including "deleting the first paragraph of the 2018 proposed rulemaking regulatory text" and the definition of research data, but in the proposed rule section, EPA states that it is "[r]evis[ing] § 30.2 by adding the [listed] definitions . . . to read as follows." The commenter provides that unlike other sections of the supplemental proposal where EPA appears to have included the full revised provision, the proposed rule text for section 30.2 appears to include only the added definitions. The commenter asserts that it is thus unclear which, if any, portions of the original proposed rule text are being "modif[ied]" or "delet[ed]" as the narrative description indicates. The commenter states that EPA's unclear presentation of the changes it is making to this section of the supplemental proposal deprives the public of the opportunity to provide meaningful feedback on this portion of the rule.
Further, the commenter (12715) contends that the definitions are also problematic because the proposed rule repeatedly refers to "significant regulatory decisions," but the proposal fails to provide any definition of the terms "significant regulatory decisions" or "significant." The commenter notes that even more confusingly, "regulatory decisions" are defined in the original proposal as "final regulations determined to be `significant regulatory actions' by the Office of Management and Budget." According to the commenter, the confusion throughout the definitions section infects the entire proposal, because understanding whether and how EPA has defined key terms is critical to understanding how the other substantive provisions of the rule would operate.
Response: The definitions in section 30.2 of the final rule are data, dose-response data, independent validation, influential scientific information, pivotal science, publicly available, reanalyze, science that serves as the basis for informing a significant regulatory action, and significant regulatory actions.
The Agency is changing the term from "regulatory decisions" to "significant regulatory actions" in the final rule. "Significant regulatory actions" is defined as, "final regulations determined to be `significant regulatory actions' by the OMB pursuant to Executive Order 12866."
14.2.6.3 Section 30.5 Alternatives
Comment: Commenter (11192) notes that EPA proposes two alternatives to outright dismissal of studies that can't make their underlying data public for legal or ethical reasons: tiered access to the data (which doesn't ameliorate privacy considerations) and a weighting scheme to devalue those studies. According to the commenter, the EPA provides no accompanying details about how these measures would be implemented, leaving either or both open to manipulation for political ends.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Implementation guidelines and statute-specific transparency regulations or programmatic regulations are forthcoming.
Comment: Commenter (12427) provides that the proposed rule and SNPRM have a:
 Lack of development, demonstration, and evaluation of alternative approaches prior to arriving at proposals for a regulation
 Lack of a detailed evaluation of whether, and if so how, the proposed rule could conflict with existing statutes
 Lack of development of experience with proposed measures by testing them prior to attempting
Response: As discussed in the final rule preamble at I. C. "What is the Agency's authority for taking this action?," this internal procedural rule does not interpret or apply provisions of a statute that EPA administers. If implementing this procedural rule would result in conflicts with existing environmental statutes, and their implementing regulations, this rule will yield to the EPA statutes and regulations.
Alternatives approaches were discussed in the March 2020 Supplemental Notice of Proposed Rulemaking, and commenters were specifically asked to comment on the alternative approaches proposed.
This final rule is effective at publication in the Federal Register. The implementation guidelines are under development and statute-specific transparency regulations or programmatic regulations are forthcoming.
Comment: Commenter (12715) contends that in the words of EPA's own SAB, "[g]iven the relatively skeletal nature of the Proposed Rule, it is not possible to define the implications of the rule with confidence." The commenter states that in particular, it is unclear which definitions are included in the proposed rule, EPA fails to define key features of the two "options" proposed for determining which data and models may be used, and the proposed rule's exemptions provision contains insufficient standards to guide the Administrator's discretion. According to the commenter, the supplemental proposal thus runs afoul of the APA's substantive and procedural requirements due to its vagueness as well as its arbitrariness, thus depriving the public of a meaningful opportunity to comment. Accordingly, the commenter asserts that if EPA does not abandon the proposal altogether, it should reclassify the proposal as, at most, an Advance Notice of Proposed Rulemaking.
Response: The definitions in section 30.2 of the final rule are data, dose-response data, independent validation, influential scientific information, pivotal science, publicly available, reanalyze, science that serves as the basis for informing a significant regulatory action, and significant regulatory actions. 
Section 30.5 of the final rules states, "The EPA shall give greater consideration to pivotal science where the underlying dose-response data are publicly available in a manner sufficient for independent validation. The Agency shall also give greater consideration to pivotal science based on dose-response data that include CBI, proprietary information or PII if these data are available through restricted access in a manner sufficient for independent validation."
Section 30.7 of the final rule provides the aspects of a study the Administrator can consider when determining whether to grant an exemption.
Comment: Commenter (10972) asks how the weighting of studies will be determined. Commenters (9362, 10972, 11484, 12715) question who is responsible for determining that "other things are equal" when deciding which studies should be given greater consideration. Commenter (12715) states that neither approach in section 30.5 address concerns regarding the arbitrary exclusion or down-weighting of relevant scientific studies. According to the commenter, Option 1 unlawfully and arbitrarily excludes studies from consideration while Option 2's weighting approach is also arbitrary due to no inclusion of criteria for discretion. Additionally, the commenter points out that the clause "other things equal" is vague and fails to provide criteria to bound the Agency's discretion.
Furthermore, the commenter (12715) contends that the arbitrariness of both alternatives for section 30.5 raises troubling conflict of interest concerns regarding the types of studies that EPA would be able to consider. The commenter provides that studies for which the underlying data are or could likely be made available for independent validation are typically Good Laboratory Practice ("GLP") studies performed by industry. The commenter notes that while EPA has requested comment on "how to ensure that, over time, more of the data and models underlying the science ... are available to the public," as well as "how to provide sufficient incentives and support to researchers to increase access to the data," there is no indication of how this would be accomplished. The commenter states that accordingly, either version of section 30.5 would strongly favor the Agency's consideration of scientific studies conducted by or on behalf of the very industries and special interest groups that will ultimately be regulated by EPA's standards. The commenter maintains that these are the entities whose studies are more likely to have underlying data that is currently available for public validation or who can, going forward, afford to comply with the proposed rule's public availability requirements.
Response: The final rule is limited to dose-response data and, as no studies are categorically excluded from consideration, the EPA will continue to rely on the full body of the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Consistent with the requirements of this rule, the EPA will identify a subset of those studies based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints that would inform a dose-response assessment, and will give greater consideration to pivotal science for which the underlying dose-response data are available.
Section 30.5 of the final rule states, "The EPA will use the following factors to assess the quality of studies identified in the systematic review: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. . . . The Agency will identify the pivotal science that was given lesser consideration and provide a short description of why lesser consideration was given." EPA did not finalized the "other things being equal" concept from the 2020 SNPRM.
The EPA agrees that additional clarification is needed regarding the other factors that the EPA will consider. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science is unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Though not exhaustive or exclusive, the EPA has identified several factors that balance some of the important technical considerations the EPA will consider in addition to data availability and that are particularly relevant to the stage of the analysis where dose-response data are used. The EPA will thus consider the following factors:
 the quality of the study relative to other studies for which the dose-response data are available; 
 the extent to which there are other studies for which the dose-response data are available;
 the sensitivity of the conclusions in the significant regulatory action or influential scientific information based on the use of the study;
 the extent to which the study is fit for the purpose or intended use relative to other studies for which the dose-response data are available;
 the use of exposures or doses in a range and duration that is relevant for the intended use and that minimizes the need for extrapolations;
 the extent to which the study is supported by other scientific evidence;
 the extent to which the study accounts for unique scientific considerations;
 the extent to which the study minimizes the use of defaults and assumptions, uses appropriate and strong statistical methods, and includes a robust representation of uncertainty and confidence intervals; and
 the study's consideration of a broad range of parametric dose-response or concentration-response models, a robust set of potential confounding variables, nonparametric models that incorporate fewer assumptions, various threshold models across the dose or exposure range, and models that investigate factors that might account for spatial heterogeneity.
These factors are intended to assist the EPA in determining the consideration to afford to pivotal science with underlying dose-response data that are not available for independent validation. The final rule requirements and the consideration of these factors apply to any data used in characterizing the relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect, regardless of the direction of that effect. Because study quality factors (including soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review) would have already been evaluated at an earlier stage in the assessment process, the studies evaluated with these additional factors are presumed to be of the highest quality available.
The EPA would like to emphasize that this final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
Comment: Commenter (12720) contends that EPA does not adequately describe the process by which "greater consideration" will be given to specific studies, and whether the characteristics of specific studies (e.g., sample size, confidence intervals of results, or overall methods validity) may compensate for any lack of full data transparency. The commenter provides that for example, the Agency typically considers thousands of studies in compiling integrated science assessments as part of the NAAQS. The commenter notes that each of these studies may satisfy the new data requirements proposed in the rule to differing degrees, and it is not clear whether or not any Agency-perceived shortcomings in meeting these data requirements may be compensated by other criteria that help EPA to determine the overall degree of study quality.
Response: The final rule is limited to dose-response data and, as no studies are categorically excluded from consideration, the EPA will continue to rely on the full body of the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Consistent with the requirements of this rule, the EPA will identify a subset of those studies based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints that would inform a dose-response assessment, and will give greater consideration to pivotal science for which the underlying dose-response data are available.
Section 30.5 of the final rule states, "The EPA will use the following factors to assess the quality of studies identified in the systematic review: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. . . . The Agency will identify the pivotal science that was given lesser consideration and provide a short description of why lesser consideration was given."
14.2.6.4 Agency Programs
Comment: Commenter (11894) provides that the proposal is thoroughly ambiguous on what Agency activities will be covered. The commenter states that on the one hand, it states that the new provisions would "not apply to any type of agency action" other than "significant" rules, however, it does not define the term "agency action." According to the commenter, "action" could mean literally everything an Agency does in the course of its work; it could mean "agency action" as defined in the APA, but it does not say that. In any event, the commenter maintains that even that term is notably ambiguous and regularly generates difficult administrative law questions. On the other hand, the rule's requirements would extend to all "influential scientific information," which is defined as:
      scientific information the agency reasonably can determine will have or does have a clear and substantial impact on important public policies or private sector decisions.
The commenter (11894) contends that given the scope of the expressed concerns, it would seem unreasonable to exempt the majority of EPA activity from its scope by a broad definition of "agency action." The commenter states that a broad definition of "agency action" would make the new restrictions applicable to studies in their less important uses, and inapplicable to studies used for the more consequential task of supporting Agency decisions, seemingly inconsistent with the policy behind the SNPRM.
One such program lacking clarity according the commenter (11894) are the "advisories" and "benchmarks" for state and local water supply systems detailing the potential hazards of pesticides and other chemicals in drinking water. The commenter states that this information has no binding legal effect, but state agencies rely on it to shape their efforts to protect public health. The commenter contends that EPA's proposal, however, does not discuss whether the issuance of these advisories and benchmarks would be regarded as "agency action," exempt from the new requirements, or the impact on the program and how EPA would address that impact, if they are not exempt.
Additionally, the commenter (11894) provides a review of the process of assessment in the Existing Chemicals program that would be impacted should the rule apply retroactively. With only one study used in the most recent trichloroethylene (TCE) assessment in which its raw data was available, the process would essentially shut down, according to the commenter. 
Response: The EPA is finalizing the applicability of the rulemaking to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods. The EPA is not applying this rulemaking to permit proceedings, site-specific actions, or enforcement actions because these actions are typically focused on individual regulated entities. Implementation guidelines and statute-specific transparency regulations or programmatic regulations are forthcoming.  
Comment: Commenter (12399) states that 40 CFR 30.3 of the proposed rule excludes anything not considered a "significant regulatory action," such as adjudications, enforcement actions, and permit proceedings. The commenter contends that some provision may be necessary to ensure the Agency does not needlessly delay permits and the like, but the rule should apply to Agency-produced science, including risk assessments and research that could have impacts in the marketplace.
Response: The EPA is finalizing the applicability of the rulemaking to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods. The EPA is not applying this rulemaking to permit proceedings, site-specific actions, or enforcement actions because these actions are typically focused on individual regulated entities.
Comment: Commenter (12720) provides that in briefings to the House Committee on Space, Science, and Technology on April 2, 2020, EPA staff have stated that the proposed rule "would open up existing regulations to being reworked" by confirming to staff that the rule's application to "prospective rulemaking" includes statutorily mandated reviews of existing standards. The commenter states that in the April 2 briefing, EPA noted that such "prospective rulemaking" includes statutorily mandated reviews of existing standards, such as the NAAQS.
The commenter (12720) contends that is highly inappropriate for EPA to make this clarification in a private setting, rather than engaging with the issue within the SNPRM itself, a setting that would provide members of the public an opportunity to comment on the issue. The commenter provides that beyond making this clarification in the briefing, EPA has not explained how the ongoing NAAQS reviews for fine particulate matter and ozone air pollution would proceed in the drastically altered scientific review context envisioned by the SNPRM.
Additionally, the commenter (12720) states that according to the memo, "EPA's envisioned implementation  -  absent from the text of the Supplemental Proposal  -  would be as follows:
      EPA staff would reach out to the researchers involved in a study that the Agency wants to consider during the development of significant regulatory actions or Influential Scientific Information.
The commenter (12720) asserts that these implementation details (none of which are included in the SNPRM), raise several glaring problems. The commenter first notes that EPA regularly consults thousands of scientific studies for influential scientific information like the Integrated Science Assessments for criteria air pollutants under the CAA NAAQS program. The commenter contends that it is not feasible for EPA staff to contact research authors in such a way, particularly because many studies involve multiple authors and conceivably permission would need to be obtained from all co-authors for any new plan to share data with EPA or the public.
Response: The EPA is finalizing the applicability of the rulemaking to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods. The EPA is not applying this rulemaking to permit proceedings, site-specific actions, or enforcement actions because these actions are typically focused on individual regulated entities.
The EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. If implementing this procedural rule would result in conflicts with existing environmental statutes, and their implementing regulations, this rule will yield to the EPA statutes and regulations.
Comment: Commenter (11495) notes that it is not specified which specific Agency actions will actually be covered under this rule, which individual(s) will be responsible for deciding if an action will be covered, and at which point in the process can this decision be made or revised. To this end, the commenter states that this lack of clarity is likely to lead to inconsistent application of these new requirements.
Commenter (12453) asserts this exclusion has long been recognized by EPA. As such, the commenter wishes to confirm that the supplemental proposal, and any promulgated final rule, does not include individual adjudications or permit proceedings, such as FIFRA or Federal Food, Drug, and Cosmetic Act (FFDCA) decisions on individual pesticide registrations or tolerances. The commenter notes that the scope would more broadly be applicable to rulemakings under FIFRA or FFDCA that do meet the definition of "significant regulatory action."
Conversely, Commenter (12399) commends the definition for "influential information" because it indicates that the rule could also apply to research that impacts both policy decisions and private sector decisions. The commenter asserts that this is important due the marketplace impacts Agency science can have. According to the commenter, it only makes sense that Agency science  -  including "nonregulatory activities" such as research studies or risk assessments  -  complies with the best available scientific standards and transparency requirements.
Response: The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods. Influential scientific information is defined as the "scientific information the Agency reasonably can determine will have or does have a clear and substantial impact on important public policies or private sector decisions," consistent with the definition of "influential scientific information" provided in the OMB Peer Review Bulletin. The EPA is not applying this rulemaking to permit proceedings, site-specific actions, or enforcement actions because these actions are typically focused on individual regulated entities.
The EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. If implementing this procedural rule would result in conflicts with existing environmental statutes, and their implementing regulations, this rule will yield to the EPA statutes and regulations.
Comment: Commenter (12467) states that the inclusion of influential scientific information is limited when compared to the full extent of regulatory and policy statements issued by the Agency and does not ensure impacted stakeholders will be afforded the full benefit of the rulemaking. As such, the commenter recommends that the requirements also be applied to Agency guidance documents due to their potential for impact on the regulated community at the same level as "significant regulatory actions".
Response: The EPA is finalizing the applicability of the rulemaking to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods. The EPA is not applying this rulemaking to permit proceedings, site-specific actions, or enforcement actions because these actions are typically focused on individual regulated entities. The rule also does not apply to guidance documents.
The EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. If implementing this procedural rule would result in conflicts with existing environmental statutes, and their implementing regulations, this rule will yield to the EPA statutes and regulations.
Comment: Commenter (11892) asks whether individual permits would be considered significant regulatory actions, as attempting to apply this rule would place a large burden on state agencies. The commenter notes that "the statement in Section §30.5 `except where explicitly stated otherwise that provisions of this subpart do not apply to any other type of Agency action, including individual party adjudications, enforcement activities or permit proceedings' should be interpreted to mean that that individual permits are not covered by this rule" because there are already "adequate appeal processes and public involvement steps in place for challenging the validity of an agency's decision without adding the additional steps and criteria in the proposed rule." 
Response: The EPA is finalizing the applicability of the rulemaking to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods. The EPA is not applying this rulemaking to permit proceedings, site-specific actions, or enforcement actions because these actions are typically focused on individual regulated entities.
The EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. If implementing this procedural rule would result in conflicts with existing environmental statutes, and their implementing regulations, this rule will yield to the EPA statutes and regulations.
Comment: Commenter (12727) asserts that there are many circumstances under which EPA develops [Influential Scientific Information (ISI)] with broad applicability to multiple programs or varying governmental actions that have yet to be identified. The commenter provides that such products include IRIS chemical reviews by EPA's Office of Research and Development (ORD) (e.g., the 2019 "IRIS Toxicological Review of Ethyl Tertiary Butyl Ether (ETBE)" and "IRIS Toxicological Review of Tert-Butyl Alcohol (TBA)"), and reports by EPA's Office of Chemical Safety and Pollution Prevention (e.g., "Continuing Development of Alternative High- Throughput Screens to Determine Endocrine Disruption, Focusing on Androgen Receptor, Steroidogenesis, and Thyroid Pathways"). Additionally, the commenter states that EPA's application of the proposed public disclosure regulations to such products is especially pernicious because it would likely be impossible -- or at least, immensely burdensome -- to determine at the time that the ISI is utilized in Agency decision- making whether, in developing the ISI, EPA omitted from consideration (or gave lesser weight to) valid scientific studies that EPA is obligated to consider fully under the relevant statutory or regulatory provisions.
For example, the commenter (12727) provides that the purpose of ORD's IRIS chemical reviews is to support EPA's mission of protecting public health and the environment by identifying and characterizing the health hazards of chemicals found in the environment. Among other things, the commenter states the EPA uses this information to help set national standards under TSCA, the CAA, the SDWA, and the Emergency Planning and Community Right-to-Know Act, and to clean up hazardous sites under Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA)., but none of these statutes allow EPA to ignore valid, pivotal scientific studies based solely on the fact that underlying data or models are not publicly available. According to the commenter, if the ORD is forced to exclude consideration of such studies when developing its IRIS chemical reviews, EPA's reliance on these reviews when taking actions under these statutes would be unlawful and arbitrary unless EPA combs through the report and available science to confirm that all valid studies have been considered. Additionally, the commenter notes that the need to undertake such a comprehensive review of ISI before relying upon it would fundamentally undermine the purpose of preparing ISI for general use by EPA's various programs, offices, and regions.
Conversely, commenters (11499, 12454) promote the application of the rule to EPA's IRIS program. Specific comments include:
Commenter (11499) states that the lack of full transparency in the IRIS program's development of toxicity values, as well as the inability to independently reproduce some of those values, continues to present obstacles to ensuring sound scientific decisions. The commenter asserts that applying the principles of the transparency rule to IRIS assessments and allowing the public early access to the data and methods used by the Agency is essential to safeguarding transparency in significant regulatory actions that rely upon IRIS toxicity values. According to the commenter, the EPA should make clear that the final transparency rule applies to data and models underlying pivotal science in IRIS assessments.
Commenter (12454) urges EPA to apply the transparency rule to EPA's IRIS program and clearly states as such when it promulgates the final rule. The commenter provides that EPA acknowledges that IRIS assessments "[a]re the preferred source of toxicity information used by EPA" and are used in "set[ting] national standards and clean[ing] up hazardous sites" even though the IRIS program was not established by statute. Due to this use, the commenter presumes that IRIS assessments and the toxicity values derived in those assessments would be considered "influential scientific information" within the meaning of the proposed 40 C.F.R. Section 30.2, since EPA "reasonably can determine [that IRIS assessments and toxicity values] will have ... a clear and substantial impact on important public policies or private sector decisions." The commenter states that the lack of full transparency in the IRIS program's development of toxicity values, as well as the inability to independently reproduce some of those values, continue to present obstacles to ensuring sound scientific decisions and warrant application of the transparency rule.
Response: The EPA is finalizing the applicability of the rulemaking to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods. The EPA is not applying this rulemaking to permit proceedings, site-specific actions, or enforcement actions because these actions are typically focused on individual regulated entities.
The EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. If implementing this procedural rule would result in conflicts with existing environmental statutes, and their implementing regulations, this rule will yield to the EPA statutes and regulations.
Comment: Commenter (11894) notes that despite the ambiguity of the definition of "agency action," it seems clear that EPA would regard pesticide registration as one of the "agency actions" to which its new rules would not apply. The commenter questions the legality of this.
According to the commenter (11894), first, under FIFRA, an applicant has the burden of showing that a pesticide meets the standard for registration. The commenter asserts that the stakes in that decision are great, since approval of a pesticide is approval for the release into the environment of a chemical specifically designed to kill living things. The commenter provides that EPA never explains how it can justify letting an applicant support licensing of such a chemical by submitting data that the Agency could not rely on for its own regulatory decisions often aimed at much smaller potential hazards. Second, the commenter states that given the scope of the information that must often be reviewed to grant or sustain a registration, it may well be that applicants will rely - directly or indirectly - on studies or models that EPA has already found do not meet the new disclosure requirements, or that would clearly fail to meet them.
The commenter (11894) notes that they are provided no indication of how EPA would react: Would a study that EPA has barred from use in other contexts still qualify to support a pesticide registration? Would any special waiver proceedings be required? Would that affect its acceptability in other contexts?
The commenter (11894) contends that these are important questions of general significance to all EPA regulatory decisions that rest in part on studies with broad social impact, some of which may be impacts on regulation, but most of which are not. The commenter maintains that such studies are relevant to EPA decisions far beyond pesticides.
Response: The EPA is finalizing the applicability of the rulemaking to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods. The EPA is not applying this rulemaking to permit proceedings, site-specific actions, or enforcement actions because these actions are typically focused on individual regulated entities.
The EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. If implementing this procedural rule would result in conflicts with existing environmental statutes, and their implementing regulations, this rule will yield to the EPA statutes and regulations.
Comment: Commenter (11491) states that the TSCA §5 requires EPA to review the likelihood that a new chemical will present an unreasonable risk to human health or the environment before it can be marketed. At present, much of the information that EPA reviews rests on CBI or otherwise would not meet the requirements of the proposal. To fill some of these gaps, EPA relies on models that likewise would often not meet those requirements.
Commenter (11894) states that TSCA requires the EPA to review the safety of all chemicals deemed "High Priority" now on the market and to do so by set deadlines. The commenter states that this is a formidable task, as the current number is approximately 45,000. To accomplish this, the commenter provides that the EPA has established a three-stage process designed to review and evaluate all the scientific information relevant to a chemical, and then gradually narrow the number to those that might warrant regulatory action. The commenter notes that comprehensive surveys that EPA is undertaking would inevitably rely in part on studies and models that will not meet EPA's proposed new requirements. Accordingly, the commenter states that whether, and to what extent, EPA's proposals would apply to this program is of obvious critical relevance to its successful operation. According to the commenter, that question cannot be answered from the proposal, which does not discuss that impact in any way. In all those cases where the review does not result in formal regulatory action, the new rules would presumably apply unless EPA adopted a very broad definition of "agency action." The commenter states that such a broad definition would have to be broad enough to cover studies performed for completely non-regulatory purposes - perhaps as part an independent university research program or studies done in other countries - that became part of EPA's review. Under that approach, the commenter suggests that such a study would be covered by the new rules in some circumstances, but not in others. The commenter asks: Does EPA intend this? Is it even workable?
According to the commenter (11894), if the new rules would apply, EPA has a legal obligation to evaluate the new burdens it would place on the program, the resources that would be available to bear them, and the impact on EPA's ability to meet the statutory deadlines. To do this, the commenter states that the EPA would have to define exactly how the new requirements would work in practice. The commenter asks: At what point in this multi-stage process of evaluation would the restrictions on the studies and models that can be considered be applied? What would be the specific reason for that application point? How could that application point be defended given the clear desire of Congress for a comprehensive survey of all data relevant to a chemical?
Response: The EPA is finalizing the applicability of the rulemaking to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods. The EPA is not applying this rulemaking to permit proceedings, site-specific actions, or enforcement actions because these actions are typically focused on individual regulated entities.
The EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. If implementing this procedural rule would result in conflicts with existing environmental statutes, and their implementing regulations, this rule will yield to the EPA statutes and regulations.
14.2.6.5 Other Government Data Sharing Policies
Comment: Commenter (11479) notes that the 2018 proposed rule claimed that it "builds upon prior EPA actions in response to government-wide data access and sharing policies," citing EPA's 2016 Plan to Increase Access to Results of EPA-Funded Scientific Research, the EPA's Scientific Integrity Policy, the Open Data Implementation Plan, the EPA Open Government Plan 4.0, and the Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental Protection Agency. The commenter provides that in EPA's proposed rule and supplemental notice, no details on how this policy will expand upon those established internal policies, nor justifies why this policy is going through the formal rulemaking process when all these efforts to improve transparency and access to data are already underway. According to the commenter, these existing policies largely govern science and policy procedures within EPA; in contrast, this proposed rule attempts to dictate how science is produced outside the Agency, clearly stepping beyond the provisions of current EPA policies and practices.
Response: This rule exclusively pertains to the internal practices of the EPA and does not regulate the conduct or determine the rights or obligations of any entity outside the federal government.
The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Comment: Commenter (12720) states that the Agency also fails to provide notice to other agencies within the federal government of new obligations placed on them as a result of the Supplemental Proposal's tiered access regime. The commenter provides that EPA evidently envisions that the CDC will play a major role in implementation the Supplemental Proposal -- a role which goes essentially unmentioned in the rulemaking according to its memo to the House Committee on Space, Science, and Technology. According to the commenter, the memo notes that EPA staff were "inconsistent concerning how the data and models would be managed," and discussed a pilot project the Agency is conducting with the CDC. Further, the commenter provides that the memo notes that EPA "raised the possibility that the data could be stored on multiple outside "secure enclaves," managed by CDC or another third party, who would also be responsible for reviewing research proposals" and that "this is the first time the Agency has described an implementation scenario that would potentially place such a large responsibility on an outside agency."
Response: This internal procedural rule does not place obligations on any federal government agency besides EPA. The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (12546) provides that the supplemental proposal differs from the 2016 EPA Plan in its ability to be applied to previously generated data while the 2016 EPA Plan only applies to information generated after its implementation. Of the two applications, the commenter contends that the 2016 EPA Plan's limitation to prospective studies is appropriate.
Response: The EPA Plan to Increase Access to Results of EPA-funded Scientific Research, requires that the Agency make the peer-reviewed journal articles and underlying data resulting from EPA-funded research available to the public.
The final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedure and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
The final rule applies prospectively to significant regulatory actions and influential scientific information and has no retrospective effect on existing significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data.
14.2.6.6 Inability for Public to Adequately Comment
Comment: Commenter (12720) contends that the "clarifications" provided to House staff by EPA all but guarantee the arbitrary effect of the Proposal. According to the commenter, researchers, scientists, and fellow executive agencies are certainly entitled to receive notice that EPA intends the burden of compliance with the Proposal to fall on them and to incorporate their concerns about that approach into their comments on the rule. The commenter states that EPA's failure to disclose these elements of the Proposal has foreclosed that possibility. The commenter provides that the fact that details were provided to congressional staff during closed-door briefings, and memorialized in a non-binding House staff memorandum, is certainly insufficient to allow for statutorily-required public comment, and Agency consideration of those public comments.
The commenter (12720) asserts that neither the research community (nor the CDC) can be expected to meaningfully comment on this proposal and its imposition on their research practices without such detail. The commenter states that the fact that this emerging element of the Proposal imposes requirements on third parties -- researchers, scientists, as well as the CDC -- makes clear that the Proposal is not merely a "a proposed internal rule of agency procedure," to which the so-called Housekeeping Statute on which EPA relies might apply.
Commenter (12727) provides that by failing comprehensively to describe the scope of its proposal, EPA has denied the public the information it needs to comment in an informed way.
Response: EPA disagrees that the public was deprived of the opportunity to provide meaningful comments. The comment period for the proposal and supplemental totaled 168 days and the EPA received over 993,000 public comments.
The rule exclusively pertains to the internal practices of the EPA and does not regulate the conduct or determine the rights or obligations of any entity outside the federal government. Any incidental impacts on voluntary behavior outside of the EPA do not render this a substantive rule.
14.2.6.7 Implementation Guidance
Comment: Commenter (11358) encourages EPA to follow the advice outlined in the SAB's April 24, 2020 comments to EPA suggesting the Agency develop a guidance document before implementing the final rule. The commenter agrees with the board's suggestion to include further context for the definitions included within both the proposal and supplemental notice and that it "may be beneficial for EPA to develop a guidance document with case studies based on past risk assessments to clarify some scenarios and how the requirement to make data and models publicly available in a manner sufficient for independent validation would be managed (p. 13)." The commenter also supports the SAB's recommendation that EPA
      consider seeking input from experts in library science, data curation management, and data retention to identify best practices and tools to ensure efficiency and utility of data that are made available (p. 14).
Response: This final rule is effective at publication in the Federal Register. The implementation guidelines are forthcoming.
Comment: Commenter (11381) states that Proposed 40 CFR 30.1 in the NPRM "directs EPA to ensure that the regulatory science underlying its actions is publicly available in a manner sufficient for independent validation." According to the commenter, the SNPRM is not as clear because it includes only those circumstances "where the Agency is making data or models publicly available".
The commenter (11381) maintains that elsewhere in proposed new Part 30 there is no text that would ensure that this rule, regardless of its final text, is ever implemented. The commenter provides that rules without means of enforcement generally are hortatory or aspirational, not regulatory. The commenter notes that it is possible, of course, that the public could persuade a court to vacate and remand a substantive regulation on the ground that EPA had failed to comply with new Part 30. The commenter states that on the other hand, courts have only once interpreted the Information Quality Guidelines (IQG) as binding even though there typically is an ample administrative record on which a determination of noncompliance with the IQGs could be based.
Comment: Commenter (11894) maintains that as with EPA's initial proposal, EPA did not provide any analysis explaining how a bar on influential studies would work in the context of specific EPA programs. The commenter states that EPA does not discuss how its proposal might actually work in practice. The commenter contends that EPA has not examined the extent of any additional resource needs or the impact on statutory deadlines, nor has EPA examined even the most obvious of the conceptual problems that would arise. The commenter provides that for example, if an exempt Agency action relied on a scientific study that, in any other context, would be classified - or perhaps had already been classified - as "influential scientific information," how would that data be handled? Similarly, the commenter notes that EPA often relies on states to take regulatory actions that would be exempt from coverage if EPA undertook them itself and asks how would the rule treat any guidance that EPA might issue to advise states on how to take these actions?
Response: This final rule is effective at publication in the Federal Register. The implementation guidelines are forthcoming.
Under the final rule no studies are categorically excluded from consideration and the EPA will continue to assess the full body of science to evaluate the quality and transparency of available science for final significant regulatory actions and influential scientific information. The EPA will also consider the degree to which pivotal science is available and give greater consideration to those studies with greater data and model availability, as described in this rule.
This final rule does not interpret or apply the provisions of any environmental statutes. If implementing this procedural rule would result in conflicts with existing environmental statutes, the procedural rule will yield to the existing statutes.
Comment: Commenter (12720) quotes the SAB's 2020 Letter to Administrator Wheeler concerning the proposal and supplemental proposal in its call that "the EPA clarify the definitions of "data and models" and "pivotal regulatory science." The commenter suggests that it would be useful to develop a guidance document that includes examples of the types of data and models of interest and requirements for reporting this information. The commenter adds that it would also be useful to clarify specific requirements for reporting information from animal toxicity and/or environmental epidemiology studies. In this statement, the commenter asserts that the definition of data fails to clarify what is covered by the supplemental proposal, leading to difficulties ascertaining the rule's effects.
Response: In the final rule the EPA is removing the term "pivotal regulatory science" and combining the definitions of "pivotal science" and "pivotal regulatory science" under the single term "pivotal science". Pivotal science means the specific dose-response studies or analyses that drive the requirements or quantitative analyses of EPA significant regulatory actions or influential scientific information.
In the final rule, "Data means the set of recorded factual material commonly accepted in the scientific community as necessary to validate research findings in which obvious errors, such as keystroke or coding errors, have been removed and that is capable of being analyzed by either the original researcher or an independent party." The definition of model has been removed from the final rule. 
This final rule is effective at publication in the Federal Register. The implementation guidelines and statute-specific transparency regulations or programmatic regulations are forthcoming.
Comment: Commenter (10042) urges EPA to codify its approach for how, when and under what circumstances it will use human health data and information in and for rulemaking and setting standards. The commenter adds that EPA should also adopt and develop standard operating procedures (SOPs) for managing human health data and information which will likely have a strong anonymization component that can or will be used to make human health risk assessments.
Response: The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available.
Comment: Commenter (12699) states that the EPA should develop objective criteria to determine when a study should be excluded. The commenter states that these criteria are in effect a means to determine when there are not genuine privacy, confidentiality, and CBI concerns; these criteria should include:
 The authors simply do not want to provide the underlying information.
 The authors have already disclosed the underlying information to third parties. This should be prima facie evidence that the information can be made available to the public.
 The EPA has disclosed the underlying information to a third party. This too should be prima facie evidence the information should be made available to the public. If the disclosure was only made because the third party is an agent (e.g. contractor) of the EPA, then the EPA should make comparable arrangements for other third parties to review the underlying information.
 The study authors or the EPA have not exhausted all available commonly used practices to de-identify information (if such de-identification is required).
Response: The final rule is limited to dose-response data and, as no studies are categorically excluded from consideration, the EPA will continue to rely on the full body of the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Consistent with the requirements of this rule, the EPA will identify a subset of those studies based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints that would inform a dose-response assessment, and will give greater consideration to pivotal science for which the underlying dose-response data are available.
Section 30.5 of the final rule states, "The EPA will use the following factors to assess the quality of studies identified in the systematic review: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. . . . The Agency will identify the pivotal science that was given lesser consideration and provide a short description of why lesser consideration was given."
Comment: Commenter (12715) states that, if the EPA adopts this proposal, it should at least make its process for excluding studies transparent; any final rule should require that in any future rulemaking or standard setting, the EPA will clearly identify each study that it rejected based on the present rule and fully explain that the basis for the rejection.
Response: The final rule is limited to dose-response data and, as no studies are categorically excluded from consideration, the EPA will continue to rely on the full body of the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Consistent with the requirements of this rule, the EPA will identify a subset of those studies based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints that would inform a dose-response assessment, and will give greater consideration to pivotal science for which the underlying dose-response data are available.
Comment: Commenter (11473) states that the EPA proposed rule does not account for conflicts of interest (and in particular financial conflicts of interest) by those requesting access to data, opening the possibility for research data that are to be used for regulatory purposes to be accessed by anyone, regardless of their affiliation. The commenter states that, thus, it will not prevent agents of vested interests from obtaining the publicly available data demanded by this proposed rule, and then running their own studies with the intention of discrediting science. The commenter states that there is already empirically based evidence that results of studies conducted by the pharmaceutical and tobacco industry favor those industries; indicating that the commenter anticipates self-serving biased analyses of EPA provided data by vested interest funded analysts under this proposed EPA approach (White and Bero, 2010; Bero, 2017; Lundh et al., 2017). The commenter states that it has been previously documented in an extensive review of a similar proposal (upon which this EPA proposal is based) made by a vested interest (a tobacco company): when such data have been accessed using a similar provision in the state of Georgia, the data were abused in an attempt to discredit the research they did not like, but the original research was later replicated by other researchers (Thurston, 1998) (attached). The commenter states that this new attempt to implement the tobacco company-inspired data release proposal will undoubtedly have similar inappropriate and untoward damage to scientific research if this so-called "Transparency" rule is implemented.
Commenter (12546) states that, considering that EPA as of yet has failed to detail a strategy on monitoring and accounting for financial conflicts of interest, a potential influx of industry-funded studies with such conflicts of interest may bias the final results of EPA's analysis, leading to less stringent regulations and policies -- with the ultimate result of reducing protections for the health of families.
Response: The EPA does not agree that its approach will lead to systematic bias towards certain types of stakeholder goals. The final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available.
 Chapter 15: Statutory and Executive Order Reviews
The EPA provides documentation of their consideration of several Executive Orders in the development of the proposal in Section IV of the notice of proposed rulemaking (NPRM) preamble and Section VII the supplemental notice of proposed rulemaking (SNPRM). Comments provided on EPA's consideration and/or compliance with Executive Orders are organized as follows:
Section 15.1: NPRM Executive Orders
       Executive Order 12291, Federal Regulation 
       Executive Orders 12866, Regulatory Planning and Review, and 13563, Improving Regulation and Regulatory Review 
       Executive Order 12898 - Federal Actions to Address Environmental Justice in Minority Populations and Low-Income Populations 
       Executive Order 13045  -  Protection of Children 
       Executive Order 13132, Federalism 
       Executive Order 13175 Consultation and Coordination with Indian Tribal Government 
       Executive Order 13556, Controlled Unclassified Information 
       Executive Order 13771, Reducing Regulation and Controlling Regulatory Costs 
       Executive Order 13777, Enforcing the Regulatory Reform Agenda 
       Executive Order 13783, Promoting Energy Independence and Economic Growth 
Section 15.2: SNPRM Statutory and Executive Order Requirements
       Executive Orders 
       Paperwork Reduction Act 
       Additional Related Comments 
The specific comments provided in this Chapter reflect the comment letters that provide arguments supported by detailed rationale, examples, and references/documentation. The Docket ID numbers for these commenters are cited throughout this Chapter. In addition, some general comment letters may also include comments related to comments included and responded to in this Chapter. These comments are summarized in Appendix B. Note that points and issues raised in these general comment letters may also be raised by the more detailed comments summarized in this Chapter. The responses provided in this Chapter adequately address these more general comments.
15.1 NPRM Executive Orders
15.1.1 Executive Order 12291: Federal Regulation
Comment: Commenters (6924, 8281-PH20) assert that EPA does not present any analysis of cost-benefit in support of the rule which is required under Executive Order 12291. Commenter (6924) states that Executive Order 12291 requires Agencies to complete a benefit-cost analysis for any major rule which has an annual impact of $100 million or more. 
Response: Executive Order 12291 was revoked in 1993 by President Bill Clinton (D), who issued a modified regulatory review program under Executive Order 12866. 
15.1.2 Executive Orders 12866: Regulatory Planning and Review, and 13563: Improving Regulation and Regulatory Review
 15.1.2.1 Executive Order 12866: Regulatory Planning and Review
15.1.2.1.1 Costs and Benefits
Comment: Commenters (2171, 5181, 6133, 6137, 6168, 6188, 6916, 6919, 9227, 11408) assert that EPA has failed to meet Executive Order 12886 requirements to analyze and provide an assessment of the proposed rule's costs and benefits. Commenters (6916, 9227) state that Executive Order 12866 requires agencies to assess the costs and benefits of proposed regulations and propose or adopt a regulation only upon a reasoned determination that the benefits justify the costs. 
Response: This is a rule of internal procedure promulgated under the EPA's housekeeping authority and imposes no enforceable duty on any state, local, or tribal governments or the private sector. Additionally, this rule does not require the EPA or other parties to disclose or host data. Hence, this rule does not impose costs on the EPA or any other party to make such data available. As described in Section III.J of the final rule preamble, the EPA recognizes there will be some incremental review and administrative costs to the Agency but anticipates these will be small.
15.1.2.1.2 Need for New Regulation
Comment: Commenter (9227) states that EPA's failure to identify a problem or inadequacy that new regulations are needed to address is contrary to Executive Order 12866. The commenter states that, before proceeding with this proposal, EPA should clearly identify the problem it is trying to solve, provide evidence that there is a problem and allow for public comment on whether a problem exists that could be addressed through EPA regulation. The commenter states that Executive Order 12866 provides that "[f]ederal agencies should promulgate only such regulations as are required by law, are necessary to interpret the law, or are made necessary by compelling public need, such as material failures of private markets to protect or improve the health and safety of the public, the environment, or the well-being of the American people." The commenter states that Executive Order 12866 directs each agency to "identify the problem that it intends to address (including, where applicable, the failures of private markets or public institutions that warrant new agency action) as well as assess the significance of that problem." 
Response: This is an internal housekeeping rule that does not impose an enforceable duty on any state, local, or tribal governments or the private sector. The purpose of the rule is detailed in the preamble of the final rule. In brief, the transparency provisions in this final rule are intended to build upon existing efforts and provide incremental progress toward the Agency's overarching goal of greater transparency. The EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. The scientific community has also embraced greater data transparency, as evidenced by data sharing and availability requirements for many high-impact journals and the emergence of several international organizations and initiatives focused on increased transparency and data sharing in science. The EPA supports these efforts and believes that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. The incremental progress made by this rule provides an important step towards prioritizing transparency in particularly impactful rules and assessments and will inform future statute-specific rulemakings.
15.1.2.1.3 Regulatory Impact Analysis
Comment: Commenters (0568, 2171, 6446, 6916, 6919) express concern that EPA failed to prepare a Regulatory Impact Analysis (RIA), which is necessary to determine the impacts of the proposal. Commenters (2171, 6919) state that this action is correctly identified as a "significant regulatory action" under Executive Orders 12866 and 13563 which then requires an RlA. 
Response: This is a rule of internal procedure promulgated under the EPA's housekeeping authority and imposes no enforceable duty on any state, local, or tribal governments or the private sector. Additionally, this rule does not require the EPA or other parties to disclose or host data. Hence, this rule does not impose costs on the EPA or any other party to make such data available. As described in preamble discussion of costs and benefits, the EPA recognizes there will be some incremental review and administrative costs to the Agency but anticipates these will be small.
15.1.2.2 Executive Order 13563: Improving Regulation and Regulatory Review
Comment: Commenters (6133, 6188, 9227) state the EPA did not provide cost-benefits analysis for the proposal, which violates Executive Order 13563. Commenter (9227) states that Executive Order 13563 reaffirms the principles and requirements of Executive Order 12866, explaining that agencies "must take into account benefits and costs, both quantitative and qualitative." 
Response: This is a rule of internal procedure promulgated under the EPA's housekeeping authority and imposes no enforceable duty on any state, local, or tribal governments or the private sector. Additionally, this rule does not require the EPA or other parties to disclose or host data. Hence, this rule does not impose costs on the EPA or any other party to make such data available. As described in preamble discussion of costs and benefits, the EPA recognizes there will be some incremental review and administrative costs to the Agency but anticipates these will be small. 
Public input was also an important part of shaping the final rulemaking. The EPA accepted public comment on the proposed rule from April 30, 2018 to August 16, 2018, and held a public hearing on July 17, 2018. In response to public comments, the EPA released a supplemental notice of proposed rulemaking and took public comment from March 18, 2020 to May 18, 2020. In addition, the EPA's SAB provided advice and comments on mechanisms for secure access to personally identifying information (PII) and confidential business information (CBI) under the proposed rule on September 30, 2020, and the proposed and supplemental rulemakings on April 24, 2020. All of these comments were considered in developing the final rule.
Comment: Commenters (6125, 6133, 6137, 9227) assert that, while EPA states in the preamble that "[t]he best available science must serve as the foundation of EPA's regulatory actions," under Executive Order 13563, the proposed rule actually hinders EPA's use of the best available science. 
Commenter (6125) states that, while Executive Order 13563 states that EPA shall base its regulations on the best available science, EPA's proposal would make this impossible by making wide areas of such science unavailable to regulatory decision makers.
Commenter (6133) asserts that, while the proposal cites and quotes from Executive Order 13563, the Executive Order not only does not support the proposal, it directly undermines the proposal. The commenter states that there is no suggestion in the cited Executive Order that "best available science" means or meant that science underlying an agency's actions must be publicly available in a manner sufficient for independent validation, nor that "pivotal regulatory science" has any meaning akin to the proposed uses in proposed 40 CFR 30.3. The commenter states that, to the contrary, no previous administration has conditioned any notion of "best available science" on the public availability of underlying data, or on the concepts behind the invented term, "pivotal regulatory science." The commenter states that EPA previously routinely used and considered science and studies for which the underlying data was not publicly available as examples of the "best available science." The commenter states that EPA did so for proposed and final regulations, along with other final agency actions, reports, studies and the like. EPA's use and consideration of such science was validated by EPA's science advisory bodies, the National Academy of Sciences (NAS), the SAB, and other scientific organizations. The commenter states that the proposal does not provide sufficient explanation for its departure from this past practice.
Commenter (6137) states that the proposal has the purpose and effect of precluding the use of some of the best available science, that is, all science based on non-public data, simply because the underlying data is not publicly available. The commenter states that best available science means "all existing scientific evidence relevant to the decision," and agencies simply "cannot ignore existing data." The commenter states that, however, the proposed rule requires EPA to ignore existing data if it is not available for public release. The commenter concludes that the proposed rule is inconsistent with Executive Order 13563, as well as those statutes or principles that require EPA to consider the best available science.
Commenter (6137) asserts that an Executive Order cannot override a statute, limit the delegated authority and the legal responsibilities provided to the EPA Administrator by federal law, add factors that are impermissible under the statute, or delay statutorily required agency action. The commenter states that weakening or delaying public health protections based on an executive order would be unconstitutional, violating separation of powers and the requirement to follow duly enacted laws passed by Congress and signed by the President. The commenter states that it would likewise be unlawful, contrary to the public health obligations and rulemaking requirements of EPA's governing statutes. The commenter states that EPA cannot consider or apply any other executive order in any way in this rulemaking without providing the requisite public notice and opportunity for comment that the Clean Air Act (CAA) and Administrative Procedure Act (APA) or proposal for why EPA believes this policy would further the goal of best available science. The commenter states that the Executive Order only states that agencies should make available to the public the scientific or technological findings or conclusions on which rules rely, as opposed to underlying raw data that EPA has targeted with this proposal. 
Response: As further detailed in the preamble of the final rule, the transparency provisions in this final rule are intended to build upon existing efforts and provide incremental progress toward the Agency's overarching goal of greater transparency. The EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. The scientific community has also embraced greater data transparency, as evidenced by data sharing and availability requirements for many high-impact journals and the emergence of several international organizations and initiatives focused on increased transparency and data sharing in science. The EPA supports these efforts and believes that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. The incremental progress made by this rule provides an important step towards prioritizing transparency in particularly impactful rules and assessments and will inform future statute-specific rulemakings.
This rulemaking is not intended to modify the Agency's interpretations of "best available science." The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. The EPA will then identify and consider pivotal science in accordance with the provisions of this rule, unless the implementation of the rule conflicts with statutory requirements and associated implementing regulations. As this rule makes clear, if implementing this procedural rule would result in conflicts with existing environmental statutes, the procedural rule will yield to the existing statutes.
The final rule requires EPA to follow best practices to evaluate potential links between exposure to a pollutant, contaminant, or substance and the effect and the nature of the dose-response relationship. EPA will continue to use the following, well-established factors when assessing study quality: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. As further detailed in the preamble to the final rule, only when characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect, will the EPA identify pivotal science and give greater consideration to pivotal science for which the underlying dose-response data are available in a manner sufficient for independent validation. Title 40 CFR 30.5 also provides additional considerations for instances when underlying dose-response data are not available for independent validation. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
15.1.3 Executive Order 12898: Federal Actions to Address Environmental Justice in Minority Populations and Low-Income Populations 
Comment: Commenters (2171, 2337, 5178, 6122, 6133, 6137, 6144, 6166, 6188, 6194, 6373, 6874, 6915, 6916, 6920, 6924, 6939, 8272, 8281-PH20) assert that the proposed rule is subject to Executive Order 12898 and needs an environmental justice analysis. Commenter (6920) urges the EPA to consider the environmental justice implications of the rule and examine ways to further protect vulnerable people and disadvantaged communities, in accordance with Executive Order 12898.
Commenters (2171, 2337, 6133, 6137, 6144, 6188, 6194, 6373, 6915) do not agree with the proposal's statement that it does not need to address Executive Order 12898 because "it does not establish an environmental health or safety standard." 
Commenters (2337, 6133, 6137, 6373, 6915) assert that Executive Order 12898 by its terms applies to Agency policies and Agency conduct of programs as well as to Agency adoption of regulatory standards. Commenter (2337) states that EPA's determination that the proposed rule is not subject to Executive Order 12898 because "it does not establish an environmental health or safety standard" too narrowly construes the scope of the Executive Order and the potentially broad impact of its proposed rule. 
Commenters (2171, 6133) state the implementation of this rule would indirectly impact the rules and guidelines that are set to protect children, people of color, the elderly, low-income, and other underserved populations. Commenter (6133) asserts that, as the proposed rule concerns the scientific evidence EPA will consider in its regulatory decision making, including epidemiological and clinic research on the health impacts of pollutants, it has the potential to directly implicate the concerns addressed by Executive Order 12898. 
Commenter (6137) asserts that EPA's claim that this proposed rule does not require an environmental justice assessment is clearly at odds with what the Agency itself recognizes it must do to comply both with Executive Order 12898, as well as its own policies. The commenter states that EPA has regularly and purposefully focused on the need for environmental justice assessments of its rulemakings. The commenter states that EPA consistently performs this sort of assessment when acting under the CAA, Clean Water Act (CWA), and other statutes. The commenter states that this information unquestionably has been relevant to public health rulemakings in the past. The commenter states that EPA's refusal to consider the environmental justice consequences of the proposed rule  -  consequences that are certainly "relevant" under State Farm  -  and its departure from its long-standing pattern and practice of considering such data without a reasoned explanation for changing course are therefore arbitrary and capricious. 
Response: This action is not subject to Executive Order 12898 because it does not establish an environmental health or safety standard. This rule exclusively pertains to the internal practices of the EPA and does not regulate the conduct or determine the rights or obligations of any entity outside the federal government. In addition, the EPA does not agree that its approach will lead to systematic bias towards certain types of stakeholder goals of any party. As described in the preamble, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information.
Comment: Commenter (6122) states that the proposal directly violates Executive Order 12898, which directs every federal agency to "...collect, maintain, and analyze information assessing and comparing environmental and human health risks borne by populations identified by race, national origin, or income. To the extent practical and appropriate, Federal agencies shall use this information to determine whether their programs, policies, and activities have disproportionately high and adverse human health or environmental effects on minority populations and low-income populations." The commenter states that, in 2011, the EPA also signed and Memorandum of Understanding to implement the environmental justice goals of Executive Order 12898. The commenter states that EPA must ensure that high quality data analyze environmental justice impacts in order to protect vulnerable populations. The commenter states that requiring public availability of raw data is not a rational metric for deciding whether a study is of high enough quality to be used for setting thresholds or enacting regulatory protections.
Commenter (6373) asserts that that Executive Order 12898 specifically addresses federal human health and environmental research and analysis, requiring federal agencies to ensure that epidemiological and clinical research activities represent "segments at high risk from environmental hazards, such as minority populations, low-income populations and workers who may be exposed to substantial environmental hazards." The commenter states that Executive Order 12898 explicitly contemplates that agencies will use epidemiological data collected from at-risk communities, which necessarily relies on private information of members from those communities, to promulgate regulations to protect those at risk.
Commenter (6194) states that EPA has not addressed how the proposal would affect the agency's fulfillment of the human health and environmental data collection and analysis obligations imposed by Executive Order 12898 (§ 3-302).
Commenter (6133) expresses concern that the proposal makes no mention of the EPA Guidance on Considering Environmental Justice During the Development of Regulatory Actions. The commenter states that the proposal directly conflicts with many of the Guidance document's provisions. The commenter states that to achieve Executive Order 12898's goals, the Guidance directs rule-writers and decision-makers to respond to three core Environmental Justice questions throughout the process: 
 How did the public participation process provide transparency and meaningful participation for minority populations, low-income populations, tribes, and indigenous peoples? 
 How did the rule-writers identify and address existing and/or new disproportionate environmental and public health impacts on minority populations, low-income populations, and/or indigenous peoples? 
 How did actions taken under #1 and #2 impact the outcome or final decision? 
Commenter (6137) asserts that the proposal violates the environmental justice requirements of Executive Order 12898 because it ignores EPA's own environmental justice plan  -  promulgated pursuant to Executive Order 12898  -  the ultimate vision of which is for EPA to "integrate[] environmental justice into everything" it does. The commenter states that to accomplish this vision, EPA sets forth eight different priority areas, the first of which is rulemaking. The commenter states that EPA aims to "institutionalize environmental justice in rulemaking," including performance of "rigorous assessments of environmental justice analyses in rules," in order to "deepen environmental justice practice within EPA programs to improve the health and environment of overburdened communities." The commenter states that, recognizing that "[r]ulemaking is an important function used by the EPA to protect human health and the environment for all communities," EPA devotes the second chapter of the plan to "Rulemaking," and through this chapter, aims to "ensure environmental justice is appropriately analyzed, considered, and addressed in EPA rules with potential environmental justice concerns, to the extent practicable and supported by relevant information and law." The commenter states that consistent with its environmental justice plan and with Executive Order 12898, EPA issued its own Guidance on Considering Environmental Justice During the Development of Regulatory Actions, recognizing how "vital" it is "that Agency rule-writers identify and address potentially disproportionate environmental and public health impacts experienced by minority populations, low-income populations, and/or indigenous peoples," 
Response: This action is not subject to Executive Order 12898 because it does not establish an environmental health or safety standard. This rule exclusively pertains to the internal practices of the EPA and does not regulate the conduct or determine the rights or obligations of any entity outside the federal government.
Public input was also an important part of shaping the final rulemaking. The EPA accepted public comment on the proposed rule from April 30, 2018 to August 16, 2018, and held a public hearing on July 17, 2018. In response to public comments, the EPA released a supplemental notice of proposed rulemaking and took public comment from March 18, 2020 to May 18, 2020. In addition, the EPA's SAB provided advice and comments on mechanisms for secure access to PII and CBI under the proposed rule on September 30, 2020, and the proposed and supplemental rulemakings on April 24, 2020. All of these comments were considered in developing the final rule.
Comment: Commenter (5178) states that reproductive justice is the human right to maintain personal bodily autonomy, have children, not have children, and parent children in safe and sustainable communities. The commenter states that we cannot fully achieve reproductive justice when environmental exposures make it harder for parents to give birth to healthy children and raise them in safe homes. The commenter states that, in order for women and families to thrive, they must be able to choose to parent without fear of pollution and contamination interfering with their health and well-being. The commenter states that a reproductive justice perspective also recognizes the inequitable environmental burdens placed on communities composed of people of color and those with fewer economic advantages, and that these burdens will continue accumulating over generations if not corrected. The commenter states that EPA has oversight over numerous exposures that pose particular dangers during pregnancy and infancy, including neurotoxicants and endocrine disruptors.
Response: This action is not subject to Executive Order 12898 because it does not establish an environmental health or safety standard. This rule exclusively pertains to the internal practices of the EPA and does not regulate the conduct or determine the rights or obligations of any entity outside the federal government.
Public input was also an important part of shaping the final rulemaking. The EPA accepted public comment on the proposed rule from April 30, 2018 to August 16, 2018 and held a public hearing on July 17, 2018. In response to public comments, the EPA released a supplemental notice of proposed rulemaking and took public comment from March 18, 2020 to May 18, 2020. In addition, the EPA's SAB provided advice and comments on mechanisms for secure access to PII and CBI under the proposed rule on September 30, 2020, and the proposed and supplemental rulemakings on April 24, 2020. All of these comments were considered in developing the final rule.
Comment: Commenter (5178) states that, in the absence of substantial new resources, efforts to replicate studies and prepare data for public release would reduce the resources available for new research and delay the regulatory process. The commenter states that the consequences of this proposal would fall most severely on groups not adequately protected by current rules, which include racial and ethnic minorities, those with low socioeconomic status, the elderly, and pregnant individuals and their eventual children. The commenter states that the theoretical  -  but as yet undemonstrated  -  benefits of EPA's proposal must be weighed against the extensive and unequally distributed costs of such an approach. 
Commenter (6373) asserts that, if EPA disregards findings from certain studies because of the proposed rule, and as a result adopts less protective national ambient air quality standards (NAAQS) for any of the criteria pollutants, then EPA will have violated Executive Order 12898.
 EPA's failure to use data collected from at-risk communities in the promulgation of NAAQS itself would be a violation of Executive Order 12898.
 Certain populations (often low-income, minority groups) are frequently subject to higher exposure to regulated chemicals, and should regulations be weakened because they are not grounded in the best science available, the exposures those communities face may increase and disproportionately burden these communities, which would result in additional adverse health consequences.
 Even when exposed to the same levels of a pollutant, communities with a higher level of disease prevalence (e.g., asthma) may suffer a higher burden if that disease is exacerbated by pollution. Therefore, EPA would fail its mandate to address the disproportionately high human health impact of air pollution.
Commenters (6133, 6137, 6188, 6373) express concern that the proposal does not address public participation of minority and low-income populations, as required under Executive Order 12898.
Commenter (6373) states that Executive Order 12898 requires that "[f]ederal agencies shall provide minority populations and low-income populations the opportunity to comment on the development and design of research strategies undertaken pursuant to this order." The commenter states that, despite the clear applicability of Executive Order 12898 to this action, there is no evidence in the record that EPA solicited input from the environmental justice community when promulgating the rule.
Commenter (6133) asserts that, because minority and low-income populations have historically been underrepresented in agency decision making, Executive Order 12898 aims to improve public participation of these populations in the decision-making process. The commenter states that, with the single English language hearing EPA held in Washington DC, EPA has not provided for meaningful participation of minority populations, low-income populations, tribes, and indigenous peoples. 
Commenter (6137) asserts that, when promulgating rules "that substantially affect human health or the environment"  -  which this proposed rule does  -  EPA must "provide minority populations and low-income populations the opportunity to comment on the development and design of research strategies undertaken pursuant to this order." Commenter (6188) states that minority populations and low-income populations are guaranteed an opportunity to comment on the development and design of research strategies undertaken per the Order's direction. 
Response: This is a rule of internal procedure promulgated under the EPA's housekeeping authorities, and imposes no enforceable duty on any state, local or tribal governments or the private sector. In addition, the EPA does not agree that its approach will lead to systematic bias towards certain types of stakeholder goals of minority and low-income populations. As described in the preamble, the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions and finalizing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. 
Public input was also an important part of shaping the final rulemaking. The EPA accepted public comment on the proposed rule from April 30, 2018 to August 16, 2018, and held a public hearing on July 17, 2018. In response to public comments, the EPA released a supplemental notice of proposed rulemaking and took public comment from March 18, 2020 to May 18, 2020. In addition, the EPA's SAB provided advice and comments on mechanisms for secure access to PII and CBI under the proposed rule on September 30, 2020, and the proposed and supplemental rulemakings on April 24, 2020. All of these comments were considered in developing the final rule. 
15.1.4 Executive Order 13045: Protection of Children 
Comment: Commenters (2171, 5178, 6130, 6133, 6188, 6373, 6915, 6916, 6920, 8281-PH20) assert that the proposed rule is subject to Executive Order 13045 and requires analysis of environmental health risks and safety risks that may disproportionately affect children. Commenters (6133, 6188, 6194) assert that Executive Order 13045 requires that federal agencies "shall make it a high priority to identify and assess environmental health risks and safety risks that may disproportionately affect children...." Commenters (6133, 6373, 6915) assert that Executive Order 13045 requires that every agency shall ensure that its policies, programs, activities, and standards address disproportionate risks to children that result from environmental health risks or safety risks. 
Commenters (6133, 6188, 6194) assert that, in the case of a "significant regulatory action" such as the proposed rule, the promulgating agency must provide an evaluation of health or safety effects of the planned regulation on children. 
Commenters (6133, 6188, 6194) assert that Executive Order 13045 requires an explanation of why the planned regulation is preferable to other potentially effective and reasonably feasible alternatives considered by the agency (Executive Order 13054, § 5-501). Commenter (6188) states that EPA has not provided any such analysis of the proposed rule.
Commenter (6373) states that the proposed rule is in direct contravention of Executive Order 13045 -- complying with the rule will prevent EPA from effectively "ensure[ing] that its policies, programs, activities, and standards address disproportionate risks to children that result from environmental health risks or safety risks." The commenter states that when EPA regulates a substance pursuant to any one of its governing statutes or sets a standard for remedial work on property that presents a risk of exposure to children, Executive Order 13045 requires EPA to identify and assess health risks to children and ensure that its standards and actions protect children. The commenter states that, to do this effectively, EPA must consider scientific studies that include private information about children, however, should the proposal be finalized, EPA would have the discretion to discount scientific studies of exposure risks to children that rely on personal data or confidential information, contrary to Executive Order 13045's mandate. The commenter states that EPA cannot regulate in a matter compliant with both Executive Order 13045 and the proposed rule.
Response: The EPA interprets Executive Order 13045 as applying only to those regulatory actions that concern environmental health or safety risks that the EPA has reason to believe may disproportionately affect children, per the definition of "covered regulatory action" in section 2-202 of the Executive Order. This action is not subject to Executive Order 13045 because it does not concern an environmental health risk or safety risk.
Comment: Commenters (2171, 6130, 6133, 6194, 6373, 6920) do not agree with the proposal's statement that it does not need to address Executive Order 13045 because "this action does not concern an environmental health risk or safety risk" (83 FR 18773). 
Commenters (6130, 6133, 6920) assert that the claim that this action "is not subject to Executive Order 13045 because it does not concern an environmental health risk or safety risk" to children is not supported by scientific evidence. Commenters (6130, 6920) state that there is a substantial body of research showing children are more sensitive than adults to environmental pollution. Commenter (6373) states that EPA's determination that the proposed rule is not subject to Executive Order 13045 because it is not a regulatory action that "concern[s] environmental health or safety risks that the EPA has reason to believe may disproportionately affect children" is arbitrary and is a tragic relinquishment of EPA's obligation to protect our children. 
Commenter (6133) states that, while EPA asserts that the proposal is not subject to Executive Order 13045 because it does not concern an environmental health risk or safety risk, EPA does not explain how it reached this conclusion. The commenter states that EPA also does not characterize children's risk to the extent data are available. The commenter states that the proposal applies to "Pivotal regulatory science" and the proposal defines regulatory science as "scientific information, including assessments, models, criteria documents, and regulatory impact analyses, that provide the basis for EPA final significant regulatory decisions." The commenter states that the proposal explains that "`Pivotal regulatory science' is the studies, models, and analyses that drive the magnitude of the benefit-cost calculation, the level of a standard, or point-of-departure from which a reference value is calculated; in other words, they are critical to the calculation of a final regulatory standard or level, or to the quantified costs, benefits, risks and other impacts on which a final regulation is based." The commenter states that, by its terms, the proposal will impact (and therefore concern) all environmental health and safety risks, including many that EPA knows disproportionately affect children.
Commenter (6194) states that the proposal would affect risk assessment analyses across a variety of environmental statutes. The commenter notes that studies that in the past informed regulations that address public health risks with disproportionate effects on young children, such as asthma from exposure to particulate matter and neurological damage from exposure to lead, could be precluded from consideration today by the proposal. The commenter states that the proposal is "intended to apply prospectively to final regulations that are determined to be `significant regulatory actions' pursuant to Executive Order 12866." The commenter states that, therefore, the proposal itself should be construed as an action subject to the review requirements of Executive Order 13045. The commenter states that this would not present a double-bite at the apple in terms of review under Executive Order 13045; rather it presents the only meaningful opportunity to consider the impact of the proposal on the protection of children. The commenter states that, once the proposal is adopted, it would preclude the use of certain science, so any analysis of feasible alternatives under Executive Order 13045 would already be limited in a way that precludes consideration of the impacts of the proposal.
Response: The EPA maintains that this action is not subject to Executive Order 13045 because it does not concern an environmental health risk or safety risk. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers.
Comment: Commenter (6133) states that EPA created a Guide to help Agency staff involved in developing actions determine whether Executive Order 13045 applies to an Agency action and, if so, how to implement the Executive Order. The commenter states that the Guide includes "a set of questions EPA staff involved in action development can ask risk assessors to ensure that the various types of information relevant to the assessment of risks to children are considered and may be useful in addressing the issue of disproportionate risks." The commenter states that the Guide explains: "[i]f a rulemaking is not covered by Executive Order 13045, but it discusses environmental health or safety, it is advisable to characterize children's risk to the extent the data are available." 
Response: The EPA interprets Executive Order 13045 as applying only to those regulatory actions that concern environmental health or safety risks that the EPA has reason to believe may disproportionately affect children, per the definition of "covered regulatory action" in section 2-202 of the Executive Order. This action is not subject to Executive Order 13045 because it does not concern an environmental health risk or safety risk. In addition, the EPA's approach will not lead to systematic bias towards certain types of stakeholder goals of any party. As described in the preamble, the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions and finalizing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information.
15.1.5 Executive Order 13132: Federalism 
Comment: Commenter (6915) does not agree with the statement in the proposal that this rule would not affect any states, and therefore has no federalism implications. The commenter states that the adoption of this proposed rule would very likely affect the protectiveness of the standards that EPA sets, which would significantly impact federal and state efforts to protect the quality of our air, water, and land, and the health and welfare of the American people. The commenter notes that some states' environmental laws and regulations explicitly adopt EPA standards in all or some instances, or at the very least require an express justification for any deviation. The commenter states that, therefore, it is clear a fundamental change in how EPA develops standards would most certainly affect state standards, and therefore would affect the health of our residents and our natural resources.
Commenter (6915) states that the proposal fails to explain whether or how the proposed rule would apply to EPA's review and approval of state standards, and, accordingly, deprives commenters of a full and fair opportunity to assess and comment on the proposal's federalism implications.
Response: This action does not have federalism implications as it imposes no enforceable duty on any state, or local governments. It will not have substantial direct effects on the states, on the relationship between the national government and the states, or on the distribution of power and responsibilities among the various levels of government.
15.1.6 Executive Order 13175: Consultation and Coordination with Indian Tribal Government
Comment: Commenters (0565, 6123, 6130, 6920) disagree with the statement in the proposal that this action does not have tribal implications as specified in Executive Order 13175 and does not require consultation with tribes. Commenters (0565, 6130, 6920) assert that this conclusion is not adequately supported and urge additional analysis of the probable impacts of the proposed rule on Tribal Nations. 
Commenter (0565) points out that EPA's own Policy on Consultation and Coordination with Indian Tribes (May 4, 2011), adopted to implement its responsibilities under Executive Order 13175, begins with the following sentence: "EPA's policy is to consult on a government-to-government basis with federally recognized tribal governments when EPA actions and decisions may affect tribal interests." The commenter states that this is based on the 1984 EPA Policy for the Administration of Environmental Programs on Indian Reservations, reaffirmed by Administrator Pruitt last year, by which EPA, in keeping with its federal trust responsibility, commits in Principle 5 to "assure that tribal concerns and interests are considered whenever EPA's actions and/or decisions may affect reservation environments." 
Commenter (0565) asserts that there is no question that the nature and scope of scientific studies and underlying data that form the basis of EPA's regulations have a bearing on the level of human health and environmental protections afforded by those regulations. The commenter states that tribal communities suffer the same affects as other communities from pollution of the nation's air, land and water; but are also disparately impacted in a variety of ways, reflected by the significantly higher instances of environmentally-based diseases that tribal communities experience compared with non-Indian populations. The commenter states that this disparity results from factors such as socio-economic conditions, lack of adequate infrastructure, and our fundamental relationship with the natural environment and reliance on a variety of plants, wildlife and habitats for subsistence and cultural practices. The commenter states that, in these circumstances, it is critically important to Tribal Governments to ensure in particular that data regarding the disparate impacts to Tribal community health from various pollution sources regulated by EPA, as well as Traditional Ecological Knowledge that can inform regulatory approaches, can continue to be taken into account in crafting and maintaining appropriate human health and environmental protections. The commenter states that, for these reasons, the proposed rule certainly has the potential to impact Tribal interests and must therefore be the subject of a meaningful consultation process.
Commenter (6125) states that, because few federally recognized Indian tribes operate their own regulatory programs, federal requirements and standards apply to most Indian lands. The commenter asks the following questions
 Has EPA considered whether it would be consistent with the federal trust responsibility to tribes to relax requirements or standards, and thus reduce protection for tribal lands solely because of a new regulation based on ignoring reliable and relevant science? 
 Has EPA considered whether such a reduction in protection on tribal lands is consistent with principles of environmental justice?
 As this rule is implemented in regard to states and tribes, who may comment or have consultation roles in developing major regulations? 
 Would states or tribes have any recourse if levels of protection are reduced under this new regulation that ignores evidence or could they keep the old standards? 
Response: This action imposes no enforceable duty on any state, local or tribal governments or the private Sector. Therefore, this action does not have tribal implications as specified in Executive Order 13175. Thus, Executive Order 13175 does not apply to this action. 
15.1.7 Executive Order 13556: Controlled Unclassified Information
Comment: Commenter (4591) asserts that, in some instances, this proposed rule will directly conflict with Executive Order 13556 and the accompanying National Archives and Records Administration implementing directive, which place limitations on the release of specific non-classified information involving privacy, security, proprietary business interests, and law enforcement investigations. The commenter questions whether controlled unclassified information (CUI) categories limiting the release of information relating to health, genetic information, proprietary business information, pesticide producers and railroad safety analysis records might prevent valid scientific studies based upon these CUI categories from being released and as a result, prevent the best possible scientific evidence from being used to develop regulations aimed at protecting public health and safety.
Response: This rule does not require the EPA or other parties to disclose or host data. In addition, the EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. As discussed more fully in the preamble of the final rule, the rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making.
15.1.8 Executive Order 13771: Reducing Regulation and Controlling Regulatory Costs
Comment: Commenter (6915) does not agree with the proposal's assertion that it does not amount to an Executive Order 13771 regulatory action. The commenter states that, rather than acknowledging costs, the only place where the proposed rule even mentions costs is in section 30.8, which states that "EPA shall implement the provisions of this subpart in a manner that minimizes costs" -- a misleading and fatally vague projection of the impacts of the proposed rule.
Response: This final rule does not regulate any entity outside the EPA. Rather, the requirements modify the EPA's internal procedures regarding the transparency of pivotal science underlying final significant regulatory actions and influential scientific information. In addition, because this rule would not change the process by which the EPA already evaluates studies for scientific merit, the incremental costs of the rule would be for EPA to determine data availability for the limited subset identified as pivotal science. This is a small increment in effort relative to the Agency's extensive existing scientific review process. As described in preamble discussion of costs and benefits, the EPA finds that the incremental costs of this rule related to the additional review of data availability are anticipated to be small. 
15.1.9 Executive Order 13777: Enforcing the Regulatory Reform Agenda
Comment: Commenter (6362) supports EPA's demonstrated commitment in this proposal to build upon the principles underlying Executive Order 13777. Commenter (6927) agrees with the statement in the proposed rule that the proposal is consistent with Executive Order 13777. 
Response: The EPA appreciates this comment.
Comment: Commenter (6103) notes the proposal relies on Executive Order 13777, which provides that regulation reform efforts shall attempt to identify regulations that rely on non-publicly available data or are not transparent enough to meet the standards of reproducibility. The commenter states that this Executive Order does not justify the rule; it merely identifies the studies this proposal will target once enacted. The commenter states that the intent of this proposal is not to make data more publicly available but to exclude studies from the rule-making process to pave the way for deregulation. The commenter states that it would be a breach of confidentiality and personal security to release the private health data of research participants and entirely unethical to repeat studies that have shown harm to humans.
Commenter (1791) asserts that, as the EPA cites Executive Order 13777 in support of this proposed rule, so too does that order guarantee the preservation of air-quality measures surrounding soot and other chemicals less than 2.5 microns in diameter. The commenter states that the text from the order "or insufficiently transparent to meet the standard of reproducibility" applies because those epidemiological studies have met the standard of reproducibility; they have been repeatedly validated. 
Commenters (6125, 6133, 6137, 9227) do not agree with the statement in the proposed rule that the proposal is "consistent with Executive Order 13777. 
Commenter (6125) states that Executive Order 13777 calls on agency regulatory reform efforts to attempt to identify "those regulations that rely in whole or in part on data, information, or methods that are not publicly available or that are insufficiently transparent to meet the standard for reproducibility." The commenter states that this language calls for individual examination of rules for future unspecified action. The commenter contends that it in no way supports EPA's proposal to impose a blanket bar on all studies relying on undisclosed data. 
Commenter (6133) states that Executive Order 13777 requires Regulatory Reform Task Forces to evaluate existing regulations and "make recommendations to the agency head regarding their repeal, replacement, or modification, consistent with applicable law." The commenter states that the Executive Order requires the task force to identify regulations that, among other things, "impose costs that exceed benefits," and "create a serious inconsistency or otherwise interfere with regulatory reform initiatives and policies." The commenter states that, in contrast, the proposal does not identify any regulations that it believes should be repealed, replaced, or modified, consistent with applicable law. The commenter states that, instead, the proposal creates a new burdensome regulation. The commenter states that, notwithstanding EPA's unsupported assertion that it "believes the benefits of this proposed rule justify the costs," the proposed rule will impose costs that exceed benefits. The commenter states that, rather than being consistent with the Executive Order, the proposed rule contradicts it.
Commenter (6137) asserts that Executive Order 13777 seeks to reduce regulation, and in no way authorizes EPA to promulgate new rules. The commenter states that this Executive Order establishes a task force to "evaluate existing regulations . . . and make recommendations to the agency head regarding their repeal, replacement, or modification." The commenter states that EPA's assertion that the proposed rule is consistent with an Executive Order focused on deregulation is tantamount to an admission that the rule's purpose and effect is to limit the development of rules that are critical to the protection of public health and the environment. 
Commenter (6137) asserts that the proposed rule is contrary to the stated purpose and policy of Executive Order 13777, which is to "lower regulatory burdens on the American people by implementing and enforcing regulatory reform." The commenter states that the proposed rule will do just the opposite as it will preclude EPA from considering certain data regarding health and environmental impacts of pollutants, contaminants, and other substances in its rulemaking process. The commenter states that the overall impact of such limitations on rulemaking aimed at protecting public health and the environment will be increased burdens on the American people.
Commenter (9227) states that Executive Order 13777 is targeted at eliminating regulations including those that are "unnecessary" and "ineffective," which the proposal clearly would be.
Response: The EPA authority for this rule is not derived from Executive Order 13777. As stated in the preamble, the EPA is authorized to promulgate this procedural rule under its authority to promulgate housekeeping regulations governing its internal affairs under 5 U.S.C. App., and Pub. L. 98 - 80, 84 Stat. 2086. Additionally, this action is consistent with Executive Order 13777, and the focus on transparency in the OMB Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and Integrity of Information Disseminated by Federal Agencies.
In addition, as described in the preamble, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. Consistent with existing Agency practice, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information. Further, the final rule only applies to dose-response data; studies informing other aspects of the EPA's evaluations are unaffected by this rule.
15.1.10 Executive Order 13783: Promoting Energy Independence and Economic Growth
Comment: Commenter (6141) states that Executive Order 13783 supports the adoption of clear, even-handed, and effective rules for ensuring scientific transparency. Commenter (6362) supports EPA's demonstrated commitment in this proposal to build upon the principles underlying Executive Order 13783. Commenter (6927) agrees with the statement in the proposed rule that the proposal is consistent with Executive Order 13783. 
Conversely, commenters (4591, 6103, 6114, 6116, 6131, 6361) assert that Executive Order 13783 does not justify the proposed rule. 
Commenter (6103) states that Executive Order 13783 states that environmental regulations should be developed through "transparent processes that employ the best available peer-reviews science and economics." The commenter states that, to achieve this, the EPA should continue to rely on the peer-review process, as it ensures transparent processes for the best available science.
Commenters (6114, 6116, 6131, 6361) state that Executive Order 13783 calls for transparency in process but does not address the issue of transparency in publication of studies. The commenters state that Executive Order 13783 simply calls for the use of the best available peer reviewed information.
Response: EPA appreciates the comments that this rule is consistent with Executive Order 13783. However, EPA authority for this rule is not derived from Executive Order 13783, as stated in the preamble, the EPA is authorized to promulgate this procedural rule under its authority to promulgate housekeeping regulations governing its internal affairs under 5 U.S.C. App., and Pub. L. 98 - 80, 84 Stat. 2086. The transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency.
Comment: Commenters (4591, 6125, 6133, 6137, 9227) do not agree with the statement in the proposed rule that the proposal is consistent with Executive Order 13783.
Commenters (6133, 6137) state that, in an effort to show the proposed rule's consistency with this Executive Order, EPA quotes the following part of the Order: "It is also the policy of the United States that necessary and appropriate environmental regulations comply with the law, are of greater benefit than cost, when permissible, achieve environmental improvements for the American people, and are developed through transparent processes that employ the best available peer-reviewed science and economics." Commenter (6133) asserts that Executive Order 13783 contradicts the proposal because the proposal would prevent EPA from promulgating regulations that comply with the law, would cost more than any benefit it could achieve, and would preclude the use of the best available peer-reviewed science.
Commenter (6137) states that the proposed rule contravenes this policy in a number of ways: it fails to comply with a number of statutes and executive orders; the costs of the rule far outweigh the benefits; it will cause substantial and far-reaching harm to public health and the environment; and it has been developed through a process that lacks transparency. The commenter states that, if implemented, the proposed rule will lead to future rulemakings that likewise will be inconsistent with this Executive Order, as it will preclude EPA from considering the best available peer reviewed science and economics, which in turn will impact the cost-benefit analysis, and may violate notice-and-comment and judicial review procedures.
Commenter (9227) states that Executive Order 13783 says nothing more than that environmental regulations should be developed through transparent processes that employ the best available peer-reviewed science. The commenter states that Executive Order 13783 calls for agencies to consider the costs and benefits "that are based on the best available science and economics" to ensure sound regulatory decision making. The commenter provides that the proposal provides no analysis of the costs and benefits of implementing this new policy, despite there likely being high costs to making research data public with little evidence of significant benefits achieved from this policy alone. The commenter states that, by arbitrarily excluding scientific information that EPA may use in its regulatory analyses, the proposal conflicts with the executive order's command to employ the best available science and economics.
Commenter (4591) states that EPA is bound through Executive Order 13783 to develop regulations through "transparent processes that employ the best available peer-reviewed science." The commenter states that the proposed rule presumes that peer-reviewed, otherwise credible scientific studies do not merit consideration in the agency's decision making, unless data supporting the study are made publicly available within the agency's specifications. The commenter states that this presumption is not in keeping with the spirit of data transparency as understood by the scientific community. The commenter states that science does not depend on the public availability of underlying data to indicate quality and reliability of evidence, and public availability of research data is not a proxy for the reproducibility of science. The commenter states that the trend toward data transparency championed by foundations and science agencies around the world is predicated on the view that peer-reviewed, published studies already have presumptive scientific merit. 
Response: This is a rule of internal procedure promulgated under the EPA's housekeeping authorities, and imposes no enforceable duty on any state, local or tribal governments or the private sector. Additionally, this rule does not require the EPA or other parties to disclose or host data. Hence, this rule does not impose costs on the EPA or any other party to make such data available. As described in preamble discussion of costs and benefits, the EPA recognizes there will some incremental review and administrative costs to the Agency but anticipates these will be small. 
The EPA disagrees that this action is inconsistent with Executive Order 13783. This rulemaking is not intended to modify the Agency's interpretations of "best available science." The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. The EPA will then identify and consider pivotal science in accordance with the provisions of this rule, unless the implementation of the rule conflicts with statutory requirements and associated implementing regulations. As this rule makes clear, if implementing this procedural rule would result in conflicts with existing environmental statutes, the procedural rule will yield to the existing statutes.
The final rule requires EPA to follow best practices to evaluate potential links between exposure to a pollutant, contaminant, or substance and the effect and the nature of the dose-response relationship. EPA will continue to use the following, well-established factors when assessing study quality: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Comment: Commenter (6137) states that Executive Order 13783 provides that greenhouse gas impact estimates should be consistent with guidance in OMB Circular A-4 (Sept. 17, 2003), which specifically requires that this analysis be based on "the best reasonably obtainable scientific, technical, and economic information available." The commenter states that, though the Circular states that, where available, peer-reviewed, transparent, and reproducible studies should be used, it neither requires nor authorizes the preclusion of consideration of scientific studies based on data that cannot be made publicly available. The commenter states that, instead, it recognizes that there will be circumstances "[w]here other compelling interests (such as privacy, intellectual property, trade secrets, etc.) prevent the public release of data or key elements of the analysis," and provides in those cases that, rather than precluding the data, the use of "especially rigorous robustness checks to analytic results" should be applied and documented. The commenter states that, for this reason, the proposed rule is inconsistent with Executive Order 13783.
Response: This rule will not preclude information from consideration. Consistent with existing Agency practice, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information. The final rule requires EPA to follow best practices to evaluate potential links between exposure to a pollutant, contaminant, or substance and the effect and the nature of the dose-response relationship. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. As further detailed in the preamble to the final rule, only when characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect, will the EPA identify pivotal science and give greater consideration to pivotal science for which the underlying dose-response data are available in a manner sufficient for independent validation. Title 40 CFR 30.5 also provides additional considerations for instances when underlying dose-response data are not available for independent validation. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations. In addition, as this rule makes clear, if implementing this procedural rule would result in conflicts with existing environmental statutes, the procedural rule will yield to the existing statutes.
15.2 SNPRM Statutory and Executive Order Requirements
 15.2.1 Executive Orders
 15.2.1.1 Executive Order 12291: Federal Regulation
Comment: Commenter (11183, 12719, 12546) provides that Executive Order 12291 requires executive agencies to perform a cost-benefit analysis for all major rules. The commenters note that this proposed rule -- which will limit EPA's use of a vast compendium of existing and forthcoming research -- does not include an analysis to illicit how this rule's costs compared to its benefits. Thus, according to the commenter, the EPA has failed to comply with this requirement.
Response: Executive Order 12291 was revoked in 1993 by President Bill Clinton (D), who issued a modified regulatory review program under Executive Order 12866. 
15.2.1.2 Executive Order 12866: Regulatory Planning and Review and Executive Order 13563: Improving Regulation and Regulatory Review
Comment: Commenters (3788, 11479, 11903, 12464, 12715, 12727, 12751) state that the supplemental proposal violates the principles of Executive Order 12866, Regulatory Planning and Review, 58 Fed. Reg. 51,735 (Oct. 4, 1993). Commenters provide that EPA has failed to estimate the costs the Agency would incur to implement the proposal, despite the availability of information upon which to base such estimates.
Commenter (12727) provides that EPA has failed to estimate the costs the Agency would incur to implement the proposal, despite the availability of information upon which to base such estimates. The commenter notes that Executive Order 12866 specifically includes costs "to the government" among the costs the Agency "shall consider." The commenter maintains that it again references costs "to the government" when describing the required assessments for significant regulatory actions. The commenter states that moreover, the costs of administering a regulation are plainly relevant to the reasonableness of an Agency's interpretation of the statutes supposedly authorizing the regulation.
Commenter (11479) asserts that EPA suggests that the rule apply to "significant regulatory actions" under Executive Order 12866, which governs regulatory planning and review, is overly broad, and would allow for even more politicization of science in EPA rulemaking. Executive Order 12866 defines "significant regulatory action" as having an annual economic effect of $100 million or more; interfering with actions taken by other agencies; altering the budgetary impact of entitlements, grants or user fees; or raising "novel legal or policy issues arising out of legal mandates, the President's priorities, or the principles set forth in this Executive order." According to the commenter, this elastic clause would grant the Agency or OMB's Office of Information and Regulatory Affairs (OIRA) the discretion to categorize any rule as significant, depending on an administration's priorities.
Response: This is a rule of internal procedure promulgated under the EPA's housekeeping authority, and imposes no enforceable duty on any state, local or tribal governments or the private sector. Additionally, this rule does not require the EPA or other parties to disclose or host data. Hence, this rule does not impose costs on the EPA or any other party to make such data available. As described in the preamble discussion of benefits and costs, the EPA recognizes there will be some incremental review and administrative costs to the Agency but anticipates these will be small. The analysis by CBO noted by the commenter is not applicable to this rule because the analyses were not for this rule and the requirements analyzed are not the same as those in this rule.
15.2.1.3 Executive Order 12898: Federal Actions to Address Environmental Justice in Minority Populations and Low-Income Populations
Comment: Several commenters (10974, 11183, 11184, 11403, 11918, 12412, 12458, 12474, 12546, 12715, 12719, 13549, 14397) state that the EPA inadequately addressed environmental justice in minority populations and low-income population impacts of the rule as required under Executive Order 12898. Commenters state that the supplemental proposal does not address the potential environmental justice implications of the research restrictions it would impose and therefore fails to meet the requirements of Executive Order 12898.These comments include the following:
Commenter (11183) provides that Executive Order 12898 directs agencies, including EPA, to identify and address "disproportionately high and adverse human health or environmental effects of their actions on minority and low-income populations, to the greatest extent practicable and permitted by law." The commenter notes that that proposed rule does not identify or address the potential impacts it may have on environmental justice communities with restriction to regulatory science that excludes private data. Thus, EPA has failed to comply with this requirement.
Commenter (12715) states that while EPA states that the supplemental proposal is not subject to Executive Order 12898 because it does not establish an environmental health or safety standard, 85 Fed. Reg. at 15,404, Executive Order 12898 is not so limited in its application. Rather, the commenter notes that it broadly applies to all Agency "programs, policies, and activities," Exec. Order 12898 at 1, and, in any event, the commenter adds that the supplemental proposal would affect countless environmental health and safety standards. In addition, the commenter states that to the extent EPA looks to existing environmental statutes as its authority for the proposed rule, those statutes generally require EPA to consider protection of human health and the environment in taking any regulatory action, which includes potential environmental justice concerns. See, e.g., Resource Conservation and Recovery Act (RCRA), 42 U.S.C. § 6981(a).
Commenter (12719) expresses that the EPA states that Executive Order 12898 does not apply to the proposed rule because "it does not establish an environmental health or safety standard." The commenter states that this statement is not accurate. As this proposal addresses how EPA will consider data that informs environmental health or safety standards, the commenter contends it is therefore relevant to environmental justice communities. The commenter asserts that disproportionately impacted communities are affected by every decision that EPA makes-- including how EPA considers or analyzes its data. EPA should consider the Transparency in Science proposal as subject to Executive Order 12898.
Commenters (11184, 12412, 12474, 13549, 14397) state that they are disheartened by the supplemental proposal's lack of consideration and analysis relating to the proposal's impact on Environmental Justice communities: the supplemental proposal merely states, without further explanation, that "this action does not have tribal implications as specified in Executive Order 13175," and "EPA believes that this action is not subject to Executive Order 12898 (59 FR 7629, February 16, 1994) because it does not establish an environmental health or safety standard." 85 Fed. Reg. at 15,404.
Commenter (10974) notes that Executive Order 12898 was promulgated to guide "each Federal agency" to "make achieving environmental justice part of its mission by identifying and addressing, as appropriate, disproportionately high and adverse human health or environmental effects of its programs, policies, and activities on minority populations and low-income populations in the U.S. ..." (59 FR 7629)
As with children, the commenter (10974) states that decades of epidemiological health research has found that Indigenous people, people of color, and those with low income are disproportionately exposed to, and impacted by, pollution. The commenter notes that this puts those individuals and communities at a higher risk of side effects from drastic changes in health research or policy making. According to the commenter, the claim that Executive Order 12898 does not apply because the supplemental proposal "does not establish an environmental health or safety standard", misses the spirit of environmental justice considerations.
Commenter (12458) expresses that, while excluding important public health data from rulemaking has the potential to harm all people, it would disproportionately harm environmental justice communities. The commenter states that the EPA has an explicit legal obligation to avoid these kinds of harms under Title VI of the Civil Rights Act and the Environmental Justice Executive Order 12898. Title VI of the Civil Rights Act requires EPA to ensure that the programs and activities it funds do not use criteria, methods, or practices that discriminate on the basis of race, color, or national origin. By restricting the consideration of some public health research, the commenter states that the proposed rule would have an adverse disproportionate impact on people of color in downstream programs and activities. 
The commenter (12458) states that the Agency's own Environmental Justice and Community Revitalization Priorities memorandum pledges to "improve EPA science to better understand the needs of underserved and overburdened communities...with an emphasis on public participation to identify solutions." Yet the commenter notes that the EPA now seeks to expand a proposed rule that environmental justice groups overwhelmingly opposed in 2018. Moreover, the commenter provides that the EPA Environmental Justice 2020 Action Agenda specifically prioritizes ensuring that environmental justice is appropriately analyzed, considered and addressed in EPA rules." The commenter concludes that environmental justice cannot be appropriately analyzed, considered, or addressed in rulemaking if large swaths of public health data are excluded from the rulemaking process.
Commenter (12715), the EPA must -- but has altogether failed to -- contend with the fact that the supplemental proposal would have significant and impermissible environmental justice implications. Specifically, the commenter notes that by limiting the scientific studies and data that EPA may consider in both crafting environmental regulations and finalizing "influential scientific information," the SNPRM and the original proposal are Agency actions that could have disproportionately high and adverse human health or environmental effects on environmental justice communities. Scientific evidence plays a crucial role in addressing the various sources of environmental pollution and reducing the public health burden attributable to environmental factors. The commenter adds that the role of such evidence is particularly important to environmental justice communities, which are disproportionately affected by environmental risk. The commenters provides as an example a recent study by EPA scientists that found that facilities emitting dangerous particulate air pollution -- like soot -- disproportionately impact low-income communities and communities of color. The commenter contends that if EPA can now ignore or give less weight to epidemiological or other studies where the underlying data is not publicly available, this could lead to even more disparities in the burdens that air pollution and other toxics impose on environmental justice communities.
Commenter (11403) states that the proposed rule's arbitrary inclusion criteria will create unnecessary and harmful biases against epidemiological studies that are the only source of data designed to protect overburdened and vulnerable populations. The commenter notes that this directly violates Executive Order 12898, which directs every federal Agency to ". . . collect, maintain, and analyze information assessing and comparing environmental and human health risks borne by populations identified by race, national origin, or income. To the extent practicable and appropriate, Federal agencies shall use this information to determine whether their programs, policies, and activities have disproportionately high and adverse human or environmental effects on minority populations and low-income populations." The commenter provides that in 2011, EPA also signed a Memorandum of Understanding to implement the environmental justice goals of Executive Order 12898.
Response: This action is not subject to Executive Order 12898 because it does not establish an environmental health or safety standard. The final rule will not lead to systematic bias towards certain types of stakeholder goals of any party. The final rule requires EPA to follow best practices to evaluate potential links between exposure to a pollutant, contaminant, or substance and the effect and the nature of the dose-response relationship. EPA will also use the following, well-established factors when assessing study quality: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. The EPA will continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. As further detailed in the preamble to the final rule, only when characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect, will the EPA identify pivotal science and give greater consideration to pivotal science for which the underlying dose-response data are available in a manner sufficient for independent validation. Title 40 CFR 30.5 also provides additional considerations for instances when underlying dose-response data are not available for independent validation. 
15.2.1.4 Executive Order 13045: Protection of Children from Environmental Health Risks and Safety Risks
Comment: Several commenters (10974, 11183, 12430, 12540, 12546, 12719) cite Executive Order 13045 in part or whole as stating that EPA has reason to believe that environmental or safety risks we encounter in "the air we breathe, the food we eat, water we drink or use for recreation, the soil we live on, and products we use or are exposed to" may disproportionately impact children, and therefore EPA must evaluate the effects of planned regulation on children and explain why this regulation is preferable to others. The commenters express concern that the proposed rule does not consider the disproportionate impacts on children that may occur with the restriction to regulatory science based on publicly available data. Thus, according to the commenters, the EPA has failed to comply with this requirement.
Commenter (12719) states that since the proposed rule concerns all environmental and health data that would contribute to rulemaking, the EPA should provide an analysis on how the proposal would affect data used to make decisions on children's environmental risks. The commenter provides that as children are known to be uniquely vulnerable to environmental exposures (lead exposures are just one example), it is difficult to understand how EPA can state that Executive Order 13045 would not apply to this proposed rule. 
Commenter (12430) expresses that the proposed regulation could disproportionately affect children by preventing U.S. EPA from considering children's health studies, thereby increasing risks and negative health outcomes for American children. The commenter adds that the U.S. EPA recognizes the supplemental proposal as significant (and submitted it to OIRA, as discuss below), but failed to conduct the evaluation of health impacts to children required by Executive Order 13045.
Commenter (10974) states that section 2-202 part (b) of the Executive Order is what EPA bases their statement that the supplemental proposal does not apply, since the proposal was deemed "economically significant" under Executive Order 12866 and was submitted to OIRA for review. The commenter notes that it has also been made plain that EPA does not consider this rulemaking to be substantive. Interestingly, the commenter states that the EPA failed to make the claim that it was not a "substantive action" under Executive Order 13045, as they attempted with the review of the housekeeping authority. In either case, the commenter does not believe that (b) applies: this is a substantive rule and, thus, a substantive action.
The commenter (10974) states that, if promulgated, this rule would have broad-reaching implications on entire fields of research. The commenter adds that the power vested in the administrator to make determinations about which studies could be used  -  without any defined criteria guiding such decisions  -  is alarmingly inappropriate. If entire fields of environmental health (and all) research are impacted by this rulemaking, the commenter contends that it follows that those more at-risk from environmental pollution  -  those who have the greatest need for stringent, health-protective pollution standards  -  will fare worse than healthy populations. According to the commenter, the EPA's statement that this would not disproportionately affect children is naïve and inept.
The commenter (10974) notes that, as stated in Executive Order 13045 itself, "...scientific knowledge demonstrates that children may suffer disproportionately from environmental health risks and safety risks." Since the signing of Executive Order 13045 over 20 years ago, the commenter states that epidemiological and other environmental health research has consistently demonstrated that children can be, and are, disproportionately exposed to, and impacted by, a variety of environmental pollutants. The commenter adds that research is among the multitudinous amount of literature currently available for consideration while EPA is conducting policy making actions.
By broadening the applicability of this rule to all data and models collected  -  not just epidemiological data and models  -  the commenter (10974) contends that the EPA is further endangering the lives of healthy and at-risk children. The commenters believe that Executive Order 13045 applies to this rulemaking, and that EPA has not done enough to address anticipated health and safety impacts on children as the Executive Order recommends.
Commenter (12546) failed to consult with critical stakeholders on the impacts of the Science Transparency Rule on children's health. The commenter notes that environmental health regulatory and science actions impacted by this rule and supplemental proposal may disproportionately affect children's health. For example, NAAQS for pollutants such as PM2.5 (fine particulate), ozone and lead all have highly relevant and important studies showing that children can be more sensitive and susceptible to these pollutants. Therefore, the commenter strongly asserts that EPA is required to conduct an analysis under Executive Order 13045: Protection of Children from Environmental Health Risks and Safety Risks.
Response: The EPA interprets Executive Order 13045 as applying only to those regulatory actions that concern environmental health or safety risks that the EPA has reason to believe may disproportionately affect children, per the definition of "covered regulatory action" in section 2-202 of the Executive Order. This action is not subject to Executive Order 13045 because it does not concern an environmental health risk or safety risk.
In addition, the final rule will not lead to systematic bias towards certain types of stakeholder goals of any party. The final rule requires EPA to follow best practices to evaluate potential links between exposure to a pollutant, contaminant, or substance and the effect and the nature of the dose-response relationship. EPA will continue to use the following, well-established factors when assessing study quality: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. The EPA will continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. As further detailed in the preamble to the final rule, only when characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect, will the EPA identify pivotal science and give greater consideration to pivotal science for which the underlying dose-response data are available in a manner sufficient for independent validation. Title 40 CFR 30.5 also provides additional considerations for instances when underlying dose-response data are not available for independent validation. In addition, as this rule makes clear, if implementing this procedural rule would result in conflicts with existing environmental statutes, the procedural rule will yield to the existing statutes. 
15.2.1.5 Executive Order 13132: Federalism
Comment: Commenters (12430, 12435, 12715) contend that the supplemental proposal violates the principles set forth in Executive Order No. 13132, Federalism, 64 Fed. Reg. 43,255 (Aug. 4, 1999).
Commenter (12715) provides that Executive Order 13132 provides that agencies must have an accountable process to ensure meaningful and timely input by state and local officials in the development of regulatory policies that have federalism implications. Federalism implications are defined as including regulations and actions that have substantial direct effects on states or local governments (individually or collectively). Id. at 1. The commenter contends that, contrary to EPA's unsupported, cursory assertion, 85 Fed. Reg. at 15,404, the supplemental proposal indisputably has substantial federalism implications because, as explained herein, states and local communities are directly and significantly impacted by health and risk-based standards established by EPA.
The commenter (12715) notes that standards adopted by EPA under the supplemental proposal's directive to exclude or demote relevant and valid scientific studies will have substantial direct effects on states and cities. The commenter provides that states may be statutorily required to adopt EPA standards or to obtain EPA approval of state-set standards and may lack the resources or institutional capacity to deviate from EPA standards. The commenter adds that states, counties and cities are also significantly impacted through incorporation of EPA standards into the regulations or programs of other federal agencies that rely on EPA standards and/or modeling. The commenter provides that this includes, inter alia, fuel economy standards set by the National Highway Transportation and Safety Administration, which have an impact on vehicle emissions of harmful air pollutants. Despite the substantial impact EPA's supplemental proposal would have on states and local governments, the commenter states that the EPA did not seek any input from states and local governments in developing the supplemental proposal, in violation of Executive Order 13132. The commenter asserts that the EPA must engage in the required consultation with state and local officials, as required by the Executive Order, before proceeding further.
Similarly, commenter (12430) states that the supplemental proposal would disrupt the cooperative relationship between the federal government and California to implement federal environmental laws, including potentially undercutting state-level air quality standards under the CAA.
The commenter (12430) provides that the CAA represents a hallmark example of cooperative federalism, as U.S. EPA and state air agencies partner to protect public health from harmful effects of air pollution. The commenter notes that an essential aspect of this relationship includes basing federal and state-level implementation decisions on the best available science. According to the commenter, this includes U.S. EPA setting of NAAQS at a level requisite to protect the public health, while states meet the standards through development and implementation of State Implementation Plans (SIP). The commenter provides that the NAAQS must be based on air quality criteria developed using the "latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare," and updated at least every five years following a thorough review by an independent scientific review committee. The commenter adds that the CAA sets out specific actions for this committee to advise U.S. EPA on areas where additional knowledge is required for reviewing NAAQS and the research efforts necessary to obtain that information, among related topics.
The commenter (12430) (representing California) notes that they rely on the legitimacy of this scientific review process for purposes of commenting on the NAAQS-setting process and developing and implementing its SIPs. The commenter states that the supplemental proposal does not address how the proposal would affect this scientific review process. If a finalized supplemental proposal prevents U.S. EPA and its CAA Advisory Committee from considering the best available science, the commenter contends that it would severely restrict U.S. EPA's decision-making when establishing, reviewing, or revising air quality criteria and NAAQS. According to the commenter, the resulting effect is likely to be confusion and disagreements between the California Air Resources Board (CARB) (and its counterparts in other states) and U.S. EPA about what studies can and should be considered. The commenter states that this is likely to harm the cooperative relationship between U.S. EPA and state air agencies, in addition to hindering the ability of U.S. EPA and CARB to meet the obligations of the CAA, and, ultimately, harming public health through the setting of substandard NAAQS.
The commenter (12430) states that the same dynamic would likely play out under other environmental laws that depend upon cooperative federalism, such as the CWA. Similarly, the commenter contends that this would risk damaging the cooperative federalism framework devised by Congress for these laws. The commenter asserts that the U.S. EPA's failure to consult with the states on the impact of this rule is also inconsistent with U.S. EPA's own primary goal set forth in its 2018-2022 Strategic Plan to create more effective partnerships with the states, among others, in carrying out shared responsibilities and communicating results to all Americans. Given the potentially significant impacts, the commenter states that the U.S. EPA's failure to analyze the cooperative federalism impacts of the SNPRM under E.O. 13132 further demonstrates the arbitrariness of U.S. EPA's rulemaking process.
Commenter (12435) states that this action will have substantial direct impact on the state's ability to use the best available scientific information to protect human health and the environment. The commenter adds that this action could also impact the state's ability to ensure more stringent cleanup requirements are used at EPA-lead Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA) sites when state standards and/or cleanup levels are based on best available science, but EPA standards and/or cleanup levels may be based on a limited set of data and models. Furthermore, the commenter states that the action could impact previous and forthcoming rules involving interstate air pollution flow and impede states with NAAQS nonattainment areas such as Colorado from legally addressing emissions in upwind states.
Response: This action does not have federalism implications as it imposes no enforceable duty on any state or local governments. It will not have substantial direct effects on the states, on the relationship between the national government and the states, or on the distribution of power and responsibilities among the various levels of government. 
In addition, this final rule will not impact EPA's ability to rely upon best available science and does not establish an environmental health or safety standard. The final rule requires EPA to follow best practices to evaluate potential links between exposure to a pollutant, contaminant, or substance and the effect and the nature of the dose-response relationship. EPA will also use the following, well-established factors when assessing study quality: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. The EPA will continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. As further detailed in the preamble to the final rule, only when characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect, will the EPA identify pivotal science and give greater consideration to pivotal science for which the underlying dose-response data are available in a manner sufficient for independent validation. Title 40 CFR 30.5 also provides additional considerations for instances when underlying dose-response data are not available for independent validation. In addition, as this rule makes clear, if implementing this procedural rule would result in conflicts with existing environmental statutes, the procedural rule will yield to the existing statutes.
15.2.1.6 Executive Order 13175: Consultation and Coordination with Indian Tribal Governments/Tribal Community Implications
Several comments (11407, 11895, 12415, 12423, 12474,) were submitted on the supplemental proposal related to Executive Order 13175, tribal government consultation requirements and tribal community implications of the proposal. This comments are provided below.
Comment: Commenters (11407, 12423, 12474, 13549, 14397) cite Executive Order 13175 legal obligations. According to commenters, the rule has potentially far-reaching and substantial impact on tribes and it should have triggered a tribal consultation process under the Executive Order.
Commenters (11895, 12474, 13549) states that they are disheartened by the supplemental proposal's lack of consideration and analysis relating to the proposal's impact on Indian Tribes and Environmental Justice communities: the supplemental proposal merely states, without further explanation, that "this action does not have tribal implications as specified in Executive Order 13175," and "EPA believes that this action is not subject to Executive Order 12898 (59 FR 7629, February 16, 1994) because it does not establish an environmental health or safety standard." 85 Fed. Reg. at 15,404. The commenters note that it is well deduced that the proposed rule will have a cascading effect on all future rules and as Tribes they will see it and will feel its repercussions.
Given the over-arching nature of the proposed rule's purview and particular impact on tribes, commenters (11895, 12423, 12474, 13549, 14397) state the Agency should have included a justification for not consulting with tribal governments under the Executive Order.
Response: This action imposes no enforceable duty on any state, local or tribal governments or the private Sector. Therefore, this action does not have tribal implications as specified in Executive Order 13175. Thus, Executive Order 13175 does not apply to this action. 
Comment: Commenters (12423, 13549) express that tribal communities are likely to experience disproportionate adverse impacts as a result of the proposed rule/supplemental. Specific comments include:
Commenter (12423) states that many reservations are disproportionately impacted from higher levels of chemicals, pesticides, and contaminants, as well as from National Priority List Superfund Sites that release contaminants in the soils, water, and air, thus impacting tribal health.
The commenter (12423) asserts that tribal specific data are needed for tribes to be represented and protected by the EPA regulations that are intended to protect the environment tribes depend on for their very lifeways. According to the commenter, if this rule is enacted, tribes would need to alter their laws concerning data rights and management. For example, the commenter notes that some tribes only allow researchers access to tribal data for the duration of the study and require that, once the study concludes, the data be returned to the tribal nation. Additionally, the commenter adds that there are tribal sovereignty implications in protecting proprietary knowledge and substantial tribal member privacy concerns because of the relatively small populations sizes. 
Commenters (12474, 13549) notes that it is well established that Tribal communities are likely to experience disproportionate exposure to harmful pollutants and chemicals. The commenters state that the proposal seeks to preclude the use of scientific research critical to establishing safeguards against this disproportionate exposure. Before finalizing any version of the proposed rule and/or supplemental proposal, the commenters assert that the EPA must listen to, analyze, and address Tribal concerns and impacts via public comments and government-to-government consultations.
Commenter (12415) contends that the rule's impact on tribes would be potentially significant and far-reaching, yet the EPA did not consult with the Confederated Tribes of the Umatilla Indian Reservation (CTUIR) (or any other tribe to our knowledge) about it. Furthermore, the commenter notes that given current circumstances regarding Coronavirus/COVID-19, particularly its acute, debilitating effects on Indian Country, there has not been adequate time nor resources available to fully engage in this rulemaking process, in addition to EPA's pervasive failures and shortcomings in terms of proper and meaningful government-to-government consultation. 
Commenter (12474) states that they recently learned that the EPA is refusing Tribal consultation on this rulemaking. According to the commenter, one reason given is that this rule is being proposed under the EPA's housekeeping authority and is an internal rule. As they state in other comments on the proposal, the commenter states that this authority is not an appropriate vehicle to address a rule of this importance. The commenter asserts that this proposal will change the way scientific analyses are performed at the EPA. The commenter contends that it is wholly insufficient for EPA to force Tribes to save our comments for future rulemakings, which will be constrained by these problematic rules. The commenter notes that the proposed new rules will cite back to this proposed rule and will not have any recourse. The commenter states that another reason given by EPA for refusing consultation is the claim that this proposal does not have Tribal implications. According to the commenter, while this proposal may not impose direct regulatory requirements on Tribes, it will most certainly have implications on the ability of Tribes to protect the health of their citizens and therefore it has Tribal implications. The commenter asserts that if the EPA moves forward with this rulemaking, it must reverse its ill-conceived refusal to undertake the necessary and essential government-to-government Tribal consultations on this rulemaking. The commenter concludes that the EPA's refusal to offer Tribal consultation is a failure to uphold the 1984 Indian Policy.
Commenter (14397) states that in 2013, a two year review by the Department of the Interior's Secretarial Commission on Indian Trust Administration and Reform resulted in the recommendation that the U.S. government clarify that: all federal agencies have a trust responsibility to Indians; that this trust responsibility demands a high standard of conduct; and each Agency should place Indian interests before those of the Agency and outside parties. The commenter contends that, quite clearly, the EPA has not followed this doctrine.
Commenter (12423) urges the EPA to correct this by postponing any rulemaking until meaningful tribal consultation has been conducted.
Commenter (12415) states that the EPA has yet to explain how the proposed rule would implicate Treaty Rights, Treaty-secured resources, trust assets, how they would be safeguarded, and how they would or could be affected if the rule is adopted. The commenter notes that the fundamental question is, how will the EPA ensure that Treaty Rights are honored, and can be exercised, if you are less able to protect the resources on which those rights depend because you have adopted less-protective rules that excluded legitimate (although nonpublic) science and data?
The commenter (12415) encourages the EPA to follow its own Guidance, which to their knowledge has not been rescinded. This Guidance, "EPA Policy on Consultation and Coordination with Indian Tribes: Guidance for Discussing Tribal Treaty Rights," merits closer scrutiny, stating in part:
      EPA recognizes the importance of respecting tribal treaty rights and its obligation to do so. . . . EPA will . . . consider all relevant information obtained to help ensure that EPA's actions do not conflict with treaty rights, and to help ensure that EPA is fully informed when it seeks to implement its programs and to further protect treaty rights and resources when it has discretion to do so. [Footnote omitted]
The Guidance summarizes the significance of treaties:
      The U.S. Constitution defines treaties as part of the supreme law of the land, with the same legal force as federal statutes. Treaties are to be interpreted in accordance with the federal Indian canons of construction, a set of long-standing principles developed by courts to guide the interpretation of treaties between the U.S. government and Indian tribes. As the Supreme Court has explained, treaties should be construed liberally in favor of tribes, giving effect to the treaty terms as tribes would have understood them, with ambiguous provisions interpreted for their benefit. Only Congress may abrogate Indian treaty rights, and courts will not find that abrogation has occurred absent clear evidence of congressional intent. [Footnote omitted]
The Guidance specifically notes as an example that ". . . protecting fish may involve protection of water quality in the watershed[,]" and that:
       . . . [T]reaty rights most likely to be relevant to an EPA action are rights related to the protection or use of natural resources, or related to an environmental condition necessary to support the natural resource, that are found in treaties that are in effect. . . . "Treaties also may contain necessarily implied rights. For example, an explicit treaty right to fish in a specific area may include an implied right to sufficient water quantity or water quality to ensure that fishing is possible. Similarly, an explicit treaty right to hunt, fish, or gather may include an implied right to a certain level of environmental quality to maintain the activity or a guarantee of access to the activity site. [Emphasis added.]
A suggested question for EPA to explore is:
      How are treaty rights potentially affected by the proposed action? This question is designed to help EPA understand how a treaty right may be affected by the proposed action. EPA should explain the proposed action, provide any appropriate technical information that is available, and solicit input about any resource-based treaty rights. It is also appropriate to ask the tribe for any recommendations for EPA to consider to ensure a treaty right is protected.
The commenter (12415) states that they are not aware if EPA has addressed this question in this rulemaking process; if so, we do not feel that we have been provided a satisfactory answer to it. 
The Guidance continues:
      EPA's next steps typically will involve conducting legal and policy analyses in order to determine how to protect the rights. These analyses are often complex and depend upon the context and circumstances of the particular situation. Issues that may arise often involve precedent-setting questions or warrant coordination with other federal agencies. It is expected that the EPA lead office or region that engaged in the tribal consultation about the potentially affected treaty rights will coordinate with the Office of International and Tribal Affairs, the Office of General Counsel, and appropriate Offices of Regional Counsel to conduct these analyses. Although the details of how to conduct such legal and policy analyses are not addressed by this Guidance, the EPA process may warrant continued or additional consultation with tribes.
Again, the commenter (12415) states that it is unclear whether EPA has performed such analyses for the proposed rule, which they contend that the EPA should.
Response: This is a rule of internal procedure promulgated under the EPA's housekeeping authority, and imposes no enforceable duty on any state, local or tribal governments or the private sector. In addition, the EPA does not agree that its approach will lead to systematic bias towards certain types of stakeholder goals of any party. As described in the preamble, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. Statute-specific transparency regulations or programmatic regulations are forthcoming and EPA will conduct required analyses and consultations as appropriate in the course of those rulemakings.
Comment: Commenter (111407) provides the EPA with specific examples of ways the EPA must use best available science to fulfil its trust responsibilities to the Ute Mountain Ute Tribe to protect public health and the environment of the Ute Mountain Ute Tribe: 
      1. EPA has a trust responsibility to intervene in disputes between delegated jurisdictions on surface water pollution and water quality standards issues, as well as approval of water quality standards. Much of the science behind those standards is peer reviewed science that goes back decades in time. flow can water quality managers cook up quick new science that complies with this proposed rule to properly regulate water quality? Is EPA going to reinvent that scientific wheel when the Ute Mountain Ute Tribe objects to a neighboring jurisdiction's standards or when an upstream entity objects to a regulatory action by the Tribe? These studies require years of research and complex formulas to protect human health and complex ecosystems from pollution in a way based on sound science. 
      2. The Ute Mountain Ute Tribe has the only operating privately-owned conventional uranium mill in the U.S. next to its Reservation. The groundwater under and around the mill are being polluted. Innovative scientific research was undertaken in partnership with the U.S. Geologic Survey, Lawrence Livermore National Laboratory, U.S. EPA, and the Tribe to better understand the age and quality of the groundwater and to use the research as a baseline into the future for best available science in protecting the groundwater and identifying the sources of the pollution. Is EPA going to fund a new study? Is EPA going to debunk the peer-reviewed work of its own scientists and those from other leading scientific agencies with this rule making?" 
Response: As described in the preamble, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions and developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. Additionally, this rule does not require the EPA or other parties to disclose or host data. Hence, this rule does not impose costs on the EPA or any other party to make data available.
15.2.1.7 Executive Order 13771: Reducing Regulations and Controlling Regulatory Costs
Comment: Commenters (12715, 12291) assert that pursuant to Executive Order 13771, agencies must assess and consider the costs of regulatory actions when making regulatory decisions. The commenter states that the supplemental proposal fails to assess costs to EPA and to researchers. Instead, the commenter notes that the EPA claims the supplemental proposal is not expected to be an Executive Order 13771 regulatory action because it relates to "agency organization, management or personnel." 85 Fed. Reg. at 15,404. Yet, the commenter asserts that EPA's proposed action does not fall within this category.
Response: This final rule does not regulate any entity outside the EPA. Rather, the requirements modify the EPA's internal procedures regarding the transparency of pivotal science underlying final significant regulatory actions and influential scientific information. 
In addition, because this rule would not change the process by which the EPA already evaluates studies for scientific merit, the incremental costs of the rule would be for EPA to determine data availability for the limited subset of dose-response studies identified as pivotal science. This is a small increment in effort relative to the Agency's extensive existing scientific review process. Therefore, as described in the preamble discussion of costs and benefits, the EPA finds that the incremental costs of this rule related to the additional review of data availability are anticipated to be small.
15.2.1.8 Executive Orders Require that the EPA Quantify Costs and Benefits of Regulatory Actions and Ensure the Public's Right to Participate in the Process
Comment: Commenter (11903) contends that the Trump Administration has assured the public, repeatedly, that it is committed to reduce the regulatory costs they will bear. The commenter provides that moreover, executive orders issued by previous Presidents address EPA's duty to identify and quantify the costs and benefits of its regulatory actions, and the public's right to participate in the process by which EPA does so. According to the commenter, those orders include Executive Order No. 12866, "Regulatory Planning and Review", 58 FR 51735 (1993), and Executive Order No. 13563, "Improving Regulation and Regulatory Review", 76 FR 3821 (2011). 
The commenter (11903) provides that as Executive Order No. 13563 indicates, our regulatory system "must identify and use the best, most innovative, and least burdensome tools for achieving regulatory ends", and "[i]t must take into account benefits and costs, both quantitative and qualitative." The commenter notes that to that end, the order directs that 
      to the extent permitted by law, each agency must, among other things: (1) propose or adopt a regulation only upon a reasoned determination that its benefits justify its costs (recognizing that some benefits and costs are difficult to quantify); (2) tailor its regulations to impose the least burden on society, consistent with obtaining regulatory objectives, taking into account, among other things, and to the extent practicable, the costs of cumulative regulations; (3) select, in choosing among alternative regulatory approaches, those approaches that maximize net benefits (including potential economic, environmental, public health and safety, and other advantages; distributive impacts; and equity); ....; and 
      "In applying these principles, each agency is directed to use the best available techniques to quantify anticipated present and future benefits and costs as accurately as possible." 
Additionally, the commenter (11903) notes that the Order specifically addresses the public's rights to participate in the process by which an Agency makes its "reasoned determination" that a rule's benefits justify its costs, tailors its rule "to impose the least burden on society", quantifies the benefits and costs "as accurately as possible", and chooses the approach that will "maximize net benefits":
According to the commenter (11903), the duties imposed by the Executive Orders do not exhaust the duties that EPA owes to the public, nor the duties EPA owes to the commenters in this rulemaking. The commenter notes that in the years since Chenery I, Overton Park, Industrial Union, and State Farm, the Federal courts have made amply clear that, when an Agency like EPA intends to promulgate a rule, it must meet the law's demands for reasoned justification. The commenter states that an Agency must provide: reasonable notice of its proposal; opportunity to comment on it; consideration of the comments received; appropriate response to comment; and a satisfactory justification for its final decision, including determinations based on evidence. The commenter contends that in sum, agencies such as EPA are obligated to avoid pretext and administrative evasiveness, and must instead engage in reasoned decision making, with comparative assessment to justify its choices, and develop a record showing that it did so. The commenter also maintains that accordingly, the law requires reasons, not just action, from EPA. The commenter asserts that the Agency's current managers should be fully familiar with the law's requirements by now, in part because they received frequent instruction from the Federal courts finding that EPA has, during the last three years, violated them.
Response: EPA has provided meaningful opportunities for input during the public comment period for both the 2018 proposed and the 2020 supplemental proposed rulemaking. The preamble for both these proposals includes a discussion of the importance of scientific integrity. The final rule further elaborates and provides the rationale for this action. In addition, EPA is providing a response to comments to address all the comments received during both the initial proposal and the supplemental. 
This is a rule of internal procedure promulgated under the EPA's housekeeping authority, and imposes no enforceable duty on any state, local, or tribal governments or the private sector. Additionally, this rule does not require the EPA or other parties to disclose or host data. Hence, this rule does not impose costs on the EPA or any other party to make such data available. As described in the preamble discussion of benefits and costs, the EPA recognizes there will be some incremental review and administrative costs to the Agency but anticipates these will be small.
15.2.2 Paperwork Reduction Act
Comment: Commenters (11894, 12464) contend that the EPA has also failed to comply with the Paperwork Reduction Act (PRA), 44 USC §§ 3501 et seq. Commenter (11894) contends that the PRA imposes obligations on every federal Agency whenever the Agency proposes measures that will lead to the submission to the Agency of information from 10 or more individuals (or entities) and that the PRA requirements apply even when the submission of information by the public is voluntary. The commenter states that the proposed rule would plainly impact the practices and actions of scientists, economists, and a wide range of others with respect to the amount of information submitted to EPA. 
According to commenter (12464), the proposal would create new reporting requirements that would apply to multiple persons who would have to make the underlying data available, either to the public in general, or through a tiered access system. According to the commenter, such reporting and data management procedures are costly and, in some instances, inconsistent with other federal policies governing information use and dissemination.
Commenter (11894) states that EPA should be required to prepare and submit an Information Collection Request (ICR) that estimates the burden expected to result from the proposed rulemaking. Further, the commenter states that the ICR should be available for public comment The commenter (11894) contends that the proposed rule totally ignores how it would impact existing reporting and recordkeeping practices. According to the commenter, if the current practices for reporting and recordkeeping of scientific research important for environmental regulation were deemed adequate, this proposed rule would be unnecessary; if they are not, EPA must intend the proposal to change those practices. The commenter notes that the PRA requires agencies to address how its proposal would modify those practices and to show why the proposal makes as little change as considered absolutely necessary. The commenter asserts that the EPA's proposal does not even begin to address this requirement.
Response: This action does not contain any information collection activities and therefore does not impose an information collection burden under the PRA. Further, as stated in the preamble, this final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science. The EPA is only required to determine if such a mechanism was available, should a subject matter expert decide to use it.
15.2.3 Additional Related Comments
Comment: Commenter (12720) states that there is no legal authority under 5 U.S.C. 301; 5 U.S.C. App.; Pub. L. 98 - 80, 84 Stat. 2086 to adopt regulation under the proposed definition of "[i]nfluential scientific information", "data.", or capable of being substantially reproduced".
Response: As explained in the final rule preamble, EPA is not relying on 5 U.S.C. 301 in promulgating this internal procedural rule. Rather, EPA is promulgating this rule in accordance with its authority to promulgate regulations governing its internal affairs, as conferred upon EPA in the Reorganization Plan No. 3, 84 Stat. 2086 (July 9, 1970), which established the EPA.
Comment: Commenter (11482) notes that EPA's lack of analysis may constitute a violation of the APA and Executive Order requirements for promulgation of Agency rules.
Response: This is a rule of internal procedure promulgated under the EPA's housekeeping authority, and imposes no enforceable duty on any state, local, or tribal governments or the private sector. Hence, this rule does not impose costs on any party other than EPA. As described in the preamble discussion of benefits and costs, the EPA recognizes there will be some incremental review and administrative costs to the Agency but anticipates these will be small.
 Chapter 16: Federal Transparency and Data Integrity Laws
The 2018 proposal states that: 
      The best available science must serve as the foundation of EPA's regulatory actions. Enhancing the transparency and validity of the scientific information relied upon by EPA strengthens the integrity of EPA's regulatory actions and its obligation to ensure the Agency is not arbitrary in its conclusions. By better informing the public, the Agency in enhancing the public's ability to understand and meaningfully participate in the regulatory process. In applying the best available science to its regulatory decision-making, EPA must comply with federal transparency and data integrity laws, and must also ensure that its decision-making is marked by independence, objectivity, transparency, clarity, and reproducibility. (83 FR 18769, April 30, 2018)
The EPA solicited comment on the proposal and how it can best be promulgated and implemented in light of existing law and prior Federal policies that already require increasing public access to data and influential scientific information used to inform federal regulation. (83 FR 18768, April 30, 2018)
This section provides responses to comments on existing transparency and data integrity laws. These comments are organized according to the following topics:
       Federal Laws and Law-Related Policies Already Address Data Concerns
       Information Quality Act (IQA), EPA's Information Quality Guidelines
       OMB Circular A-110, Freedom of Information Act (FOIA)
       OMB Memorandum 3-13
       Safe Drinking Water Act Regulations Have a Strong Scientific Basis and the Act Outlines How the Information is to be Shared
The specific comments provided in this Chapter reflect the comment letters that provide arguments supported by detailed rationale, examples, and references/documentation. The Docket ID numbers for these commenters are cited throughout this Chapter. In addition, some general comment letters may also include comments related to comments included and responded to in this Chapter. These comments are summarized in Appendix B. Note that points and issues raised in these general comment letters may also be raised by the more detailed comments summarized in this Chapter. The responses provided in this Chapter adequately address these more general comments.
16.1 Federal Laws and Law-Related Policies Already Address Data Access Concerns
Comment: Several commenters (2908, 6111, 6145, 6188, 6194, 6915, 6916, 9227) contend that federal laws and law-related policies already govern the quality of and access to raw data, including the Information Quality Act (IQA), Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity, or Information Disseminated by the Environmental Protection Agency (Oct. 2002) (EPA's Information Quality Guidelines), Office of Management and Budget (OMB) Uniform Administrative Requirements, Cost Principles, and Audit Requirements for Federal Awards ("OMB Uniform Guidance"), Data Access Act, OMB Circular A-110, Freedom of Information Act (FOIA), and OMB Memorandum 13-13. 
Response: The EPA also agrees with commenters that government agencies are making great strides in data transparency pursuant to the laws and guidance cited by the commenters. However, EPA promulgated this rule because improvements can still be made over existing policies and mechanisms. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions.
Commenters (2249, 2908, 2906, 4931, 5120, 6145. 6147, 6149, 6168, 6194, 6373, 6899, 6925, 9227) suggest that the proposal is unnecessary, inconsistent and/or redundant considering existing laws, regulations, guidance or ongoing work by the scientific community that advance transparency-related goals. It is unclear to what extent this proposed rule would be redundant to or conflict with these existing initiatives. Commenters also assert that the proposed rule would unnecessarily add another layer of federal oversight to the vast collection of existing federal regulations.
Response: The EPA agrees with commenters that government agencies are making great strides in data transparency pursuant to the laws and guidance cited by the commenters. However, EPA promulgated this rule because improvements can still be made over existing policies and mechanisms. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. The provisions of the final rule will not prevent consideration of the best available science and build upon, and do not supersede, existing executive orders and other policies that focus on transparency. Further, as stated in the preamble, this rulemaking is not intended to develop the Agency's interpretation of best available science. The EPA will continue to consider all peer-reviewed science evaluated in the EPA's influential scientific information and significant regulatory actions, consistent with existing study quality assessment factors and the statutory mandates specific to that rulemaking. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations.
Commenter (6133, 6194, 6880) state that the provisions in the proposed rule that would exclude consideration of relevant scientific findings and conclusions if the underlying data is not publicly available are inconsistent with the 2009 Presidential Memorandum for the Heads of Executive Departments and Agencies on Scientific Integrity cited in the proposal.
Commenter (6133) states that the sentences quoted from the 2009 Presidential Memorandum in footnote 2 of the proposal are inaccurately used to support the proposed rule.
Response: In consideration of public comments, the EPA made several modifications of the original proposed text to more fully consider data privacy concerns. The EPA agrees that it is important not to categorically exclude any study because the data underlying a study at the stage required by this rulemaking may not be available for independent validation. Therefore, rather than finalizing an approach that would provide for a categorical exclusion of these studies, the EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. The EPA is also including consideration of privacy concerns in the Administrator's exemption in 40 CFR 30.7. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations. The requirements in this final rule provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings. The EPA disagrees with the commenter's suggestion that implementation of the final rule would lead to suppression or alteration of scientific findings by political officials.
16.2 Information Quality Act (IQA) and Implementation Guidelines
16.2.1 Information Quality Act
Comment: Commenters (6126, 6194, 6916, 9227, 12727, 12430, 13788) state that the rule provisions are inconsistent with the IQA, FOIA, and OMB and EPA guidelines implementing the. The commenters contend that adding additional regulatory requirements and constraints is both impractical and unnecessary to achieve those goals.
Response: EPA disagrees that the final rule is inconsistent with the IQA. Implementation of the final rule will help to maximize both the quality, objectivity and utility of the pivotal science underlying the Agency's significant regulatory actions and influential scientific information. EPA's implementation of the final rule will be consistent with the Agency's obligations under the statutes cited by the commenter because the final rule does not require that the Agency disseminate information that is subject to limitations on disclosure such as confidentiality and privacy. The final rule also does not require the categorical exclusion of studies which was included in the 2018 proposal. The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions.
16.2.2 Proposed Rule Consistency with OMB Information Quality Guidelines
Comment: Commenters (0555, 0844, 6141) support EPA's position that the proposal is consistent with or not radically different from the OMB Guidelines. Commenters (6102, 6150) assert that previous Administration commitments to improve transparency have included several policies that support increased transparency, including the OMB Guidelines, and that the proposed rule will help ensure that the policy goals reflected in the OMB Guidelines and memoranda are implemented as part of EPA's rulemakings. 
Commenter (6364) emphasizes that EPA's regulatory science should be guided by and consistent with the OMB Guidelines, and they urge the EPA to reference this point in §30.10 of the rule to ensure that peer review satisfies the rigor of "independently validating key findings and ensuring that the quality of published information meets the standards of the scientific and technical community."
Response: EPA agrees that this rule is designed to build upon various guidelines on transparency and appreciates these comments. The role of independent peer review is addressed in 40 CFR 30.6 and it specifically directs the EPA to conduct peer review consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review. The rule also gives the EPA discretion to conduct additional peer review of individual studies identified as pivotal science.
Comment: Commenters (1011, 4595, 6125, 6133, 6137, 6168, 6188, 6194, 6880, 6916, 9227, 12727) provide that EPA's assertion that the cited OMB Guidelines are consistent with the proposal is incorrect nor does the Agency explain why it believes the rule is consistent with these guidelines. 
Response: EPA does not agree that this rule is inconsistent with the OMB Guidelines, rather it will complement and augment them and other transparency efforts. As discussed in the preamble to the final rule, the rule requirements are not changing the pre-existing requirements of the OMB Final Information Quality Bulletin for Peer Review. The preamble to the Bulletin states that "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review does not typically include reanalysis of the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information) and, thus, peer review is not considered a replacement for the data availability requirements of this rule. Therefore, the EPA is finalizing the role of independent peer review in 40 CFR 30.6. The provision directs the EPA to conduct peer review consistent with the requirements of the OMB Final Information Quality Bulletin for Peer Review. In order to account for the instances when the EPA determines that prior peer review was adequate, as defined by OMB, 40 CFR 30.6 gives the EPA discretion to conduct additional peer review of individual studies identified as pivotal science.
Additionally, in consideration of public comments, the EPA made several modifications of the original proposed text to more fully consider data privacy concerns. The EPA agrees that it is important not to categorically exclude any study because the data underlying a study at the stage required by this rulemaking may not be available for independent validation. Therefore, rather than finalizing an approach that would provide for a categorical exclusion of these studies, the EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. The EPA is also including consideration of privacy concerns in the Administrator's exemption in 40 CFR 30.7. Because the final rule does not categorically exclude any studies, including human health studies, from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations. 
Also, in consideration of public comments, the EPA provided a discussion of the costs and benefits of this rule in the preamble to the final rule. Briefly, this is a rule of internal procedure promulgated under the EPA's housekeeping authority and imposes no enforceable duty on any state, local, or tribal governments or the private sector. Additionally, this rule does not require the EPA or other parties to disclose or host data. Hence, this rule does not impose costs on the EPA or any other party to make such data available. The EPA recognizes there will be some incremental review and administrative costs to the Agency but anticipates these will be small.
Codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. The rule requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical and privacy considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings. 
Finally, the EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. This final rule does not interpret or apply the provisions of any environmental statutes, nor does it modify the Agency's interpretations of "best available science." It therefore cannot conflict with the Data Quality Act. The EPA will continue to consider all peer-reviewed science, consistent with existing study quality assessment factors and corresponding statutory mandates. The EPA will then identify and consider "pivotal science in accordance with the provisions of this rule," unless the implementation of the rule conflicts with statutory requirements and associated implementing regulations. As this rule makes clear, if implementing this procedural rule would result in conflicts with existing environmental statutes, and their implementing regulations, this rule will yield to the EPA statutes and regulations.
16.2.3 EPA Information Quality Guidelines-Specific Comments
Comment: Commenter (5374) states that the proposed requirements are consistent with the EPA Guidelines and support for the proposed rule. The commenter provides that the EPA issued these guidelines to formalize and maximize the quality of disseminated information, particularly with respect to the objectivity, utility, and integrity of scientific data. The commenter asserts that no matter who conducts the work, the EPA should have access to all raw data so that it has the opportunity to review and independently verify data, results and conclusions before relying upon them in a pesticide risk assessment.
Response: EPA agrees that this rule is consistent with and augments EPA Guidelines on transparency. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. As this rule makes clear, if implementing this procedural rule would result in conflicts with existing environmental statutes, and their implementing regulations, this rule will yield to the EPA statutes and regulations.
As discussed in the preamble to the final rule, the EPA maintains that the appropriate stage of data is the stage consisting of the raw data modified to remove obvious errors. The EPA disagrees with the commenter that the raw data are the appropriate stage of data because third parties who are not familiar with the data collection process may not be able to appropriately identify errors in a raw dataset, which may inadvertently produce a skewed analysis.
Comment: Commenter (6903) asserts that the proposed rule is unnecessary because the EPA has already addressed the question of public data in its policies and guidelines. The commenter provides that the EPA Guidelines establish the Agency's culture and practice to ensure the quality of data used and disseminated by EPA and delineate clearly the Agency's commitment to public access to data. The commenter further states that the EPA Guidelines also establish a clear and effective policy that considers making data publicly available. For example, the commenter states that the EPA Guidelines provide that "[i]n cases where the Agency relies on proprietary models, these model applications are still subject to our Peer Review Policy."
Response: EPA does not agree that this rule is unnecessary. As detailed in the preamble to the final rule, the rule requirements for the EPA's independent evaluation of the availability of data are necessary and critical to prioritizing data transparency in the pivotal science underlying its significant regulatory actions and influential scientific information. The transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency. 
Comment: Commenters (6111, 6133) provide that the EPA Guidelines took the position that the standard set forth in the Safe Drinking Water Act (SDWA) -  "the best available, peer-reviewed science"  -  should apply to all the Agency's risk assessments.
Commenters (6160, 6137) assert that, according to the EPA Guidelines (2005), EPA uses a "weight-of-evidence" approach that "considers all relevant information and its quality, consistent with the level of effort and complexity of detail appropriate to a particular risk assessment." EPA Guidelines 6.4. According to commenter (6137), the proposed rule contravenes these requirements and will prevent EPA from considering relevant information by precluding consideration of certain data. The commenter further states that, pursuant to the EPA Guidelines, EPA must ensure that the information it disseminates "is accurate, reliable and unbiased." EPA Guidelines 6.4(A). To do so, it uses: 
      the best available science and supporting studies conducted in accordance with sound and objective scientific practices, including, when available, peer reviewed science and supporting studies; and (ii) data collected by accepted methods or best available methods (if the reliability of the method and the nature of the decision justifies the use of the data).
Id.
Commenters (6137, 9227) assert that, given that the Proposed Rule eliminates from consideration any scientific study where the underlying data cannot be made publicly available, it undoubtedly precludes the use of the best available science in certain situations.  
Commenter (6194) states that EPA's guidance  directs that "influential scientific, financial, or statistical information" should be held to a higher degree of quality and transparency standards and that the EPA Guidelines clarify that "if access to data and methods cannot occur due to compelling interests such as privacy, trade secrets, intellectual property, and other confidentiality protections, EPA should, to the extent practicable, apply especially rigorous robustness checks to analytic results and carefully document all checks that were undertaken."
Commenter (6160) adds that the EPA Guidelines already address the transparency of data and methodology, including "(1) the source of the data used, (2) various assumptions employed, (3) the analytic methods applied, and (4) the statistical procedures employed." The commenter asserts that, because the EPA already has these methods in place, the proposed rule does nothing to improve upon them while creating unnecessary obstacles for EPA's consideration of the best available evidence for policy and rulemaking.
Response: This proposed rule will not prevent consideration of the best available science and will complement existing executive orders and other policies that focus on transparency. As stated in the preamble, this rulemaking is not intended to develop the Agency's interpretation of best available science. The EPA will continue to consider all peer-reviewed science evaluated in the EPA's influential scientific information and significant regulatory actions, consistent with existing study quality assessment factors and the statutory mandates specific to that rulemaking.
Additionally, EPA agrees that it is important not to categorically exclude any study because the data underlying a study at the stage required by this rulemaking may not be available for independent validation. Therefore, EPA is not finalizing the proposed option in the 2020 SNPRM that would categorically require that for studies to be considered pivotal science the underlying data would need to be available for independent validation. Because the final rule does not categorically exclude any studies from EPA's consideration, the Agency will not be prevented from evaluating and considering all quality, relevant studies when promulgating significant regulatory actions and developing influential scientific information. Further, the final rule only applies to dose-response data; therefore, all quality, relevant studies will continue to inform other aspects of the EPA's evaluations. 
Comment: Commenter (6915) states that the rule provisions are inconsistent with the EPA's Information Quality Guidelines.  The commenter asserts that the guidelines direct the EPA to "ensure reproducibility according to commonly accepted scientific, financial, or statistical standards. . . . [I]f access to data and methods cannot occur due to compelling interests such as privacy, trade secrets, intellectual property, and other confidentiality protections, EPA should, to the extent practicable, apply especially rigorous robustness checks to analytic results and carefully document all checks that were undertaken." Id. at 21.
Commenter (6188) states that, according to EPA's own IQGs, EPA "maintain[s] a robust quality system that addresses [information collected through contracts with EPA; information collected through grants and cooperative agreements with EPA; and information submitted to EPA as part of a requirement under a statute, regulation, permit, order or other mandate] by including regulatory requirements for quality assurance for EPA contracts, grants, and assistance agreements. This includes an Agency-wide Quality System that requires, among other things, a quality assurance manager and Quality Assurance Project Plan (QAPP) for projects and tasks that involve the use of environmental data, and a Peer Review System that requires major scientifically- and technically-based Agency work product to be peer reviewed.
Commenter (6145) asserts that the EPA's Information Quality Guidelines demonstrate that EPA is already required to have transparency in its reliance on scientific studies used in the establishment of regulations. The commenter notes that these safeguards are sufficient to ensure that the science relied upon by EPA is the best available science and that it is subject to peer review and available to the public. According to commenter for example, with regard to ensuring and maximizing the quality of "influential" information (which includes information in support of agency actions including rules). 
Commenter (9227) asserts that, pursuant to EPA's guidelines, EPA must ensure that the information it disseminates "is accurate, reliable and unbiased." EPA Guidelines 6.4(A). The commenter contends that, given that the Proposed Rule eliminates from consideration any scientific study where the underlying data cannot be made publicly available, it undoubtedly precludes the use of the best available science in certain situations.
Response: The EPA disagrees that the provisions of this rule are inconsistent with the EPA's IQGs or that the guidelines accomplish the same goals as the rule. As detailed in the preamble to the final rule, the rule requirements for the EPA's independent evaluation of the availability of data are necessary and critical to prioritizing data transparency in the pivotal science underlying its significant regulatory actions and influential scientific information. The transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency.
Comment: Commenter (6147) asserts that EPA already has many policies in place that assure the best available science is used in all elements of its analysis, including the following:
 Risk Characterization Policy (USEPA, 1995). 
 USEPA's Information Quality Guidelines (IQG) (USEPA, 2002).
 USEPA Quality Policy (2008).
Response: The transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency. The EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. For example, the EPA's 2016 Plan to Increase Access to Results of EPA-Funded Scientific Research noted that "transparency is a core EPA value" and that increased availability of research data would accelerate scientific breakthroughs that support the Agency's mission and policymaking efforts. The EPA's Open Government Plan 5.0 also details the EPA's progress in implementing the tenets of the numerous data transparency initiatives in the federal government prior to 2018, including OMB M-10-06 , the Office of Science and Technology Policy Memorandum of February 22, 2013 , and OMB M-13-13 . 
The scientific community has also embraced greater data transparency, as evidenced by data sharing and availability requirements for many high-impact journals and the emergence of organizations, such as the Center for Open Science, and international initiatives like Findable, Accessible, Interoperable, and Reusable (FAIR) data principles; Facilitate Open Science Training in European Research (FOSTER); and Guidelines for Transparency and Openness Promotion (TOP) in Journal Policies and Practices that incentivize greater transparency in research.
The EPA supports these efforts and is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. As more fully discussed in the preamble of the final rule, the EPA also agrees with commenters that the scientific community and government agencies are making great strides in data transparency; however, improvements can still be made over existing policies and mechanisms. Many scientific publications, for example, require authors to make a data availability or data access statement, which discloses where and under what conditions the underlying study data are available. Yet the EPA cannot solely rely on data availability statements made in published research because initiatives toward greater data sharing and transparency amongst scientific journals and international organizations are still being implemented, are inconsistently enforced, and the true accessibility of data in a public repository is still limited. Further, as detailed in OMB's Final Information Quality Bulletin for Peer Review, "the intensity of peer review is highly variable across journals" and "prior peer review and publication is not by itself sufficient grounds for determining that no further review is necessary." Peer review by either journals or independent advisory committees, such as the EPA's Science Advisory Board, is typically conducted without access to the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information). Should peer reviewers have access to the underlying data, they could conduct a more in-depth review that may include independently reanalyzing the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of uncertainty in the original study. Thus, peer review is not considered a replacement for the data availability requirements of this rule and implementation of this rule could provide independent peer reviewers as well as subject matter experts within the EPA and the public with the opportunity to evaluate pivotal science in ways that otherwise would not have typically occurred with existing EPA or peer review processes. 
16.3 OMB Circular A-110, Freedom of Information Act
Comment: Commenters (0569, 6194, 6915, 9227) provide that, in 1999, the Shelby Amendment or Data Access Act (attached as a rider to the Omnibus Appropriations Act of fiscal year 1999, P.L. 105-277), mandated that OMB Circular A-110 be amended to ensure that all published data from federally sponsored research used for policy and rulemaking be made available through procedures established under the FOIA. Commenter (0569) asserts that this process accomplishes EPA's goal of allowing the public to verify the soundness of science underlying policy decisions. According to commenter (6915), the law promotes public access while protecting privacy by excluding medical and business-related confidential data from disclosure. See 2 C.F.R. § 200.315 (which superseded OMB Circular A-110, which has now been incorporated into the Uniform Administrative Requirements, Cost Principles, and Audit Requirements for Federal Awards). Specifically, commenter (6194) states that OMB Circular A-110 guidance defines "research data" in part as "recorded factual material commonly accepted in the scientific community as necessary to validate research findings," but exempts from disclosure any "medical information and similar information the disclosure of which would constitute a clearly unwarranted invasion of personal privacy, such as information that could be used to identify a particular person in a research study."
Response: EPA does not agree that FOIA mechanisms already achieve the goals of this rule. Availability via FOIA mechanisms does not mean that these data are immediately or easily available to the public. Some commenters cited the EPA's Freedom of Information Act Annual Report Fiscal Year 2019 (2020), which lists a median response time for "expedited processing" of FOIA requests by the EPA as 493 days (Ref. 42). Also, FOIA applies to federal agencies and cannot be used to compel other entities to release information. For these reasons, the EPA is modifying the definition in the final rule to add the following at the end of the definition: "the public must be able to access the information on the date of publication of the proposed rule for the significant regulatory action or dissemination of the draft influential scientific information for public review and comment" to increase the opportunity for independent validation to occur closer in time to the development of significant regulatory actions and influential scientific information.
Comment: Commenter (6955) states that there is no discussion of FOIA provisions of OMB Circular A-110 for obtaining research data. Hence, according to the commenter, there is no discussion of the differences between the two approaches and no claim that the new proposal offers any added value. The commenter asserts that the omission of discussion of OMB's FOIA provisions, as covered (briefly) in OMB Circular A-110 (OMB 1999, Fischer 2013), is an indication that the proposed rule is incomplete and another indication that it would be best if it were withdrawn. The commenter asserts that it appears that the proposed public-access part of the proposal is only needed, if the restriction of all studies to compliant ones survives to the final rule. In any case, according to the commenter, by omitting mention of the OMB and IQA FOIA options, the proposed rule gives the impression it is a public-access rule, rather than a study exclusionary rule.
Response: EPA disagrees that the final rule is incomplete because there is no discussion of FOIA. The final rule also does not require the categorical exclusion of studies which was included in the 2018 proposal. The Agency will consider all studies relevant to the development of a significant regulatory action or influential scientific information but will give greater consideration to the studies identified as pivotal science when the underlying dose-response data are available for independent validation. Also see the preceding comment response.
16.4 Open Data Policy
Comment: Commenter (6915) states that OMB Memorandum 13-13,  mandates, among other things, broader public access to federal and federally funded data and information and provides that information collection should be done in a way to support information dissemination. The commenter states that this includes building redaction, slicing, and exporting into how data are collected to reduce the cost of public access later on. The commenter states that the memorandum also requires agencies to create data catalogs to include datasets "that can be made publicly available but have not yet been released." Id.
Response: As EPA states in the preamble, this rule builds upon prior EPA actions in response to government-wide data access and sharing policies, including OMB Memorandum 13-13. However, improvements can still be made over existing policies and mechanisms. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions.

 Chapter 17: Consistency with Recommendations of Third-Party Organizations Who Advocated for Open Science and Policies Adopted by Major Scientific Journals
The proposal states that it "takes into consideration the policies or recommendations of third-party organizations who advocated for open science" and "the policies recently adopted by some major scientific journals, spurred in some part by the `replication crisis." 83 Fed. Reg. at 18,770. The proposal specifically cites to web sites for proceedings of the National Academy of Sciences, PLOS ONE (Public Library of Science), Science, Nature and The Economist. The proposal also states that "[t]hese include policies and recommendations from: The Administrative Conference of the United States' Science in the Administrative Process Project; National Academies' Reports on Improving Access to and Confidentiality of Research Data, Expanding Access to Research Data, and Access to Research Data in the 21st Century; the Health Effects Institute; Center for Open Science; members of the Risk Assessment Specialty Section of the Society of Toxicology, the Dose Response Section of the Society for Risk Analysis, and the International Society for Regulatory Toxicology and Pharmacology; and the Bipartisan Policy Center's Science for Policy Project." 83 Fed. Reg. at 18,770 n.10.
Comments were received on the notice of proposed rulemaking's claim that the EPA took into consideration the policies and recommendations of cited third-party organizations and the scientific community. Some commenters provide specific comments on third-party organizations and scientific journals cited as supporting the proposal where as others supported or opposed the proposal's claims more generally. Commenters also highlighted third-party and scientific community work and expertise and recommend that these experts be consulted in the development of the Policy on Regulatory Transparency. In this section these comments are organized according to the following topics:
       Clarification of "takes into consideration the policies or recommendations"
       Misrepresentation of Third-Party and Major Scientific Journal Recommendations
The specific comments provided in this Chapter reflect the comment letters that provide arguments supported by detailed rationale, examples, and references/documentation. The Docket ID numbers for these commenters are cited throughout this Chapter. In addition, some general comment letters may also include comments related to comments included and responded to in this Chapter. These comments are summarized in Appendix B. Note that points and issues raised in these general comment letters may also be raised by the more detailed comments summarized in this Chapter. The responses provided in this Chapter adequately address these more general comments.
17.1 Clarification of "takes into consideration the policies or recommendations"
Comment: Commenters (6133, 6137, 6194, 6448, 6915, 6916, 9227) request clarification of the basis of how the cited third-party organizations and major journals are considered in the proposal.
Commenters (6133, 6194, 6448, 6915, 6916, 9227) note that the proposed rule states that it "takes into consideration the policies or recommendations of third-party organizations who advocated for open science," but it is not clear which policies or recommendations were included as a basis for the rule beyond listing names of institutions. 
Specifically, commenter (6133) states that the proposal does not explain how it takes into consideration the sources cited in footnotes 10-12. The commenter explains that, to the extent EPA is relying on these policies or recommendations, it has not provided enough information to evaluate that reliance and it must withdraw the proposed rule. The commenter further states that, consistent with the proposed rule's other citations, EPA points to nothing in the policies or recommendations from these third-party organizations that supports the proposed rule's preclusion of peer-reviewed science from consideration in regulatory decision making. 
Commenter (6137) asserts that, not only is the proposed rule a drastic departure from EPA's prior policies, positions, and procedures, but it likewise is predicated on irrational and unsupported conclusions. The commenter states that a close examination of generally applicable data access policies and guidelines  -  including policies and recommendations of third-party organizations and major scientific journals upon which EPA allegedly relies  -  reveals that EPA's proposed rule is not based on a reasoned explanation and has no rational connection to facts in the record, but rather is entirely baseless.
Commenter (6194) argues that even when a specific report is referenced, such as the National Academies' Reports on Improving Access to and Confidentiality of Research Data, Expanding Access to Research Data, and Access to Research Data in the 21st Century, the proposed rule fails to cite the specific sections EPA purportedly considered or relied upon or the specific policies embodied in these reports that are allegedly advanced by the proposed rule. Id. at 18,770 n.10. The commenter states that other references are similarly vague.
Commenters (6915, 9227) state that the proposal does not identify any groups or reports that advocate precluding consideration of non-public data in regulatory decision-making.
Commenter (9227) expresses that the EPA does not explain how EPA's current use of science is inconsistent with any recommendations from third-party organizations or inadequate considering them. The commenter concludes that the EPA's failure to provide any specific information or citations in support of its conclusory statements make it impossible to meaningfully comment on the support for EPA's Proposal.
Response: EPA notes that the articles and third-party organizations cited in the April 30, 2018, notice of proposed rulemaking are not cited in the preamble of the final rule.
However, EPA referenced those sources in the April 30, 2018 notice of proposed rulemaking because EPA agrees with the general proposition advanced therein that increasing public access to the data underlying scientific studies is beneficial. The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions.
17.2 Misrepresentation of Third-Party and Major Scientific Journal Recommendations
 17.2.1 General Comment Positions on Misrepresentation
Comment: Commenters (0670, 1589, 2787, 4591, 4831, 5172, 6046, 6125, 6126, 6133, 6137, 6155, 6166, 6168, 6880, 6912, 6915, 6916, 6924, 8281-PH20, 9227) assert that EPA misrepresents one or more of the third-party organizations and scientific journals/scientific sources cited by the EPA as supporting the proposed rule. One commenter (6155) notes that EPA does not actually state that the references support the rule, but only that they are "consistent" with it. Commenter (6125) notes that the proposal has included only an undifferentiated data dump of citations without identifying passages of relevance or explaining references to policies of other federal agencies. Commenters (6924, 6133, 6137, 6125, 6155, 6880) assert that the policies and recommendations of many of the cited third-party organizations do not support the EPA's proposal to preclude the consideration of peer-reviewed studies in regulatory decision making. They contend that in addition to misrepresenting these sources, the proposal rejects the policies or recommendations of many of the cited third-party organizations. Commenter (6133) asserts that all the cited scientific journal policies are for prospective publication, do not suggest disregarding consideration of studies without public data, and have exceptions to protect confidential or private information.
Commenters (2787, 6912, 6192, 6155) state that the proposal invokes policies from Science, and the Proceedings of the National Academies of Science as being consistent with the proposed rule, but each of these organizations has argued against, criticized, or distanced themselves from the rule. Commenter (6912) adds that, in addition to the criticism EPA received from the individuals, organizations, and publications it cited as supporting the proposed rule, virtually the entire scientific community opposed the proposed rule. The American Association for the Advancement of Science, sixty-nine public health and scientific groups, and almost 1000 individual scientists publicly opposed the rule. Many of these commenters note that the editors of all the scientific journals cited by EPA stated apropos of the proposed rule that "[e]xcluding relevant studies simply because they do not meet rigid transparency standards will adversely affect decision-making processes." Commenter (6912) further states that a co-author of the scientific articles cited by EPA stated that "[i]f the proposed rule is approved, science will be practically eliminated from all decision-making processes. Regulation would then depend uniquely on opinion and whim."
Commenter (6125) states that, upon examination, the other Executive Branch and EPA documents cited by EPA's proposal provide little support for its conclusions. According to the commenter, many of them expressly recognize that disclosure may not be possible, and ​no agency of the federal government that funds scientific research involving confidential health information provides for the automatic unfettered disclosure of data even when funding new studies. Instead, the commenter states that they make this a balancing judgment with the balancing factors being the added cost of disclosure, the feasibility of providing for disclosure, and the marginal gain in accuracy from disclosure. For example, the commenter states that the National Institutes of Health (NIH) policy allows investigators to promise subjects that their data will not be shared with other researchers, but only if the application provides an adequate justification.
Response: EPA agrees with the third-party organizations referenced in the April 30, 2018, notice of proposed rulemaking because EPA agrees with the general proposition that increasing public access to the data underlying scientific studies is beneficial. EPA notes that the articles and third-party organizations cited in the April 30, 2018, notice of proposed rulemaking are not cited in the preamble of the final rule.
The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions.
The EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
The articles and third-party organizations cited in the April 30, 2018, notice of proposed rulemaking are not cited in the final rule. The preamble of the final rule provides discussion (with citations) of how the scientific community has embraced greater data transparency.
17.2.2 Administrative Conference of the United States' Study
Comment: Commenters (6125, 6133, 6137, 6912, 6916) specifically refer to the study for the Administrative Conference of the United States (ACUS) as not supporting the EPA's proposal.
Commenters (6125, 6133, 6192) report that the author of the ACUS' Science in the Administrative Process Project and the Bipartisan Policy Center's Science for Policy Project, Wendy Wagner, a law professor at the University of Texas, addressed EPA's use of her study as follows:
      I really don't know what the problem is that they think they're fixing," she said, adding that many of her co-authors "would laugh and hoot" at some of the scientific ideas expressed in the rule.
      They don't adopt any of our recommendations, and they go in a direction that's completely opposite, completely different," she told me after reading the rule. "They don't adopt any of the recommendations of any of the sources they cite. I'm not sure why they cited them.
For example, the commenters state that the ACUS Science in the Administrative Process Project, which EPA listed, recommends that agencies take a flexible approach to data disclosure: "To the extent practicable and in compliance with applicable legal restrictions, privileges, protections, and authorities, agencies should seek to provide disclosure of data underlying scientific research[.]" 78 Fed. Reg. 41,352, 41,358 (July 10, 2013). This recommendation thus states only that agencies should "seek to provide disclosure" of data "to the extent practicable" and acknowledges that many legitimate barriers to disclosure of data may exist. Id. Furthermore, ACUS contemplates scenarios where "data are not subject to legal or other protections" but where "the data's owners nonetheless will not provide such access." Id. In these cases, ACUS does not recommend that agencies deprive themselves of data. Rather, ACUS writes, "agencies should note [that the data's owner did not provide access] and explain why they used the results if they chose to do so." Id. Thus, the ACUS's policy in no way supports exclusion of studies simply because the underlying data is not publicly available. 
According to the commenters, this recommendation thus states only that agencies should "seek to provide disclosure" of data "to the extent practicable" and acknowledges that many legitimate barriers to disclosure of data may exist. Id. Furthermore, ACUS contemplates scenarios where "data are not subject to legal or other protections" but where "the data's owners nonetheless will not provide such access." Id. In these cases, ACUS does not recommend that agencies deprive themselves of data. Rather, ACUS writes, "agencies should note [that the data's owner did not provide access] and explain why they used the results if they chose to do so." Id. Thus, the ACUS's policy in no way supports exclusion of studies simply because the underlying data is not https://intranet.ord.epa.gov/sites/default/files/2020-01/Creating%20a%20Dataset%20in%20ScienceHub%20for%20EPA%20Public%20Access.pdf publicly available. 
Commenters (6137, 6916) state that the EPA contends that its proposed requirement to make data available is supported by the ACUS' Science in the Administrative Process Project. The commenters argue that the ACUS' policy does not support the exclusion of studies simply because the underlying data is not publicly available. The commenters assert that ACUS' policy supports the use of studies even when the underlying data cannot be made publicly available, including in cases where there are factors other than legal restrictions that limit the ability to publicly disclose data.
Response: The final rule does not reference the Administrative Conference of the United States (ACUS) Science in the Administrative Process Project or the Bipartisan Policy Center's Science for Policy Project.
With regard to concerns for cases that underlying data cannot be made publicly available, the Agency supports the use of restricted or tiered access for data and models that have confidential business information (CBI), proprietary data, or personally identifiable information (PII). Restricted or tiered access in this rule means that the underlying data or models are be available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
17.2.3 National Academies of Science (NAS) Reports
Comment: Commenters (4831, NAS, et al.) and other commenters (4591, 5172, 6046, 6924) express that EPA's contention that the National Academies of Science (NAS) Report(s) [Innovations in Federal Statistics: Combining Data While Protecting Privacy (NASEM, 2017b), and Federal Statistics, Multiple Data Sources, and Privacy Protections: Next Steps (NASEM, 2017c).] support the proposal is misleading. The commenters contend that there are important differences between the type of data that is the subject of the proposed rule and the data that is used strictly for statistical purposes that is the subject of the reports. 
Commenter (4831) explains that the EPA's proposed rule references the NAS reports to identify current approaches for protecting confidentiality while providing data for statistical purposes, such as those used by the Federal Statistical Research Data Centers. The commenter states that the reports consider the kinds of data that are typically collected and acquired under pledges of confidentiality for exclusively statistical purposes  -  pledges that are backed by strong statutory protections, with criminal penalties for violations.
According to the commenters (4831), there are several differences in the confidential microdata collected from individuals and businesses by federal statistical agencies through surveys, versus data and results from the kinds of studies that are within the scope of the EPA proposed rule. The commenters assert that these differences have important implications about making data publicly accessible. The commenters state that what works well in the federal statistical environment may not translate effectively to EPA, where stakeholders might be strongly motivated to discount study results that run counter to their regulatory preferences.
In addition, the commenters (4831) assert that the EPA's proposed rule ignores the inherent risks involved in data disclosure, the everchanging risk landscape, and the efforts needed to mitigate those risks  -  all of which are discussed in the cited National Academies reports. For example, the commenters provide that the security of data held by federal agencies is exposed to new and evolving threats. In addition to cybersecurity concerns, the commenters state that computer scientists and cryptographers have demonstrated that statistical analyses of data sets that generate highly precise results  -  such as geographic specificity or other characteristics that identify respondents  -  may result in privacy breaches (NASEM, 2017b; NASEM 2017c). The commenters report that this presents a new challenge that federal statistical agencies are just beginning to address.
Commenter (6924) asserts that the rule is not consistent with the principles of open science, inappropriately codifies how science should be conducted, and codifies science-policy decisions in direct conflict with consensus reports from the NAS. The commenter reports that the EPA cites NAS studies and deliberations on the very important issue of data sharing and transparency as supporting the proposal, but the commenter asserts that the NAS reports contain conclusions that are in contradiction to the rule, such as:
 Clear statements that unfettered public access to data for any purpose, as proposed in the rule, is inappropriate, risky and unjustified. For example their reports state: "Since unrestricted access can cause harm to individuals and also conflicts directly with respect for individual autonomy, it is not an appropriate policy," and "The panel concludes that no one way is optimal for all data users or all purposes," and "The actual data collected for statistical purposes from households, individuals, business establishments, and other organizations through censuses and surveys under a pledge of confidentiality are never made available to users." 
 Acknowledgement that all parties' interests in and purposes for performing an analysis and/ or validation of data would not be beneficial, as assumed in the rule, and thus protections are required. For example: "If not carefully considered, a reanalysis may be used as a tool to delay action. It also requires the participation of the original investigators," and "[The panel recommends] restriction of access to public-use data to those who agree to abide by the confidentiality protections governing such data and the institution of meaningful penalties for willful misuse of those data." This recommendation is meant to address potential use of data for non-research purposes, such as by marketers and companies for advertising.
 Assertion that simple de-identification of private information is not enough to protect an individual's data, which is counter to statements in the rule: "Statistical disclosure involves using data available outside the survey to breach the protection thought to have been afforded a survey data set by various data deletion and masking techniques."
Commenter (4591) reports that the recommendations from one report cited in the proposed rule of the National Academies of Science, Engineering, and Medicine, "Optimizing the Nation's Investment in Academic Research," were incorporated into the 21st Century Cures Act and discusses the potential burdens of data sharing policies, but it does not suggest that a policy such as the one being proposed by the EPA would be beneficial.
Commenters (5172, 6046) report that the proposed regulation cites a panel report published by the National Academies: "Expanding Access to Research Data: Reconciling Risks and Opportunities." According to the commenters, rather than providing support for the proposal, the National Academy Report conflicts with the proposal. The commenters explain:
 The executive summary of the report makes an important statement: "Research using detailed confidential data is needed ... for well- informed policy making ..." EPA's proposal conflicts with this statement; the proposal would deny the agency the use of studies that rely on confidential data. 
 None of the report's 19 recommendations call for the government to stop relying on studies that involve confidential data.
Response: EPA supports making data and model publicly accessible to allow for reanalysis for independent validation. EPA is also committed to protecting CBI and PII according to all existing rules and regulations. The Agency supports the use of restricted access for data and models that have CBI, proprietary data, or PII. Restricted access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The EPA would like to emphasize that this rule does not impose requirements on any entity outside of EPA. This is a rule of internal procedures and does not direct or require any outside entity or EPA to establish data sharing mechanisms. Further, the rulemaking does not require the EPA to collect, store, or publicly disseminate data and models underlying pivotal science.
The studies noted in these comments above and/or referenced in the 2018 proposed rule are not cited in the final rule or its preamble.
17.2.4 National Research Council Reports
Comment: Commenter (6916) contends that two National Research Council Reports EPA cites also fail to support requiring release to the public of "data and models underlying scientific studies that are pivotal to the regulatory action." According to the commenter, far from supporting the concept of allowing reliance on studies only where the data are available to the public, these reports suggest "exploiting the research potential of microdata and maintaining acceptable levels of confidentiality" and advocate "nuanced case-by-case judgment[s]" for releasing underlying data. Nor, according to the commenter, do the articles the Agency cites on the reproducibility of studies support its Proposal, and EPA does not explain why it thinks they do. In fact, the commenter asserts that at least one of the cited articles, by Steven Goodman, is entirely irrelevant - it does not demonstrate that there is a problem with the reliability of science in support of Agency decision making, nor does it attempt to. Rather, the commenter states it discusses the confusion in the lexicon about reliability and attempts to more properly define the term "reproducibility." The commenter asserts that in no way does it support eliminating consideration of studies because the underlying data is not publicly available. 
Response: EPA shares with the National Research Council, support for making data and models publicly accessible to allow reanalysis for independent validation. The Agency supports the use of restricted access for data and models that have CBI, proprietary data, or PII. Restricted access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. See Section III.A.1 of the preamble to the final rule for a full discussion of how the scientific community has embraced greater data transparency and numerous cited examples that support data transparency.
17.2.5 Health Effects Institute (HEI)
Comment: Commenter (5015) agrees with the continuing need to enhance transparency and data access, but notes that these issues are not new, and have been addressed now for over 15 years by administrations from both parties and by the scientific community: 
 This has included Guidelines for the Information Quality Act adopted by the Office of Information and Regulatory Affairs (OIRA) in 2002, numerous actions by the scientific community and journals to enhance access, and most recently the requirements for enhanced data access across the Federal Government promulgated by the Office of Science and Technology Policy (OSTP) in February 2013
 The commenter would strongly urge EPA to review the progress already made under these several major initiatives, and to carefully consider whether or not there are additional efforts that could further enhance transparency, before proceeding with a final rule.
Commenter (6912) states that the president of the Health Effects Institute noted that because it is costly and difficult to protect patient privacy, the kind of data reanalysis the rule promotes should only be done sparingly. According to the commenter, while the Health Effects Institute has a policy not to take public positions on regulatory questions, its president stated that the Institute's own data policy, which EPA claims supports the proposed rule, is "not quite what 's being proposed here.",
Response: As discussed in the preamble to the final rule, the EPA's Open Government Plan 5.0 also details the EPA's progress in implementing the tenets of the numerous data transparency initiatives in the federal government prior to 2018, including the Office of Management and Budget (OMB) M-10-06, the Office of Science and Technology Policy Memorandum of February 22, 2013, and OMB M-13-13. The journal articles and their underlying data resulting from intramural and extramural research funded by EPA are made available to the public.
EPA is committed to protecting CBI and PII according to all existing rules and regulations. The Agency supports the use of restricted access for data and models that have CBI, proprietary data, or PII. Restricted access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information. The EPA would like to emphasize that this rule does not impose requirements on any entity outside of EPA. This is a rule of internal procedures and does not direct or require any outside entity or EPA to establish data sharing mechanisms. Further, the rulemaking does not require the EPA to collect, store, or publicly disseminate data and models underlying pivotal science.
It should be noted that this final rule does not require reanalysis of a study's underlying data. Lewandowsky et al. (2020) evaluated the cost-effectiveness of reanalysis studies under various scenarios and concluded that reanalysis studies are most cost-effective when they are focused on studies of the greatest interest to the scientific community (in this study, the number of citations was a surrogate for interest). This finding is consistent with results in other studies that found and encouraged narrowing the focus of attempted reanalysis studies to those studies of greater significance (see preamble of final rule). In this final rule, rather than considering all studies that support the assessment of the relationship of a dose or exposure of a pollutant, contaminant, or substance to the magnitude of a predicted health or environmental impact as "pivotal science," the EPA is balancing transparency and feasibility by focusing on those studies that inform the quantitative relationship between the dose or exposure of a pollutant, contaminant, or substance and an effect.
17.2.6 Center for Open Science
Comment: Commenter (6137) provides that the Center for Open Science does not advocate for excluding science merely because the underlying data is not public, but rather counsels flexibility in determining what level of transparency is appropriate for a given study. The commenter explains that its Transparency and Openness Promotion (TOP) Guidelines for journals "recognize[] that not all of the standards are applicable to all journals or all disciplines. Therefore, the commenter explains that rather than advocating for a single set of guidelines, the TOP Committee defined three levels for each standard." These levels "provide flexibility for adoption depending on disciplinary variation." The commenter further explains that the Center for Open Science has invited journals to "suggest revisions that improve the guidelines or make them more flexible or adaptable for the needs of particular subdisciplines." As with other organizations upon which EPA relies, it too has adopted a flexible approach to transparency and thus provides no support for EPA's rigid exclusion.
Commenter (6912) states that the executive director of the Center for Open Science, whose policies were cited as supporting the proposed rule, stated that it was "20 percent off the rails." 
Commenter (1589) states that the proposed rule cites open science advocates in footnote 10 and changes made in scientific journals in footnote 11, without understanding them or discussing their recommendations and conclusions.
Commenter (6360) states that, while all EPA applications of science should be covered by this rulemaking, they recommend that EPA draw from (or adopt) existing transparency screening tools developed by consensus in the scientific community to further define the scope and applicability of the rule's requirements.
Specifically, the commenter (6360) states that, many scientific institutions have developed requirements that scientists must follow to publish their claims in participating journals. The commenter states that there are several examples of such systems that have been developed with minimum levels of transparency to ensure a claim can be objective and reproducible.
The commenter (6360) points out that one important, open source system, the TOP guidelines, has adopted by over 5,000 major scientific organizations and journals. The commenter states that the TOP guidelines include eight standards, each with three levels of increasing stringency. The commenter states that Journals select which of the eight transparency standards they wish to adopt for their journal and select a level of implementation for each standard. For example, the commenter states that the scientific journal Science adopted most of the Level III TOP standards effective January 1, 2017. According to the commenter, claims made in a journal requiring Level III disclosure are more likely to be recognized as scientific facts than those published to Level I standards and the TOP Level II and Level III requirements mirror somewhat the applicability of the proposed standards in EPA's rulemaking. The commenter recommends that EPA require Level III transparency for all significant regulatory actions, and that all other EPA policy decisions have at least Level II standards for transparency to the public. The commenter states that as EPA takes comment on a proposed decision and notes that the underlying judgements in the applied science meet Level II criteria, the public could evaluate the claims and provide valuable feedback to EPA regarding the appropriate rigor of its data transparency procedures.
The commenter (6360) asserts that the TOP criteria require researchers to post data, analytic methods, and research materials to a trusted depository. The commenter recommends that for EPA applications of science, this depository should be the public docket of a proposed rule or the accompanying administrative record associated with a proposed decision. However, for sensitive information, the commenter recommends that EPA adopt the depository standards in common use by other federal agencies, journals, and EPA itself.
Response: EPA shares with the third-party organizations, support for making data publicly accessible to allow reanalysis for independent validation.
The Agency agrees that it is important not to categorically exclude any study because the underlying dose-response data at the stage required by this rulemaking may not be available for independent validation. EPA acknowledges and agrees with commenters that there may be pivotal science where the underlying data are not publicly available or available through restricted access. The final rule is limited to dose-response data and, as no studies are categorically excluded from consideration, the EPA will continue to rely on the full body of the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. The EPA will also consider the degree to which pivotal science is available and will give greater consideration to pivotal science for which the underlying dose-response data are available, as described in the rule.
The Agency supports the use of restricted access for data and models that have CBI, proprietary data, or PII. Restricted access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (6362) suggests that EPA revise its statement that the proposed rule "takes into consideration the policies or recommendations of third-party organizations who [sic] advocated for open science." The commenter states that the recommendations referenced by EPA actually emanate from a survey of the members of three professional organizations whose memberships represent repositories of knowledge and experience in regulatory assessment. As such, the commenter requests that reference 10 in EPA's proposal be revised to also cite the publication, Expert Opinion on Regulatory Risk Assessment, A Survey by the Center for Media and Public Affairs (CMPA) and Center for Health and Risk Communication (CHRC) at George Mason University (December 6, 2013). 
Response: Thank you for the recommendation. The final rule does not cite these references.
17.2.7 Risk Assessment Specialty Section of the Society of Toxicology and Dose-Response Specialty Group (DRSG) of the Society for Risk Analysis (SRA)
Comment: Commenters (1945, 1949, 6137) dispute that the EPA cites "policies and recommendations from . . . members of the Risk Assessment Specialty Section of the Society of Toxicology, the Dose Response Section of the Society for Risk Analysis, and the International Society for Regulatory Toxicology and Pharmacology[.]" 83 Fed. Reg. at 18,770, n.10. The commenters assert that responses to an online poll by individuals do not represent endorsement of Society of Toxicology (SOT), Dose-Response Specialty Group (DRSG) of the Society for Risk Analysis (SRA). 
Commenter (6137) asserts that, in reality, these "policies and recommendations" are merely responses to an online questionnaire by a limited number of members of the three societies. The commenter recommends that the EPA not rely on a limited set of responses to an online poll, especially when the Agency does not possess information about conflicts of interest among respondents. 
Commenter (1945, SOT) objects to Footnote 10. The commenter argues that the footnote ascribes support for the proposed rule from "members of the Risk Assessment Specialty Section of the Society of Toxicology." The commenter states that this is incorrect. The commenter, who represents the SOT, asserts that while our individual members are free to support or oppose issues as they see fit, it is not appropriate to infer that this constitutes an endorsement from either the Specialty Section or the SOT as a whole. The commenter respectfully requests that any and all references to "members of the Risk Assessment Specialty Section of the Society of Toxicology" be removed from the Final Rule. 
Commenter (1949), the current Chair of the DRSG of the SRA, reports that the DRSG has never adopted any "policies or recommendations" on this or any other topic (see http://www.sra.org/drsg). According to the commenter, although "members of" the DRSG may have made statements on this topic in their personal or professional capacities, views of individual members should not be misconstrued as representing the view of the DRSG or of the SRA. Moreover, the commenter explains that the DRSG has a broad membership (including employees of the Environmental Protection Agency) that may have diverse views on this topic.
Commenter (1949) requests the following:
 In the Final Rule and any accompanying preamble text, EPA removes any and all references to the "Dose-Response Section [sic] of the Society for Risk Analysis."
 In the comment response prepared for the rule, EPA clarify the record that "third party organizations" whose policies and recommendations were considered do not include the Society for Risk Analysis or the Dose-Response Specialty Section.
Response: The final rule does not include references to the members of the Risk Assessment Specialty Section of the Society of Toxicology, the Dose Response Section of the Society for Risk Analysis, and the International Society for Regulatory Toxicology and Pharmacology.
17.2.8 Bipartisan Policy Center Report
Comment: The Bipartisan Policy Center (BPC) (0670, 6126) and other commenters (6133, 6137, 6166, 6880, 6915, 6916, 9227) assert that the proposed rule is not consistent with the BPC Report. The commenter suggests that EPA follow the procedures the BPC took to developing the report and consider the recommendations in the report which are relevant to EPA.
Commenter (0670, BPC)) reports that the EPA's press release for the proposed rule states that "Strengthening Transparency in Regulatory Science" is "consistent with ... recommendations from" the BPC Report. The commenter further reports that the "Background" section of the Federal Register notice for the proposed rule states: "The proposed rule takes into consideration the policies or recommendations of third-party organizations who advocated for open science" and cites the BPC Report, among others. The commenter states that they want to be clear that the proposed rule is not consistent with the BPC Report in substance or intent. Other commenters (6133, 6166, 6915, 9225, 9227) referred to the BPC comment stating that EPA had mischaracterized the BPC Report. One of these commenters (6915) states that EPA's false assertion of consistency with the policies and positions of leading science groups misleads the public and inhibits their informed participation
Several commenters (0670, 6166, 9225, 9227, 6125, 6137) assert that while the Science for Policy Project panel encouraged greater transparency and access to data, the report never suggested excluding studies from consideration in developing regulations if data from those studies were not publicly available. The commenters assert that the BPC notably concluded in its comments on this proposal that the proposal misrepresented its conclusions:
      While the Science for Policy Project panel encouraged greater transparency and access to data:
      The report never suggested excluding studies from consideration in developing regulation if data from those studies were not publicly available. Indeed, the panel's overarching recommendation for assembling the "best available science" reads: "Agencies and their scientific advisory committees should cast a wide net (emphasis added) in reviewing studies relevant to regulatory policy, and should make their methods for filtering and evaluating those studies more transparent."
Commenter (6880) states that the BPC's Science for Policy Project states "[s]tudies used in the formulation of regulation should be subject to data access requirements equivalent to those under the Data Access Act (Shelby Amendment) and its implementing circular regardless of who funded the study." The commenter asserts that the implementing circular is OMB Circular A-110 (https://www.gpo.gov/fdsys/pkg/FR-1999-10-08/pdf/99-26264.pdf), which directs federal agencies to use and share research data but specifically notes that "research data" do not include trade secrets and "[p]ersonnel and medical information and similar information the disclosure of which would constitute a clearly unwarranted invasion of personal privacy." (https://bipartisanpolicy.org/wpcontent/uploads/sites/default/files/BPC%20Science%20Report%20fnl.pdf.)
Commenter (6916) states that the BPC Report cited by the Agency recommends that decisionmakers should "cast a wide net" and "base policy on a thorough review of all relevant research and provisions of the relevant statutes," giving peer reviewed papers "great weight," but nowhere promotes the idea that sensitive or confidential raw data should be publicly released as suggested by EPA's proposed rule. According to the commenter, to the contrary, the paper recommends reliance on existing authorities and guidance, such as the Data Quality Act and the OMB Guidelines.
Commenter (6103) references the fact that the think tanks whose research was used to justify this new rule have said that they do not support the proposal. As an example, the commenter states that the BPC, whose research was used to support the supposed need for transparency, has stated in their comments on this proposal that the substance and intent of their research is not consistent with the administration's new policy. The commenter states that the BPC report from 2009 cited by EPA in the proposed rule supports looking at a wide array of data and making the method of evaluation more transparent, which does not coincide with the 'transparency' the EPA proposed rule seeks to promote. The commenter asserts that the EPA should use the best available science in the most effective way to truly fulfill its mission of protecting human health and the environment.
Response: The final rule does not reference the Bipartisan Policy Center's Science for Policy Project for policies and recommendations from third party organizations that were taken in consideration. The Bipartisan Policy Center is mentioned once, but only in noting that the Center was mentioned by a public commenter.
The Agency agrees that it is important not to categorically exclude any study because the underlying dose-response data at the stage required by this rulemaking may not be available for independent validation. EPA acknowledges and agrees with commenters that there may be pivotal science where the underlying data are not publicly available or available through restricted access. The final rule is limited to dose-response data and, as no studies are categorically excluded from consideration, the EPA will continue to rely on the full body of the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. The EPA will also consider the degree to which pivotal science is available and will give greater consideration to pivotal science for which the underlying dose-response data are available, as described in the rule.
The Agency supports the use of restricted access for data and models that have CBI, proprietary data, or PII. Restricted access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
17.2.9 EPA's Proposal is inconsistent with policies adopted by major scientific journals
Comment: Commenters (4877, 4596, 6103, 6109, 6125, 6133, 6137, 6144, 6153, 6166, 6168, 6351, 6358, 6891, 6912, 6937, 8275, 9224, 9227) state that EPA's claim that the proposed rule is informed by the policies recently adopted by some major scientific journals (EPA cites Proceedings of the National Academy of Sciences, PLoS ONE, Science, and Nature) is incorrect and that the EPA's proposal is inconsistent with policies adopted by major scientific journals.
Commenter (9227) notes that while scientific journals and other institutions have encouraged making data and models publicly available, there is widespread recognition in the scientific community that doing so is often legitimately constrained due to legal and ethical protections on the confidentiality and privacy of data, or because the data is unavailable. Moreover, the commenter states that no scientist or scientific organization supports the Proposal's approach of excluding research for which the underlying data cannot be disclosed. According to the commenter, none of the materials EPA cites support such an extreme approach. To the contrary, the commenter states that the scientific community recognizes that the quality of a study is not determined by whether the underlying data is publicly available and has long utilized a variety of tools for ensuring the integrity and rigor of research findings. For all these reasons, the commenter notes that numerous representatives of the scientific community -- including editors of the very scientific journals whose policies EPA cites to in the Proposal, the American Association for the Advancement of Science, members of the Science Advisory Board (SAB), and other scientists cited to by EPA -- have already voiced serious concerns about the Proposal.
Commenter (6133) contends that these various journal policies are more flexible in their terms for data sharing and nuanced in their practical approaches than what EPA fundamentally misrepresents in the Proposal and that an examination of these sources indicates, in fact, that the language of the Proposal is not consistent with best practices and is unworkable in practice. 
Commenter (6137) asserts that EPA provides no information as to how these journals' policies supposedly informed the data access guidelines and policies.
Commenter (4877) states that the data sharing policies that these journals use explicitly recognize the wide variety of workflows in use across scientific disciplines and provides for data sharing at different levels of openness. The commenter notes that they expressly state that not in every case can all data be shared, and in some cases, it is permissible for data to not be shared at all for legitimate and stated reasons. According to the commenter, as these journals  -  and the research community as a whole  -  well knows, this does not mean that the data underpinning these studies cannot be assessed. Commenters (6153, 6358, 6874) assert that reviewers can judge the merits of studies relying on non-public data, because "scientists are trained in assessing research publications by judging the articulation and logic of the research design, the clarity of the description of the methods used for data collection and analysis, and appropriate citation of previous results."
Commenter (6109) contends that EPA is proposing to dismiss from consideration research that would be admissible to these major scientific journals based upon an incorrect interpretation of their data-sharing policy and a disregard for the rigors of peer review.
Several commenters (4596, 6103, 6109, 6125, 6137, 6144, 6153, 6166, 6168, 6351, 6358, 6891, 6912, 6937, 8275, 9224, 9227) refer and/or support a statement submitted by the editors-in-chief of Science, Nature, PLOS, Proceedings of the National Academy of Sciences , and Cell issued in response to this proposed rule stating that this policy will exclude important studies from consideration in the rule-making process, thus adversely impacting the decision-making process. They note that these organizations concluded in their issued joint statement on the proposal:
      [I]t does not strengthen policies based on scientific evidence to limit the scientific evidence that can inform them; rather, it is paramount that the full suite of relevant science vetted through peer review, which includes ever more rigorous features, inform the landscape of decision making. Excluding relevant studies simply because they do not meet rigid transparency standards will adversely affect decision-making processes.
(http://science.sciencemag.org/content/early/2018/04/30/science.aau0116) 
Commenter (6125) states that, clearly, the joint statement indicates that their policies on publication offer no support for the premise that studies are unreliable unless their underlying data is made publicly available. 
Commenter (6144) argues that the EPA is incorrect in claiming that they are following accepted practice on open science that major science journals and the NAS have adopted. The commenter states that the open data initiatives adopted by these journals acknowledge that Transparency and Openness of Promotion (TOP) standards "recognize the array of workflows across scientific fields and make the case for data sharing at different levels of stringency; in not every case can all data be fully shared." The commenter states that these standards are in accordance with OSTP's 2013 memo Increasing Access to the Results of Federally Funded Scientific Research, which notes the importance of making data available in realizing the goal of increased transparency in the federal government, while protecting confidentiality, personal privacy, CBI, and "preserving the balance between the relative value of long-term preservation and access and the associated cost and administrative burden."
Commenter (6163) states that the journal policies and the TOP standards do not inhibit the use of research if the underlying data is not able to be published, such as sensitive PII and CBI.
Commenter (6164) asserts that the proposed policy wrongly claims to parallel TOP guidelines adopted by peer-reviewed scientific journals like those managed by the commenter (American Geophysical Union (AGU)). According to the commenter, unlike EPA's proposal, the TOP standards encourage open data while recognizing that each scientific discipline has its own constraints, as expressed by editors-in-chief of peer-reviewed publications cited in the proposed rule. The commenter states that, at times, underlying scientific information cannot be made public, particularly when it includes personal identifiable information that ethically and/or legally cannot be shared. The commenter states that even in these cases, the merits of a scientific study can still be evaluated by reviewers during the peer-review process and by scientists who are trained to assess scientific conclusions.
The commenter (6168) states that, at the very least, transparency requires careful and explicit discussion of the data, assumptions and boundary conditions, sources of error, accuracy, precision, and statistical significance. The commenter states that scientific journals also require introductory sections to orient the reader to the purpose of the study and the evidence from which the study is designed, methods sections to precisely define and describe the study design, and discussion sections to articulate results, their limitations, and their implications. Scientific journals also require extensive contextualization, and to take their data transparency requirements out of context is disingenuous and damaging. According to the commenter, providing raw data without context can cloak scientific research as much as providing no data; providing raw data encourages not transparency but even greater manipulation, which can be used to produce false uncertainty.
Commenter (6133) asserts that the proposed rule is not, as it claims, "consistent with requirements for many scientific journals." (EPA cites Taylor & Francis, Elsevier, PLOS and Springer Nature) Specifically, the commenter points out that the Taylor and Frances journal policy for data transparency is much more nuanced than EPA claims and offers a range of options for data submission, demonstrating the need for flexibility and discipline-specific concerns with respect to the public sharing of sensitive data. Additionally, the commenter states that the Springer Nature Research Data Policy is similarly flexible, describing requirements across a spectrum for four types of underlying research data. The commenter states that for only one of four types of research data is data sharing required as a condition for publication. The commenter states that the frequently asked questions document for the Springer Nature Data Policy notes that "[t]he policies apply to all research that support publications but reasonable restrictions on data availability are permitted to protect human privacy, biosafety or respect reasonable terms of use for data obtained under license from third parties." The proposed rule's categorical exclusion and prohibition are thus, according to the commenter, flatly inconsistent with the Springer Nature Research Data policy cited in the proposed rule.
The commenter (6133) also states that the Springer Nature polices began during the first quarter of 2016 and did not apply retroactively, as the Proposal would. 
Commenter (6133) asserts Elsevier's corresponding policy is optional for authors, and states that the journal: "will . . . [e]ncourage and support researchers to share research data where appropriate and at the earliest opportunity, for example by enhancing our submission processes to make this easier." The commenter reports that a frequently asked questions page further explaining this policy says that the "policy is clear in that we encourage and support authors to share their research data rather than mandating them to do so and provide tools and services to enable them to do this effectively. Where there is community support for (often discipline-specific) mandates regarding data deposit, submission and sharing, some of our journals may reflect this with their own mandatory data sharing policies." The commenter states that this same supporting frequently asked questions resource from Elsevier says that Elsevier "respect[s] authors who need to keep research data under embargo." The proposed rule, by contrast, according to the commenter, does not allow researchers to keep their research data under embargo. Nor does the proposed rule offer such discipline-specific flexibility and, as a result, is neither practically workable nor consistent with the policies of the world's leading scientific journals. 
Commenter (6133) explains that the Elsevier policy does not apply strict data release requirements to include publicly accessible information. It says that "[r]esearch data should be made available free of charge to all researchers wherever possible and with minimal reuse restrictions." It further states that "[r]esearchers should remain in control of how and when their research data is accessed and used, and should be recognized and valued for the investments they make in creating their research data and making it available." The commenter states that, under the proposed rule, researchers retain no such control over their data; the proposed rule ignores these harmful ramifications. 
Commenter (6133) contends that the PLOS Data Availability Policy notes that, for studies involving human participants, "data must be handled so as to not compromise study participants' privacy." The PLOS Policy itself links to the National Institutes of Health Data Sharing Workbook, which states that: 
      It is rarely sufficient to simply remove names, addresses, telephone numbers, Social Security Numbers, and the like. Deductive disclosure of individual subjects becomes more likely when there are unusual characteristics or the joint occurrence of several unusual variables. Samples drawn from small geographic areas, rare populations, and linked datasets can present particular challenges to the protection of subjects' identities.
To the extent data availability, even broadly defined, is contemplated in the Proposal, the commenter (6133) contends that it is done so prospectively, not retroactively. Unlike the Proposal, the PLOS Policy does not apply retroactively to all relevant studies: "[t]he data policy was implemented on March 3, 2014. Any paper submitted before that date will not have a data availability statement. However, for all manuscripts submitted or published before this date, data must be available upon request." 
Response: The EPA agrees with commenters that the scientific community and government agencies are making great strides in data transparency; however, improvements can still be made over existing policies and mechanisms. Many scientific publications, for example, require authors to make a data availability or data access statement, which discloses where and under what conditions the underlying study data are available. Yet the EPA cannot solely rely on data availability statements made in published research because initiatives toward greater data sharing and transparency amongst scientific journals and international organizations are still being implemented, are inconsistently enforced, and the true accessibility of data in a public repository is still limited (Refs. 28, 29, 30, 31, 32, 33). For example, Christensen et al. (2019) evaluated 1,072 peer-reviewed articles and "found that rates of data availability for empirical articles published after journals adopted data-sharing policies differ widely between journals, from 0 percent to 83 percent, with a mean of 35 percent" (Ref. 32). Stodden et al. (2018) noted they were only able to retrieve the dataset and code for 44 percent of the 204 computational studies published in Science in the 16 months after the publisher instituted its data availability requirements (Ref. 34). Therefore, the rule requirements for the EPA's independent evaluation of the availability of data are necessary and critical to prioritizing data transparency in the pivotal science underlying its significant regulatory actions and influential scientific information. Finally, focusing the final rule requirements on the underlying dose-response data is intended to address public comments concerning clarity of the rule, potential unintended consequences, and the potential for far-reaching impacts. The requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings. 	
The Agency supports the use of restricted access for data and models that have CBI, proprietary data, or PII. Restricted access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
The Agency agrees that it is important not to categorically exclude any study because the underlying dose-response data at the stage required by this rulemaking may not be available for independent validation. EPA acknowledges and agrees with commenters that there may be pivotal science where the underlying data are not publicly available or available through restricted access. The final rule is limited to dose-response data and, as no studies are categorically excluded from consideration, the EPA will continue to rely on the full body of the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. The EPA will also consider the degree to which pivotal science is available and will give greater consideration to pivotal science for which the underlying dose-response data are available, as described in the rule.
The EPA would like to emphasize that this rule does not impose requirements on any entity outside of EPA. This is a rule of internal procedures and does not direct or require any outside entity or EPA to establish data sharing mechanisms. Further, the rulemaking does not require the EPA to collect, store, or publicly disseminate data and models underlying pivotal science.
 This final rule applies prospectively to significant regulatory actions and influential scientific information and has no retrospective effect on existing significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. 
The final rule and preamble do not cite PLoS ONE. Science. and Nature are mentioned once, but only in noting that each were mentioned by a public commenter.
17.2.10 Not Representative of Scientific Community Recommendations
Comment: The commenters (6891) assert that the EPA apparently did not consult with the individuals and organizations it cited in support of its proposed rule, and ignored the scientific community, essentially all of which opposed the rule. According to the commenters, the disconnect between EPA's proposed rule and the scientific community suggests one of two things. At best, EPA didn't understand the scientific community's recommendations for handling challenges related to reproducibility. At worst, EPA understood perfectly well that the scientific community was not proposing to exclude public health studies from rulemaking; but chose to distort the scientific community's efforts for its own industry-championed; anti -  science. campaign. The commenter asserts that, in either case, EPA's justification for the proposed rule "runs counter to the evidence."
Commenter (6144) expresses concern that among those not consulted in the crafting of this rule was the National Academies of Science, Engineering, and Medicine (NASEM), which has held several committee meetings and carried out a series of reports detailing how scientific literature can be evaluated transparently without the full disclosure of underlying datasets. The commenter reports that, in a comment on the rule, NASEM urged EPA to seek objective and expert guidance on the rule and its implementation and offered itself as an independent review body. The commenter also states that the BPC was also cited in EPA's proposed rulemaking and had to clarify in a comment to the agency that "the proposed rule is not consistent" with its report on the use of science in policymaking that EPA cited in "substance or intent." The commenter states that, while BPC supports enhanced transparency, "the report never suggested excluding studies from consideration in developing regulation if data from those studies were not publicly available." 
According to the commenter (6144), the proposal has faced widespread opposition from the scientific community, academic institutions, states, and decisionmakers. In fact, the comment reports that nearly 1,000 scientists asked former EPA Administrator Scott Pruitt to abandon this proposal, arguing it "would greatly weaken EPA's ability to comprehensively consider scientific evidence," and undermine the agency's capacity to protect Americans from serious health threats.
Commenter (6909) states that there has been robust discussion in the scientific community about how to improve both reproducibility and replicability, and numerous recent articles and papers have addressed this topic. The commenter reports that various entities  -  including groups of researchers, scientific societies, funders, and the federal government  -  have begun to develop policies to facilitate data sharing and improve both reproducibility and replicability. For example, the commenter reports that the American Association of Universities and the Association of Public Land Grant Universities convened a working group on public access for federal agencies and universities. Peer reviewed scientific journals have joined this effort as well, with many adopting policies designed to encourage data sharing. As an example, the commenter reports that in 2015 the Center for Open Science developed a set of Transparency and Openness Promotion (TOP) guidelines, which were published in Science Magazine. 
In addition to the TOP standards, the commenter (6909) reports that scientific journals have begun to develop other innovative techniques for encouraging data-sharing, such as affixing "open-data badges" to articles whose data are available. Notably, the commenter states that the EPA itself has had an Open Government Policy since 2010. The most recent iteration of this plan includes efforts to increase public access to EPA-funded research publications and their related data. The commenter reports that other federal agencies have also developed and implemented policies aimed at increasing public access to the results of research funded by the federal government." The commenter adds that there is an active and ongoing dialogue about what policies to put in place in order to improve transparency and reproducibility while simultaneously protecting confidentiality and intellectual property. The commenter (6909) concludes that, if EPA is truly concerned about improving transparency and reproducibility while protecting confidentiality and intellectual property, it should engage in this dialogue and with the scientists who are having it, and it should leave room for a realistic and appropriate transition period. 
Response: Most public commenters on the purpose of the 2018 proposed rule and the 2020 SNPRM supported the concept of greater transparency, but questioned the "problem" the EPA was trying to fix. The EPA continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. This rule is designed to build upon OMB M-19-15, which highlights the need to characterize the sensitivity of an agency's conclusions to analytic assumptions, as well as other federal guidance documents that require greater data transparency. The EPA's attention to data transparency is also responsive to the broader interest in greater data and model transparency observed in the numerous transparency initiatives in the scientific community and federal government, as well as the criticism the EPA has received from members of the public, scientific community, and Congress on the transparency of the scientific basis for EPA's decisions in previous influential scientific information assessments and regulatory actions[,][,][,][,]. The EPA's continued progress toward maximizing transparency is vital to building and maintaining trust with the public and credibility in the Agency's decisions. 
The EPA's SAB reviewed the proposed rulemaking and issued a letter to the Administrator on April 24, 2020. The letter recognized the importance of establishing transparency and enhancing public access to data and models. The report stated it supported the adoption of the precept of, "of sharing accurate data and information to increase credibility, high-quality outcomes and public confidence in science."
17.2.11 Cited Economist Article Does Not Support EPA's Proposal
Comment: Commenter (6125) asserts that the Economist article cited by EPA in the proposed rule does not endorse EPA's proposal. The commenter contends that the Economist discusses the importance of the verification of scientific studies and how it is problematic that much current science cannot be replicated due to many causes. The article's proposed solutions include: tightening standards, particularly in statistics, registering research protocols in advance and monitoring them, and: "[w]here possible, trial data also should be open for other researchers to inspect and test." The commenter supports these statements.
Response: In the 2018 proposed rule, the EPA used the term "replicate" in the proposed regulatory text at 40 CFR 30.5 but did not define it at 40 CFR 30.2. The term replicate is no longer used as the EPA has decided, based on public comment and as discussed in the preamble, that the intent of the term in the proposed regulatory text at 40 CFR 30.5 was "reanalyze" rather than "replicate." In the 2020 SNPRM, the EPA proposed using the term "reanalyze" instead of "replicate" and proposed at 40 CFR 30.2 a definition for "reanalyze." Given that proposed 40 CFR 30.5 also included the term "independent validation" and that this term directly relates to "replicate," the EPA also proposed a definition at 40 CFR 30.2 for this term.
Comment: Commenter (6360) asserts that another advantage of using an existing system is that EPA would reduce the burden on researchers and applied science developers. The commenter explains that since TOP and other systems are widely-adopted now and are growing in their reach, scientists understand their obligations under these systems if they want to be published in certain journals. The commenter states that if they must turn in their data to be published in Science, for example, turning it over to EPA for public review is not a significant additional burden.
Response: This final rule describes how the EPA will determine the consideration to afford pivotal science of the EPA's final significant regulatory actions and influential scientific information for which the conclusions are driven by the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect based on the availability of the underlying dose-response data and other applicable factors. This rule exclusively pertains to the internal practices of the EPA and does not regulate the conduct or determine the rights or obligations of any entity outside the federal government. The final rule does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science. Rather, EPA is describing how it will handle studies based on whether the underlying data and models are publicly available.

 Chapter 18: Alternative Approaches to the Proposed Rule
The proposed rule preamble solicited public comment on whether additional or alternative sources of authority are appropriate bases for this proposed regulation. The proposal further states that the EPA believes that a generally applicable regulatory provision of the type proposed here is the appropriate vehicle to establish and implement the policies articulated in the proposal, in the interests of consistency, predictability, and transparency across the functions that EPA performs. The EPA specifically solicits comment on whether alternative or additional regulatory or other policy vehicles are appropriate to establish and implement these policies, and whether further regulatory or other policy vehicles at the programmatic or statutory level would be appropriate as alternative or additional steps the Agency may take to further the policies articulated in the proposal.
Several comments were received in response to EPA's solicitations. In this section these comments are organized according to the following topics:
       Base Policies and Requirements on Existing Guidance
       Approaches Specific to EPA-Funded Research
       Suggested Changes to the Rule Approach
       Alternative Policies to Increase Transparency in EPA Rulemaking without Releasing Private Data to Public
       Facilitate Data Publication/Accessibility
       Implement, Emulate or Work with Federal Programs
The specific comments provided in this Chapter reflect the comment letters that provide arguments supported by detailed rationale, examples, and references/documentation. The Docket ID numbers for these commenters are cited throughout this Chapter. In addition, some general comment letters may also include comments related to comments included and responded to in this Chapter. These comments are summarized in Appendix B. Note that points and issues raised in these general comment letters may also be raised by the more detailed comments summarized in this Chapter. The responses provided in this Chapter adequately address these more general comments.
18.1 Base Policies and Requirements on Existing Guidance
Comment: Commenters (0559, 6125, 6126, 6156, 6161, 6357, 6360, 6924, 6955) suggest that EPA base any transparency in science policies or requirements on existing guidelines, or industry-developed guidance/protocols instead of proceeding with the proposed rulemaking. Commenter (6360) suggests that adopting an existing system would reduce the burden on researchers and applied science developers since they already understand their obligations under these systems. Commenter (6924) lists three example protocols and guidelines that can be followed to improve the scientific basis of evaluating studies without releasing data to the public: CONSORT, ARRIVE and STROBE. The other commenters suggest that EPA base policies or requirements on the specific guidance or practices below.
Response: While EPA intends to build upon existing data sharing and peer review efforts both in the Federal government and broader scientific community, this rulemaking intentionally goes further. The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions.
EPA disagrees with the commenter that, if EPA were to adopt an existing data sharing system in place of the requirement of this rule, it would then reduce the burden on researchers. This final rule does not impose any burdens or requirements on external parties.
18.1.1 National Academy Guidance
Comment: Commenters (0559, 6357) urge EPA to follow guidance from the National Academy of Sciences (NAS) when constructing rules and policies governing data transparency and access.
Commenter (0559) urges EPA to construct rules governing transparency and data access that are consistent with the NAS report, Science and Decisions: Advancing Risk Assessment. The commenter states that this document has numerous recommendations related to transparency in regulatory actions, including recommendations regarding how scientific information should be collected, evaluated, and assimilated in risk assessments.
Commenter (6357) states that the National Academies of Sciences, Engineering, and Medicine (NASEM) have convened independent panels of experts to develop multiple comprehensive documents outlining robust standards and methods for systematic reviews and decision-making, including several guidance documents focused on the environment and human health.[,][,] The commenter encourages EPA to utilize systematic review methods and decision-making processes consistent with the relevant recommendations stemming from National Academies reports. The commenter also recommends that the review process be clearly established prior to the review being conducted, including criteria determining inclusion of studies and how studies would be weighted regarding the strength of evidence. According to the commenter, such transparency about the process would help meet the EPA's stated goal of strengthening "public confidence in the health and environmental protections underpinning EPA's regulatory actions."
Response: The EPA acknowledges the valuable recommendations and standards provided by the National Academies, and their impact on EPA practices. The EPA has considered recommendations from the National Academies of Sciences, and the Agency continues to believe that codifying internal procedures aimed at prioritizing transparency in significant regulatory actions and influential scientific information into regulation will improve the opportunity for the public to access the EPA's scientific analyses and resulting regulatory actions in a way that is beneficial to the scientific process, the Agency's mission, and the public's health and safety. 
EPA notes that this rule does not establish new requirements for systematic review protocols. In general, EPA intends to continue to follow its existing practices for identifying, reviewing, and evaluating available scientific evidence under the different environmental statutes EPA implements. Where there is convincing and well-substantiated evidence (consistent with Agency guidelines on hazard identification and dose-response assessment) to support a relationship between exposure and effect, the EPA will identify a subset of those studies for use in characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect. This will be based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both will be identified as pivotal science. Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of personally identifiable information (PII), confidential business information (CBI), or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation.
Comment: Commenter (6125) states that if EPA has issues with its current risk assessment and benefit analysis procedures that currently follow NAS recommendations and its own existing guidelines, a more straightforward and fruitful approach would be to update the guidelines as needed, showing in far more detail than provided in the Proposal the details of their reasoning, the conditions in which they should be employed, and the scientific bases for requiring consideration of alternative models in all risk assessments. The commenter suggests that draft guidelines should then undergo peer review by expert panels of the NAS and/or its own Science Advisory Board (SAB) and Science Advisory Panel (SAP). While guideline development is arduous and time consuming, the commenter notes that it has several advantages over rulemaking, including 1) guidance is developed by experienced risk assessors; 2) it is a science-based process that involves consensus building across the EPA as well as with the broader science community; and, 3) critically, it maintains the separation between risk assessment and risk management so that EPA's decision makers benefit from the availability of science that is produced in an independent and objective fashion. The commenter states that this has been the practice at EPA since 1983 when the NAS published Risk Assessment in the Federal Government: Managing the Process.
Response: EPA disagrees with the commenter's assessment that this rule is meant to address issues with the Agency's current risk assessment and benefit analysis procedures that currently follow NAS recommendations and its own existing guidelines. Rather, the purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. EPA intends to continue to follow its existing guidance and practices for identifying, reviewing, and evaluating available scientific evidence under the different environmental statutes EPA implements.
The Agency will develop internal guidelines as necessary to further develop the implementation of this rulemaking, and will do so in a manner consistent with Agency procedure. EPA notes for the commenter that it did consult with its SAB on the scientific and technical basis of this rule. 
18.1.2 U.S. Commission on Evidence-Based Policymaking Recommendations
Comment: Commenter (6126) suggests that EPA carefully consider adoption of the recommendations from the U.S. Commission on Evidence-Based Policymaking, their relevance to EPA, and opportunities to request new statutory authority to implement these recommendations. The commenter states that the recommendations of the commission are directly relevant to the context of this proposed rule because the commission offered recommendations focused on the agencies that fall within the jurisdiction of the Chief Financial Officers Act (CFO Act). According to the commenter, the recommendations have been endorsed by 36 former heads of federal statistical agencies (leaders who are charged with protecting government-collected confidential data) and numerous professional associations that represent more than 120,000 members of the U.S. evidence-building community. The commenter states that the implementation of the recommendations could also be valuable precursors to regulatory actions that affect how EPA will use scientific research in the future.
Commenter (6126) also encourages EPA to ensure any further actions related to the proposed regulation adhere to five principles that the commission unanimously agreed on for evidence-based policymaking:
          Privacy. Individual privacy and confidentiality must be respected in the generation and use of data and evidence.
          Rigor. Evidence should be developed using well-designed and well implemented methods tailored to the questions being asked.
          Transparency. Those engaged in generating and using data and evidence should operate transparently, providing meaningful channels for public input and comment and ensuring that evidence produced is made publicly available.
          Humility. Care should be taken not to over-generalize from findings that may be specific to a particular study or context.
          Capacity. The capacity to generate and use data and evidence should be integrated within government institutions and adequately funded and staffed.
Response: EPA takes seriously its charge to make evidence-based policy, and invites the commenter to learn more about EPA's implementation of the Foundations for Evidence-Based Policymaking Act of 2018 at https://www.epa.gov/data. The scope of this final rule is more narrowly focused than the five principles from the commission, as it does not alter how the Agency collects or handles data. Rather, the purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information.
18.1.3 Good Laboratory Practice
Comment: Commenter (6161) recommends that the Agency adopt a requirement that studies that warrant consideration as key or pivotal studies conform with Good Laboratory Practice (GLP) or "the spirit of GLP" to help ensure quality, rigor, transparency, and reproducibility. The commenter provides the following explanation:
      Whether in the "pure" sciences or regulatory sciences, the principal reason for requiring transparency of scientific data is to enable reproducibility studies and validation. For the Agency to require transparency but ignore findings of irreproducibility defeats the purpose. The Agency requires that scientific testing data submitted by industry as the basis for product registrations, hazard evaluations, or other purposes comply with GLP. The reason for the requirement is to ensure that the Agency's evaluations are based on data of sufficient quality, rigor and reproducibility. Recognizing that not all provisions specified by formal GLP programs can be applied to all studies, the concept of the spirit of GLP is often adopted by regulatory agencies and study sponsors. This concept should also be adopted by the Agency.
      The Organization of Economic Cooperation and Development (OECD) defines GLP as: (...a quality system concerned with the organisational process and the conditions under which non-clinical health and environmental safety studies are planned, performed, monitored, recorded, archived and reported.)
      The definition of GLP makes clear that the principles of GLP are applicable to all scientific studies, not just to "guideline studies." In the environmental sciences, the regulated community often adopts a requirement that a laboratory, contractor, or grantee follow "the spirit of GLP" as a condition of funding a study that may serve a regulatory purpose. In practice, conforming with the spirit of GLP involves preparation of a protocol that describes the details of a study. This description includes safety precautions, lists of required equipment, sampling procedures, chains of custody, analytical methods, sample archiving, QA and QC measures, calculating and reporting results, and predefining and documenting criteria for including or excluding data to help avoid bias. As a study progresses, changes may be required. A GLP-compliant protocol describes procedures for amending the protocol to record the changes. In this way, protocols promote successful study replication and assist in the types of detailed peer reviews that can (and should) be a feature of regulatory science. 
      In the environmental sciences, adherence to GLP principles is not uniform. Scientific studies sponsored by the regulated community to meet regulatory requirements are likely to conform with GLP. However, it has been PCTC's experience that studies in the environmental sciences conducted in some academic institutions and non-regulatory government agencies show little or no evidence of conforming with GLP. We have found protocols prepared by government agencies that would be rejected out-of-hand by EPA if submitted by industry in support of product registration or evaluation. Scientist-to-scientist requests and FOIA [Freedom of Information Act] responses sometimes demonstrate the complete absence of protocols or even something as simple as an outline of a study design. Quality control measures for analytical chemistry are generally in place, but quality management for other aspects of environmental studies seems to be the exception rather than the rule. 
      The lack of quality and integrity management in the environmental sciences in many government and academic research institutions indicates a lack of rigor in the discipline. While this may have been accepted during the 1970s and 1980s when many fields in the environmental sciences were in their infancy, it is unacceptable today. The Agency should not be making policy that impacts the lives and livelihoods of citizens based on science of unacceptably low quality or doubtful integrity. 
Response: EPA disagrees with the commenter that the Agency makes policy based on science of unacceptably low quality or doubtful integrity, and further disagrees that this rule is the venue by which to establish GLP requirements for pivotal science EPA considers in its final significant regulatory actions and influential scientific information. Assessing study quality is an essential part of EPA's existing systematic review practices. However, this rulemaking is more narrowly focused on improving the transparency of EPA assessment and regulatory decision-making, and therefore establishes how EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information.
18.1.4 Transparency and Openness Promotion Guidelines
Comment: Commenter (6360) recommends that while all EPA applications of science should be covered by this rulemaking, EPA should draw from (or adopt) existing transparency screening tools developed by consensus in the scientific community to further define the scope and applicability of the rule's requirements. The commenter specifically recommends adopting the Transparency and Openness Promotion (TOP) guidelines, requiring Level III transparency for all significant regulatory actions and Level II for all other EPA policy decisions. The commenter further suggests that when EPA takes comment on a proposed decision and notes that the underlying judgements in the applied science meet Level II criteria, the public could evaluate the claims and provide valuable feedback to EPA regarding the appropriate rigor of its data transparency procedures. Based on these comments, EPA could decide that the final agency action requires Level III transparency. The commenter provides the following background on TOP guidelines:
      The TOP guidelines have been recently adopted by over 5,000 major scientific organizations and journals. Published in Science in 2015, the TOP guidelines include eight standards, each with three levels of increasing stringency. Journals select which of the eight transparency standards they wish to adopt for their journal and select a level of implementation for each standard. For example, the scientific journal Science adopted most of the Level III TOP standards effective January 1, 2017.
       By clearly adopting a particular level to a specific application, a participating journal or organization allows the public to evaluate how likely claims made under its banner are scientific fact. Researchers know they must strive to meet Level III requirements and must take due care in their research to receive the benefits of publication in a high-impact journal with Level III requirements. Therefore, claims made in a journal requiring Level III disclosure are more likely to be recognized as scientific facts than those published to Level I standards.
       The Level II and Level III requirements mirror somewhat the applicability of the proposed standards in EPA's rulemaking. The tiered levels and specific requirements in TOP offers an existing, widely-used approach for EPA to consider that would allow it to prioritize its resources and its requirements with respect to applied science developers. EPA can calibrate the transparency requirements of particular science applications with the magnitude of the policy decision in question. 
Response: The EPA appreciates advances in transparency made in the scientific community, and believes that the requirements set out in this final rule are compatible with the TOP system. However, the EPA notes that this rule does not have a categorical prohibition on finalization of the significant regulatory action or influential scientific information if it does not meet Level III requirements. Rather, this rule requires that the EPA shall give greater consideration to pivotal science where the underlying dose-response data are publicly available in a manner sufficient for independent validation. The Agency shall also give greater consideration to pivotal science based on dose-response data that include CBI, proprietary information or PII if these data are available through restricted access in a manner sufficient for independent validation. For pivotal science where there is no access to dose-response data, or access is limited, the Agency may still consider these studies but will give them lesser consideration unless the Administrator grants an exemption under 40 CFR 30.9. These flexibilities are necessary to enhance the implementability of this rule, which is an incremental step in increased transparency for the Agency.
18.1.5 Radiation Effects Research Foundation
Comment: Commenter (6955) suggests that one option to consider with individualized health or other count date is the approach taken by the Radiation Effects Research Foundation (RERF) in its handling of the A-bomb survivor data. The commenter explains that the raw data is binned into multiple covariate categories and used for RERF papers, as well as for study public access which allows for considerable post-hoc analysis and protects individual identification of subjects, while limiting data dredging somewhat.
Response: The EPA acknowledges the example the Radiation Effects Research Foundation provides for handling A-bomb survivor data, but notes that this final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. 
18.1.6 Federal Data Strategy
Comment: Commenter (6126) encourages EPA to explore opportunities to use the proposed Federal Data Strategy (being developed as part of the President's Management Agenda) for improving its activities related to data infrastructure, management, and use. The commenter states that the OMB and working groups are requesting extensive feedback about how to develop the strategy and what it should contain, including partnerships with external organizations to host public forums.
Response: This rule is independent but complementary to the broader Federal Data Strategy. The EPA supports this and other data-sharing efforts and is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. The EPA is focusing on the underlying dose-response data for this rulemaking because of the influence these data have on particularly impactful decisions at the Agency.
18.1.7 National Center for Health Statistics Research Data Centers
Comment: Commenter (6924) proposes providing secure access to data for research purposes also improves data sharing as an alternative to requiring publicly available data. The commenter suggests that EPA consider the National Center for Health Statistics Research Data Centers as a model -- these centers have successfully provided access to sensitive information without breaches of privacy/confidentiality.
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available.
18.2 Approaches Specific to EPA-Funded Research
Commenters (1994, 2180, 2234, 6141, 6924, 6937) made suggestions for actions EPA could take to improve transparency in EPA-funded research instead of proceeding with the proposed rule. The suggestions provided fall into three categories: (1) suggestions to enforce existing requirements, (2) suggestions for updates to grant awards or cooperative agreements, and (3) suggestions for how research funding could be used to improve data transparency.
18.2.1 Enforce Existing Requirements
Comment: Commenter (1994) suggests that EPA can take incremental actions to improve the transparency of scientific studies sponsored through EPA research grants. For example, the commenter suggests that EPA can continue to improve how it enforces requirements for awardees of EPA grants to document research protocols and data management plans.
Commenter (6937) suggests that EPA could more vigorously implement the existing requirements for transparency and submission to data archives where possible for grant programs.
Response: In addition to this rule, the EPA is actively implementing its Open Access Plan and invites the commenter to learn more about how the EPA is improving access to the research it funds. This rule is focused on strengthening the transparency of the science the Agency considers in its significant regulatory actions and influential scientific information. The scope of this rule consequently expands beyond only the science which the EPA funds.
18.2.2 Update Grant Awards/Cooperative Agreements
Comment: Commenter (6924) suggests that EPA reference data sharing guidelines from the National Institutes of Health (NIH) in its grants and/or cooperative agreements. The commenter explains that the NIH guidelines are applicable to the sharing of final research data for research purposes and they have already undergone an extensive deliberative process with input from the scientific community.
Response: In addition to this rule, the EPA is actively implementing its Open Access Plan and invites the commenter to learn more about how the EPA is improving access to the research it funds. This rule is focused on strengthening the transparency of the science the Agency considers in its significant regulatory actions and influential scientific information. The scope of this rule consequently expands beyond only the science which the EPA funds.
18.2.3 Use EPA Funding to Enact Change
Comment: Commenter (2180) finds that the proposed rule will block peer-reviewed, high-quality scientific studies from being used in environmental decision making and suggests that instead, EPA could improve the rigor of science that is used in rulemaking by increasing the funding from the Agency to conduct scientific research.
Commenter (2234) states that the federal government's most significant roles in promoting a body of reliable science in the service of public well-being and economic opportunity has been, and should continue to be, through the funding of born-open data collection programs (e.g. systematic environmental monitoring) and information technology infrastructure to make those data easily findable, available, interpretable, and reusable. The commenter suggests that policies that would truly strengthen transparency in regulatory science include (1) ensuring the continued public availability of government-held datasets such as the Toxics Release Inventory (TRI); (2) ensuring continued public investment in environmental monitoring programs such as those for radon-detection and water quality; and (3) providing funds for EPA scientists and external researchers supported by EPA grants to manage, preserve, and disseminate their data in trusted repositories.
Commenter (6141) suggests that EPA should show preference for the funding and development of studies for which underlying data and models are publicly available or otherwise transparent.
Commenter (6955) cautions that if the proposed rule goes forward and data sets are made available, EPA should have funding for new research available to dig into the questions that will inevitably be raised by critics about earlier research products.
Commenter (6955) also suggests that funding parallel studies on other populations is usually the best way of validating models.
Response: The EPA disagrees that this final rule will block peer-reviewed, high-quality scientific studies from being used in environmental decision making. This rule is focused on strengthening the transparency of the science the Agency considers in its significant regulatory actions and influential scientific information. The scope of this rule consequently expands beyond only the science which the EPA funds. The EPA disagrees with the commenter that this rule will block scientific studies from being used in environmental decision making. The rule makes clear no studies are categorically excluded from consideration, the EPA will continue to rely on the full body of the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Further, in addition to this rule, the EPA is actively implementing its Open Access Plan and invites the commenter to learn more about how the EPA is improving access to the research it funds.
18.3 Suggested Changes to the Rule/Approach
18.3.1 Make the Rule More General
Comment: Commenters (6151, 6356) propose that the scope of the rulemaking should be more general. Commenter (6151) cautions against using a single approach for the various EPA programs and statutes and suggests that EPA consider setting a foundation of transparency expectations that would apply to all programs and provide specific policies and/or approaches at the programmatic or statutory levels.
Commenter (6356) states that codifying the proposed rule at 40 C.F.R., Chapter 1, Subchapter B at Section 30 would limit the rule applicability to "Grants and other Federal Assistance." Therefore, the commenter suggests that this rule should--like other general administration issues such as "Cross-media Electronic Reporting," "Nondiscrimination," "NEPA [National Environmental Policy Act] Procedures," and "Small Business" --be codified under Subchapter A, potentially under 40 C.F.R. § 25, entitled "Public Participation." The commenter elaborates that since the proposed regulation is broad and general in nature and should include for instance, EPA's own research as well as other research that is not produced pursuant to an EPA grant, it should not be codified in 40 C.F.R Part 30, titled "Uniform Administrative Requirements for Grants and Agreements with Institutions of Higher Education, Hospitals and Other Non-Profit Organizations" which would limit the action's impact if it is adopted.
Response: The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. The EPA disagrees that placement of the final rule in 40 CFR Part 30 will limit the impact of the action. Further, the EPA plans to promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers.
18.3.2 Applicability of Science Covered by the Rule
Comments: Commenter (6155) encourages EPA to focus this rulemaking or any future proposed rulemaking on the specific issue of increasing transparency in risk assessments that use dose-response relationship studies to help establish appropriate thresholds or standards, noting that strategies for establishing complete and comprehensive data management policies for risk assessment include how to identify uncertainties, how to perform uncertainty analyses, how to identify and address variability (including exposure variability and variability in human susceptibility), how to deal with problems of aggregation (including exposure routes and types of non-threshold risk), and how to use and do peer review in various contexts and programs administered by EPA under federal law. The commenter provides examples of several regulations and guidance that already exist to assist in identifying scientifically based dose-response thresholds, including relevant provisions from the many regulations and guidance (listed below). According to the commenter, these reports, guidelines, and memoranda address the elements of risk assessment (e.g., identification, dose-response assessment, exposure assessment, and risk characterization) and suggest policies for improving data availability and quality, for validation of data bases and data collection priorities and other data needs.
 The regulatory analysis and report that the National Research Council (NRC) did under Clean Air Act (CAA) section 112(o) to review the risk assessment methods used by the EPA to assess hazardous air pollutants and to instruct revisions to EPA's risk-assessment guidelines. This NRC report was required to be made to both the Senate and the House and was also published by the NRC in its final form as a book entitled Science and Judgment in Risk Assessment (National Academy Press 1994  --  Copyright 1994 by the National Academy of Sciences).
 The Guidelines for Carcinogen Risk Assessment that were promulgated in response CAA section 112(o)(7). These final regulations were published in the Federal Register at 70 Fed. Reg. 17766 (April 7, 2005). The "Proposed Guidelines for Carcinogen Risk Assessment' at 61 Fed. Reg. 17960 (April 23, 1996) were published almost nine years earlier.
 The original NRC risk assessment guidance document: Risk Assessment in the Federal Government: Managing the Process (National Academy Press 1983).
 Understanding Risk: Informing Decisions in a Democratic Society (National Academy Press, 1996).
 "Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental Protection Agency," U.S. Environmental Protection Agency Office of Environmental Information, EPA/260R-02-008 (October 2002) https://www.epa.gov/sites/productionlfiles/2017-03/documents/epa-info-quality-guidelines.pdf.
 OMB "Final Information Quality Bulletin for Peer Review," (December 16, 2004) issued by Joshua B. Bolten, Director. ("The bulletin includes guidance to federal agencies on what information is subject to peer review, the selection of appropriate peer reviewers, opportunities for public participation, and related issues. The bulletin also defines a peer review planning process that will permit the public and scientific societies to contribute to agency dialogue about which scientific reports merit especially rigorous peer review.") Available at http://www.cio.noaa.gov/services_programs/pdfs/OMB_PeerReview Bulletin_m05-03.pdf.
Commenter (6375) states that the regulatory language that EPA proposes provides general requirements that apply to the Agency's use of science under the many statutes that it implements and contends that the scientific analyses required for these different programs and the sources of the scientific information underlying those analyses differ. Therefore, the commenter suggests that the Agency consider supplementing the general language that it has proposed with statutory specific regulations, concluding that a statutory-specific rulemaking would provide greater certainty going forward.
Conversely, commenters (6141, 6161, 6360) suggest the rule be expanded to apply to all science applications and all EPA rulemakings or regulatory proceedings. Commenter (6141) points out that significant efforts are now being made by the scientific community to make data available wherever possible and to develop studies based on publicly available data where appropriate. The commenter suggests that EPA's proposed rule could strengthen and expand upon this trend by further requiring the public availability of the underlying studies as a prerequisite for EPA relying on the findings of those studies in any future EPA rulemakings or other regulatory proceedings. Commenter (6161) suggests that the Agency should expand the proposed transparency rule to encompass all science (not just dose-response data and models) considered in development of regulations, guidance, or policies. 
Response: The EPA is promulgating this procedural rule under its housekeeping authority. The EPA plans to develop internal implementing guidance and promulgate either statute-specific transparency regulations or programmatic actions implementing this procedural rule, as appropriate, to clarify how the Agency will implement the provisions from this final rule for specific programs authorized under the statutes the EPA administers. This action is consistent with Executive Orders 13777 and 13783, and the focus on transparency in the OMB Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and Integrity of Information Disseminated by Federal Agencies and OMB Memorandum 13-13: Open Data Policy  -  Managing Information as an Asset. This rule is designed to build upon prior EPA actions in response to government-wide data access and sharing policies. The EPA supports these efforts and is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements.
18.3.3 Additions to the Proposed Rule
If the proposed rule goes forward, commenters (2119, 2479, 2746, 2792, 6112, 6141, 6161, 6360, 6873, 6955, 9227) propose additions to the proposed rule. Their comments and proposed additions are included in the following paragraphs.
18.3.3.1 Disclosure of the Funding Source of Research Would Improve Transparency
Comment: Commenters (2119, 2479, 2746, 2792) suggest that the rule require disclosure of the funding source for research. Commenter (2792) asserts that a scientific report by a company might easily skew the result to favor one of their products, whereas a similar report by an independent lab would, hopefully, present the objective results without any bias. 
Commenter (2119) states that the greatest improvement in transparency would come from requiring all research used in policy decisions to have all funding sources disclosed. The commenter states that this, more than anything else, will help policy makers to decide about the likely veracity of the data used. The commenter adds that one big problem with the proposed rule is that it does nothing to address the problem of selectively reporting only the data that supports a particular position. The commenter states that, although there is no foolproof way for policy makers to determine that (the truth or accuracy of the data presented), disclosing funding sources is probably the best method available.
Commenters (2479, 2746) propose that EPA incorporate a funding transparency requirement for all studies to allow for evaluation of potential bias or conflict of interest. Commenter (9227) notes that undisclosed financial bias is an area of concern for study transparency that EPA should address.
Response: The EPA is focusing on the underlying dose-response data for this rulemaking because of the influence these data have on particularly impactful decisions at the Agency. Risk assessments and regulations that target emissions and risk reduction of one or more pollutants, contaminants, or substances are integral to the Agency's mission and the underlying dose-response data that inform the quantitative value used to evaluate and mitigate potential risk are critical to understanding the assessment or regulatory action. The EPA does not find that requiring disclosure of the funding source of a study should be necessary for the EPA to grant that study full consideration because reanalysis of the underlying dose-response data would be sufficient to validate the results. The EPA would like to emphasize that this final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms.
18.3.3.2 Suggested Rule Changes Related to Usage of Published Datasets
Comment: Commenter (6955) offers several suggestions for additions to the rule related to usage of the published datasets. The commenter suggests that any user of public data in an EPA proceeding should be required to:
 List any a-priori hypotheses planned to be tested 
 Carry out a multiple comparison analysis for any regulatory filing that uses post-hoc analysis of accessible study data
 Report sources of funding and conflicts of interests
 Include a statistician on the team
 Include "post-hoc analysis" in the title of any submission that relies on post-hoc analysis, as is done in scientific literature for drug testing
 Certify that neither they nor other sponsor consultants did additional post-hoc analyses other than reported or listed in the paper, thereby facilitating the accuracy of correction of statistical tests for multiple comparisons
 Ideally, only queries of the data in the database would be allowed and stored by the agency to allow counting of the amount of data dredging that took place, facilitating correction for multiple comparisons.
Commenter (6955) defines a post-hoc analysis in this context as analysis carried out with full knowledge of the results of the original study whose data is being used; analysis that amounts to more than simply checking that the original results follow from the original method of analysis. The commenter also suggests that exemptions to the access rule should be automatically granted (i.e., no public access to the dataset) when there are multiple studies covering the same issue, because tests on multiple datasets provide the standard form of validation.
Commenter (6955) cautions that even with the precautions described above, problems can still be expected with post-hoc analysis. The commenter provides an example from the experience of the Radiation Effects Research Foundation (RERF) datasets:
      There have been two papers that came to opposite conclusions about the low-dose radiation region using different statistical methods: one found a hormesis dose response (although with no uncertainty analysis of the outcome parameters) and one found a supralinear response (with uncertainty analysis). These conflicting studies actually confirm what is well known, namely that dose response at low doses in the A-bomb data is uncertain. But, suppose that only one of the two non-RERF studies had appeared. It would have looked like the default dose response had been challenged. Taken together, they give a different impression, namely that the default lies in the middle of an error range. Stakeholders, of course, will find ways to prefer and promote the one whose result they like.
Response: The EPA disagrees that the rule should impose requirements on users of published datasets, as this is a rule of internal procedure that does not impose requirements on any party other than the EPA. This rule imposes no requirements on researchers or their institutions. In its own dose-response assessments, this rule requires that the EPA describe critical assumptions and methods used and shall characterize the variability and uncertainty of the assessment. The EPA shall evaluate the appropriateness of using default assumptions on a case-by-case basis. The EPA shall clearly explain the scientific basis for critical assumptions used in the dose-response assessment that the EPA relied on for the significant regulatory action or influential scientific information. Further, the EPA disagrees that it should automatically exempt some studies from this rule, but notes that the Administrator may consider whether a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis when determining whether to exempt a study from this final rule.
18.3.3.3 Modernize the Conceptual Framework for Making Regulatory Decisions
Comment: Commenters (6112, 6115) suggest that in addition to the proposed rule, the Agency should modernize its entire conceptual framework for making regulatory decisions, including how the Agency manages its risk assessment methods, assumptions, models, uncertainty, and non-risk factors at the risk management stage.
Commenter (9227) suggests that if EPA proceeds with the proposed rule, the Agency should also address other significant transparency-related concerns across the spectrum of peer-reviewed science instead of just focusing on the area of concern targeted by regulated industries. The commenter states that EPA appears to assume that the only way to enhance transparency in regulatory science is to ensure that the underlying data and modeling for individual studies are publicly available. The commenter notes that significant concerns have been raised about other non-public aspects of the modern scientific research and publication process that may undermine the accuracy of scientific results such as the rising concerns about the increasing numbers of predatory pay-to-publish journals, which provide little-to-no guarantee of scientific integrity of their published studies.
Response: This rule does require that the EPA: describe critical assumptions and methods used in its dose-response assessment and characterize the variability and uncertainty of the assessment; evaluate the appropriateness of using default assumptions on a case-by-case basis; and clearly explain the scientific basis for critical assumptions used in the dose-response assessment that the EPA relied on for the significant regulatory action or influential scientific information. The EPA notes that this rule marks an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements.
Modernizing the entire conceptual framework for making regulatory decisions is outside of the scope of this rulemaking. Rather, the EPA is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements.
18.3.4 Subtractions from the Proposed Rule
Comment: Commenter (6881) suggests that the proposed rule is not the appropriate mechanism to effect change in risk assessment modeling practice at the Agency and that instead, EPA should address questions related to uncertainty in dose response modeling in a different venue, such as the Risk Assessment Forum and National Academies guidance. The commenter elaborates that:
      The science associated with dose response modeling within risk assessment, ranging from better understanding of biological mechanisms to statistical modeling and risk estimation methods, continues to evolve. Because of the complexity of risk assessment and the fact that it operates at the science-policy interface, EPA regularly seeks guidance from the National Academies on where the field is and where it is going.[,][,][,] EPA also has the Risk Assessment Forum, where senior scientists grapple with difficult issues related to risk assessment to allow for consistent implementation across the Agency, as well as periodic reports and guidance documents that provide best practices for cancer and non-cancer risk assessment (e.g., https://www.epa.gov/risk/risk-assessment-guidelines). These all represent deliberative processes that carefully examine the state of the science and offer conclusions regarding best practices. In contrast, this paragraph within the proposed rule is highly prescriptive in a manner that would add very little that is constructive and would most serve to lengthen and delay the risk assessment process. If the Agency wishes to re-examine questions related to how uncertainty is best characterized in dose response modeling, there are ample mechanisms to do so beyond a short paragraph with no citations tucked into a proposed "transparency" rule.
Commenter (5173) suggests that Rulemaking should not cover scientific methods or approaches which are constantly advancing with scientific innovations and discoveries. The commenter states that to use the best available science, the Agency needs to be able to adapt and implement changing science, which this rule would prevent it from doing.
Response: The EPA has determined that rulemaking is the appropriate venue for this action. The EPA will continue to use established guidelines for identifying and integrating evidence. This rule establishes additional requirements for how the EPA should consider the availability of dose-response data that constitutes pivotal science. The EPA is focusing on the underlying dose-response data for this rulemaking because of the influence these data have on particularly impactful decisions at the Agency. The EPA does not find that this rule will categorically prevent the Agency from using the best available science in its significant regulatory actions or influential scientific information.
18.3.5 No Action Necessary
Comment: Commenter (0671) states that no other alternative or additional regulatory or other policy vehicles are appropriate for the stated purpose because the stated purpose is counter to the CAA and its amendments.
Commenters (2427, 4840) suggest that the proposed rule is not necessary because there are already various means for public access to studies that EPA uses, and in some cases their underlying data, without the release of confidential information, including the FOIA, which provides an avenue to request raw data, including a process ensuring that sensitive data is protected. Commenter (2427) elaborates that many health studies are based on highly private information, which cannot be publicly released without the risk of identifying the individuals that participated in the studies. But that does not mean those studies can't be verified. For example, the data underlying the Harvard "Six Cities" Studies has been extensively re-analyzed by independent institutions, confirming the essential findings. However, researchers found that public disclosure of the personal data underlying the studies  -  if it included enough data to be useful for research purposes  -  would violate the privacy of study participants.
Commenters (6373, 6937, 9227) state that EPA will better achieve the stated goals through internal guidance and controls and suggest EPA withdraw the proposed rule. Commenter (6373) states that EPA has existing strict guidance and controls that achieve consistency, predictability, and transparency in the science that has historically supported its regulatory actions and notes that the Rule imposes new requirements that would make EPA regulation inconsistent, unpredictable, and opaque. Commenter (6937) states that regulation is not needed because internal EPA policies, which can be more flexible, and can do the job more effectively. Commenter (9227) states that the proposed rule's provisions addressing dose-response models unnecessarily and inappropriately memorialize highly complex and technical scientific issues into regulation, and that these issues are more appropriately dealt with in guidance, a more flexible vehicle better equipped for adapting to new scientific understanding and in this way supporting use of best available science.
Response: In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. As this rule makes clear, if implementing this procedural rule would result in conflicts with existing environmental statutes, and their implementing regulations, this rule will yield to the EPA statutes and regulations. Further, this rule does not require that sensitive data containing PII, CBI, or proprietary data be made publicly available. The EPA disagrees that the goals of this rulemaking could be better achieved through internal guidance. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
18.3.6 Delay Rulemaking
Comment: Commenters (6126, 6873, 6944, 6955) suggest that the rulemaking be delayed. 
Commenter (6126) suggests that prior to pursuing further regulatory action, EPA could first issue and implement policies that establish the requisite data, policy, and analytical infrastructure such as developing policies and procedures to enable a statistical unit authorized by OMB under the legal framework of the Confidential Information Protection and Statistical Efficiency Act (CIPSEA). The commenter states that the use of CIPSEA authority at EPA would require the unit that collects confidential data to fulfill a pledge of confidentiality and comply with existing guidance on how to do so. According to the commenter, as of the date of the proposed rule, EPA has no such CIPSEA-eligible or CIPSEA-designated unit and establishing such a unit prior to further action on the regulation would help fulfill an expectation that any confidential, identifiable data collected by EPA would indeed remain confidential. The commenter further states that the designation of a CIPSEA unit could also build public trust in EPA's ability to maintain and protect the described data, if identifiable data will be maintained by the agency. 
Commenter (6944) suggests that the rulemaking process start over and involve all interested and concerned parties, including a gathering of pertinent career EPA staff, a selection of concerned business and industry representatives, scientific research and educational representatives, and knowledgeable environmental and regulatory organizations, all equally represented, to discuss actual problems that need to be corrected in the research and regulatory process. The commenter suggests that the group be given several months to work together on this and then present a new rule with clear descriptions regarding:
 How the scientific process will be protected.
 How studies will be made available.
 A clear  -  transparent and understandable  -  process established by which various affected entities may access the decision process in a manner that will satisfy privacy concerns at many levels.
 A clear understanding that the use of the term "pivotal regulatory science" is a smoke screen. If something is "pivotal", it is policy driven. The science is objective and fact driven. The Regulation is based upon Policy, determined by Congress and EPA staff and Administrator. Application of the Policy is supported by scientific study.
 How all parties will provide input to ensure that EPA understands needs of the private sector and reasonably incorporates them into policy and regulatory process with the agreement of environmental and scientific community in a transparent manner.
Commenter (6944) contends that the above suggested process is the best way to implement and work to "focus on transparency in OMB's Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and Integrity of Information Disseminated by Federal Agencies" (pg2, clm3, 2) in the performance of the very important mission for which the EPA was created in 1970.
Commenter (6955) suggests that EPA defer consideration of such a proposal while the academic community is working out issues with public access to underlying data.
Commenter (6873) suggests that an alternative, short-term approach could be for EPA to work diligently with investigators to try to get the data they need, which would allow for more time to consider any broader and more ambitious effort. The commenter acknowledges that working with investigators to collect the data could prove challenging and may involve providing funding to try to reconsent study participants from work done 20 years ago, for instance. The commenter states that this would provide more time to consider any broader and more ambitious open efforts.
Response: The EPA issued a supplemental notice of proposed rulemaking (SNPRM) in 2020 to clarify, modify and supplement certain provisions included in the 2018 proposed rulemaking in response to some of the public comments received on the 2018 proposed rulemaking. The EPA finds that the two-year interval between proposal and finalization of the rule has given the Agency time to improve the rule and incorporate recommendations based on extensive public comment. The EPA notes that it is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements.
The EPA would like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures and does not direct or require any outside entity or the EPA to establish data sharing mechanisms. Further, the rule does not require the EPA to collect, store, or publicly disseminate data and models underlying pivotal science.
18.4 Alternative Policies to Increase Transparency in EPA Rulemaking without Releasing Private Data to Public
18.4.1 Other Effective Ways to Vet Scientific Results that Do Not Require Public Access to Research Data
Comment: Commenter (6168) reminds EPA that robust transparency requires not only consistency in requirements for data but also requires recognizing the right to privacy and anonymity for research participants. The commenter elaborates: researchers, particularly in the fields of epidemiology and other environmental health sciences, have navigated the narrow path of transparent yet protected data for decades; much academic environmental health research is conducted with integrity. Yet it is these fields that are the vulnerable for attack from industry due to the delicate balance of protection and openness. 
Commenters (1008, 1994, 2429, 4831, 6128, 6182, 6906, 6955) suggest that there are other effective ways to vet scientific results that do not require public access to research data.
Commenter (1994) references a recent article published by John Ioannidis in PloS Medicine on why "[a]ll science should inform policy and regulation." The commenter states that Mr. Ioannidis notes that there are a variety of ways to judge the credibility of scientific findings, and that many worthy scientific studies lacking transparency in a strict sense are otherwise highly credible, due to the fact that they followed stringent research protocols and were replicated by many other subsequent independent studies.
Commenter (6128) suggests that even if the data is not made openly available, reviewers can be given confidential access to key data where appropriate. The commenter also notes that researchers are trained to assess the logic and articulation of a study's research design, its methodology for data collection and analysis, as well as the appropriate citation of previous results, which helps to ensure rigorous quality control throughout the research lifecycle even when data cannot be made available.
Commenter (6182) provides an excerpt from the United States Department of Energy's (DOE's) "Energy Literacy" guide to support the idea of transparency through evaluation of research components other than the dataset:
      Part of scientific inquiry is to evaluate the results of scientific investigations, experiments, observations, theoretical models, and the explanations proposed by other scientists. Evaluation includes reviewing the experimental procedures, examining the evidence, identifying faulty reasoning, pointing out statements that go beyond the evidence, and suggesting alternative explanations for the same observations. Although scientists may disagree about explanations of phenomena, interpretations of data, or the value of rival theories, they do agree that questioning, response to criticism, and open communication are integral to the process of science.
Commenter (4831) suggests that EPA could improve transparency by documenting its procedures and methods for collecting, evaluating, and analyzing data, by specifying assumptions, and by characterizing uncertainties, as described in numerous reports issued by the National Academies. Examples of reports include:
 Application of Systematic Review Methods in an Overall Strategy for Evaluating Low-Dose Toxicity from Endocrine Active Chemicals (NASEM, 2017a),
 Review of EPA's Integrated Risk Information System (IRIS) Process (NRC, 2014),
 Critical Aspects of EPA's IRIS Assessment of Inorganic Arsenic: Interim Report (NRC, 2013),
 Review of the Environmental Protection Agency's Draft IRIS Assessment of Formaldehyde (NRC, 2011),
 Finding What Works in Health Care: Standards for Systematic Reviews (IOM, 2011),
 Science and Decisions: Advancing Risk Assessment (NRC, 2009), and
 Models in Environmental Regulatory Decision Making (NRC, 2007). 
Commenter (2429) suggests that the Agency could encourage the registration of research protocols, which would help improve the quality of the science used in the regulatory process and allow the Agency to make strong evidence-based regulations that would protect public health and the environment.
Commenter (6906) notes that verifiable science does not always include underlying data sets due to confidentiality, convention at the time of publication (for older studies) or unavailability of data. The commenter states that the scientific community would use a variety of factors to evaluate conclusions in such studies when dose response and modeling data is not available. This would include evaluation of methods, evaluation of mechanisms and related logical inferences, converging evidence of conclusions from multiple sources and replicability of the study and its conclusions based on provided information and similar published studies.
Commenter (6955) suggests that instead of requiring access to the datasets, EPA should focus on any statistical weaknesses in the papers and studies that are considered in regulatory analysis. The commenter explains that unidentified or uncorrected post-hoc analysis can also appear in published studies, particularly when no evidence is available about hypotheses planned before data collection, and notes that adding more post-hoc analysis from stakeholders to the mix is likely to make the problem worse.
Response: This rule is not meant to be at the exclusion of other transparency initiatives, but rather build upon open data initiatives in the federal government and scientific community. This rule is meant to advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to national ambient air quality standards (NAAQS), risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
18.4.2 Weight Studies According to Data Transparency
Comment: Commenters (1006, 5185, 6141, 6179, 6875, 6955) suggest that EPA preferentially weight studies, data, and/or models with available underlying data when considering the study, data, and/or model for use in rulemaking instead of disqualifying studies without available underlying data.
Commenter (6179) suggests that EPA could qualify discussion of such a study with a statement to the effect that "due to sensitive underlying dose response data and models, the robustness of this study's results could not be subjected to outside replicability and verification." The commenter further states that while the study would remain in a rulemaking's record, the Administrator would be free to consider uncertainties in the study arising from the lack of external review when making regulatory decisions. The commenter notes that administrators have taken similar methodological considerations into account when weighting studies in past NAAQS reviews."
Commenter (5185) suggests that in certain situations -- particularly when relevant publicly available data is limited -- the Agency may want to consider a flexible and non-binary approach to transparency requirements, including assigning greater decision-making weight to publicly available data while still allowing for the consideration of non-available data. The commenter notes that such an approach may be particularly useful for retrospective studies in which it is impossible or impractical to make data available and proposes that this concept be concurrently explored and addressed through the agency's proposed rule to increase the consistency and transparency of consideration of costs and benefits in the rulemaking process.
Commenter (6955) suggests that in a regulatory proceeding, EPA should preferentially weight studies whose data has been made publicly available as well as studies that meet recent recommendations for improvement put forth in the scientific literature. In that way, the commenter contends that all the available information can be considered, while still promoting public access of research data.
Commenter (6141) suggests that EPA should give some increased weight or value to studies, data, and/or models for which the underlying data are available and evaluate the validity of other studies lacking such transparency based on the various qualitative factors. The commenter provides examples of such qualitative factors: the type of peer view, the charge questions provided to the reviewers, how relevant peer review comments were integrated into the planned action, and information about qualifications of the reviewers. The commenter concludes that this approach gives added meaning and weight to EPA's commitment for transparency by reducing or diminishing the value of those studies to the extent that they lack the criteria for scientific transparency.
Response: The EPA broadly agrees with the commenters. In this final rule, the EPA will not categorically exclude any studies from consideration. Rather, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. See 40 CFR 30.5(d) for the factors the EPA shall consider in determining the degree of consideration to afford pivotal science for which the dose-response data are not available for independent validation. The Agency will identify the pivotal science that was given lesser consideration and provide a short description of why lesser consideration was given.
18.4.3 Encourage Scientific Community-Led Efforts to Self-Improve Transparency
Comment: Commenters (2429, 6875, 6937) note that the scientific community is already engaged in efforts to improve study transparency and EPA should encourage and support these efforts instead of proceeding with the proposed rule. Commenters (2911, 5164) suggest that any changes to EPA's transparency process should be made with the full consultation with and support of the scientific community.
Commenter (2429) suggests that the Agency could promote the resources that are currently being implemented within the scientific community such as promotion of the STROBE statement, aimed at Strengthening the Reporting of Observational studies in Epidemiology by providing authors a checklist of requirements necessary for complete and adequate reporting of research.
Commenter (6937) suggests that EPA could achieve the actual stated goal of replication, improve transparency, maintain the ability to protect public health using the best‐available science, and do so without excessive litigation and at lower cost to EPA and to the public by fostering dialog and protocol development on transparency, openness, and replication by fostering scientific town hall meetings, research, special journal issues, and/or focused conferences involving the stakeholders and brokers ‐‐ scientific associations, industrial associations, journals, the National Academy, and the Health Effects Institute. 
Commenter (6875) states that EPA should work to encourage the continuation of journals requiring authors to submit their data to public repositories.
Response: The transparency provisions in this final rule are intended to build upon existing efforts within and outside the federal government and provide incremental progress toward the Agency's goal of greater transparency. The EPA acknowledges that the scientific community has embraced greater data transparency, as evidenced by data sharing and availability requirements for many high-impact journals and the emergence of organizations, such as the Center for Open Science, and international initiatives like Findable, Accessible, Interoperable, and Reusable (FAIR) data principles; Facilitate Open Science Training in European Research (FOSTER); and Guidelines for Transparency and Openness Promotion (TOP) in Journal Policies and Practices that incentivize greater transparency in research[,].
The EPA supports these efforts and is pursuing an incremental approach to maximizing transparency in the science that it relies upon to ensure that implementation is done in a thoughtful and deliberate way that focuses on the EPA's most impactful actions, minimizes unintended consequences, and informs future transparency requirements. As further described in Section II.B, the EPA is focusing on the underlying dose-response data for this rulemaking because of the influence these data have on particularly impactful decisions at the Agency. Risk assessments and regulations that target emissions and risk reduction of one or more pollutants, contaminants, or substances are integral to the Agency's mission and the underlying dose-response data that inform the quantitative value used to evaluate and mitigate potential risk are critical to understanding the assessment or regulatory action. In addition, the data underlying the dose-response assessment are more distinct than the broad range of data informing an entire risk assessment. Therefore, the EPA is concentrating its current efforts to increase transparency on a well-defined step in the quantitative assessment of risk supporting specific Agency actions. This final rule provides an important step in furthering the progress already being made toward maximizing transparency and will provide important insight for developing future statute-specific requirements.
18.4.4 Use Independent Third-Party to Evaluate Study Components, Results, Rules, Data Disclosure Issues 
Comment: Commenters (4840, 5118, 6141, 6168, 6355, 6937. 6955) propose using a qualified independent third-party to evaluate the study components and/or the results when the data cannot be automatically made publicly available instead of disqualifying the study or results from the rulemaking process. Commenters (6357, 8178) suggest having a third-party be involved with all proposed regulations (new and updated). Commenter (2336) suggests using a third-party to resolve data disclosure issues.
Commenter (2336) notes that historically, for dose response function estimates that imply a need for major changes in allowable levels of a particular pollutant, a qualified independent third party free of potential conflicts of interest would be recruited to reanalyze the dose response function at issue, on the condition that the third-party analysts maintain the confidentiality of the input data necessary to perform the reanalysis. The commenter states that this procedure is consistent with the Notice of Proposed Rulemaking's statement that some data may be fully accessible through "public data repositories," and other data may have "controlled access in federal research data centers," meaning, presumably, that the public may have to apply for access and sign "nondisclosure agreements" to access the data. [83 FR 18771].
Commenter (4840) notes that one option to address the importance of transparency and access to the data underlying important studies without injuring the scientific enterprise or discouraging scientists from pursuing pressing scientific questions is to require reanalysis by third parties. The commenter notes that this would not require new regulation and that certain journals, for example, require authors to agree to make their data available to editors and others upon request.
Commenter (6355) states that restricting what scientific evidence can be used to inform policy does not strengthen policy; instead it undermines confidence in the scientific underpinnings of regulations and opens the door to misinformed or weakened regulations. The commenter suggests that the transparency and scientific underpinnings of EPA's policy and regulatory decisions could instead be improved through external review by objective and independent scientists with current knowledge of the `best available science', and public dissemination of resulting comments and recommendations; scientists should be vetting science-based regulatory decisions, rather than the public vetting the science; while data accessibility is desirable, objectivity and neutrality are essential.
Commenter (5118) suggests that instead of the proposed rule, an independent council should review the analyses used in rulemaking and the report of this council could be released for public scrutiny. Specifically, the commenter proposes that:
      An Epidemiology Peer Review Council with the goal of creating a transparent document reflecting a thorough review be established by EPA. This Council will consist of six independent sub-committees with relevant experts as follows:
            Ethics: All aspects of assuring the personal safety and identities of the individuals in the cohorts are protected. Will state clearly why individual protected personal data is or is not needed to make a decision.
            Endpoint evaluation: Relevant experts knowledgeable about the endpoint will discuss factors like how many in a cohort are needed to make a meaningful difference. Identify what is known about how this endpoint can be altered by environment and any known chemicals.
            Exposure evaluation
            Statistical evaluation
            Analytical Chemistry
            Animal Toxicity and Structure Activity Correlations
      Each sub-committee will articulate why additional date are or are not needed. The Council will consist of qualified individuals from EPA, U.S. Food and Drug Administration (FDA) or other agencies and consultants as needed. The Council will consider the reports of the six sub-committees and make their recommendations especially with regard to additional data needed to support transparent regulatory decisions.
      The report of the Council will append each of the six subcommittee reports as well as any dissenting opinions.
      The Council owns the decisions and since all responsible individuals will be identified, the report is thus transparent. Science Advisory Panels may further review the Council report.
      In conclusion, controversies associated with epidemiological reports may not be eliminated but the Council should contribute to minimizing these controversies.
Commenter (6937) suggests that in cases where making data public is very difficult due to privacy or other concerns, the EPA employ third party trusted replication such as that done for the Harvard Six Cities Study (Six City Study) on air pollution by the Health Effects Institute.
Commenter (6168) suggests that scientific rigor, contextualization, transparency, and accessibility all can be improved through the incorporation of multiple, "third party" perspectives in study design and analyses. The commenter states that this means seeking ways to include -- rather than exclude -- cutting edge scientific research and expanding participatory knowledge making. The commenter asserts that incorporating participant-centered values and research design will improve both scientific rigor and public health outcomes. The commenter also states that inclusion of only scientist or industry perspectives has resulted in regulations that exclude public health concerns, such as the removal of the neurotoxic gas hydrogen sulfide from EPA's initial list of Hazardous Air Pollutants in 1991 despite evidence that low dose, chronic exposure is detrimental to human health. The commenter concludes that development, sharing, and stewardship of science and data with the public, including marginalized communities, will increase real transparency and accessibility while improving knowledge of the true costs and benefits of both chemicals and regulations. 
Commenter (6955) states that if the goal is to make sure that the data and methods of analysis lead to the results stated, which has been a lot of the scientific motivation for public access, the approach taken with the Six Cities studies could be followed. The commenter specifically suggests that one of the hypothetical EPA approved organizations would be given the data solely for auditing purposes and to repeat the analysis, and not to change it. In this way, the commenter states, the original peer review would not be bypassed. 
Commenter (6141) cautions that the peer-review process is subject to shortcomings such as adophenia (interpreting patterns in random or unrelated data), confirmation bias, and hindsight bias. The commenter suggests that EPA should consider post-publishing verification of studies and including a broader cross-section of those affected by the regulatory activity in its analysis of studies that it relies on. 
Commenter (8178) suggests having a shadow agency (at the regulated industries' expense) do an independent investigation of all proposed regulations, similar to the shadow cabinets in Britain. 
Commenter (6357) recommends the establishment of an independent scientific advisory committee composed of experts in the field, including those funded by EPA, to guide improvements in transparency and validation of research included in the regulatory process. The commenter suggests that the advisory committee be tasked with summarizing their findings and distributing them directly to the public so that updates to the EPA's rules can be made in a way that supports as much transparency as is practically and ethically possible, while ensuring the inclusion of relevant studies that inform the agency's actions and protect human health.
Commenter (2336) suggests that EPA follow the example of the Clean Air SAB's role for resolving data disclosure issues. The commenter explains:
      The EPA is a line agency generally subject to direct political oversight, but there are some aspects of its duties that need to be insulated from political considerations, at least to the extent that the scientific integrity of a study can be judged independently of the political ramifications of using it to formulate regulations. To fill that need, Congress provided for a 48-member Science Advisory Board in the Clean Air Act. Although the Board's recommendations are advisory, the EPA Administrator, as a practical matter, is obligated to provide a cogent and convincing rationale for rejecting the Board's determinations.
      This is the approach that should be followed in resolving data disclosure issues. The extent to which the use of input data that is not public detracts from the scientific validity of an estimated dose response function depends heavily on the circumstance of each study. The centrality of confidential data to the study's findings, and the various forms of limited disclosure that might allow the study to be independently analyzed without sacrificing privacy interests, are factual, technical matters. Such issues should, in the first instance, be decided by an expert, objective technical board that might include lawyers and as scientists. If a political appointee overrules the findings of such a board, he or she should be required to provide specific technical reasons for doing so. There should also be a right of appeal from a political decision to overrule to the appropriate Federal district court.
Response: The EPA notes that in this final rule, no studies will be categorically excluded from consideration in significant regulatory actions or influential scientific information. Further, this final rule does not require the EPA or any third party to reanalyze scientific studies; rather, it requires the Agency to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. However, in 40 CFR 30.7(4) the rule does allow the Administrator to consider whether a third-party has conducted independent validation of the study's underlying dose-response data through reanalysis when the Administrator is determining whether to exempt the study from the requirements of this final rule.
18.4.5 Increase Financial Incentive Transparency
Comment: Commenter (2429) suggests that the Agency could encourage the registration of research protocols and create a public repository where environmental scientists from industry and academia can declare sources of funding. The commenter notes that these tasks would help improve the quality of the science used in the regulatory process and allow the Agency to make strong evidence-based regulations that would protect public health and the environment.
Commenter (8178) suggests that given that the ostensible purpose of the proposed regulation is to increase the transparency in EPA decision making, a reasonable alternative would be to require each person expressing a position on a EPA action to disclose her or his salary and/or compensation for that statement and for any supporting study. The commenter explains that this could be similar to what a party may discover about an expert witness offered in federal court, which would allow the public to act as more informed participants in any debate.
Response: This rule exclusively pertains to the internal practices of the EPA and does not regulate the conduct or determine the rights or obligations of any entity outside the federal government. 
18.4.6 Other
Comment: Commenter (6868) suggests several discrete actions the Agency could take to increase transparency and public trust in how EPA develops policy other than the proposed rule:
    Making the schedule of the EPA Administrator/Acting Administrator available to the public,
    Granting media representatives full access to public meetings,
    Ensuring public access to factual information on the environmental and public health effects of climate change,
    Minimizing the role regulated industry representatives play on EPA advisory committees, and
    Meeting court ordered deadlines for promulgation of major regulatory actions and providing full cooperation with any pending federal investigations of agency staff misuse of office.
Response: These suggestions are outside the scope of this rule. The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions.
Comment: Commenter (6875) states that EPA released a document titled Plan to Increase Access to Results of EPA - Funded Scientific Research in 2016 in response to a 2013 memo released by the White House Office of Science and Technology Policy (OSTP). The commenter states that the OSTP memorandum entitled "Increasing Access to the Results of Federally Funded Scientific Research" directs Federal departments and agencies that spend more than $100 million per year on research and development (R&D) to develop and submit a plan to OSTP to increase public access to peer-reviewed, scientific research publications and research data resulting from agency-funded R&D. The commenter asks: What is the status of this plan and what does this rule cover that this document does not regarding data funded and produced by EPA?
Response: The EPA's Public Access Plan is specific to scientific research that the EPA produces and funds, and is distinct in scope from this rulemaking. The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. The commenter can learn more about the EPA's Public Access Plan at https://www.epa.gov/data/increasing-access-results-epa-funded-scientific-research.
Comment: Commenter (6098) states that the Proposal verbiage suggests that "public" refers to "general public" i.e. all citizens, giving the impression that the average citizen conduct validation of data and methods. The commenter asserts that validation of scientific studies require a number of criteria and should only be performed by those that can meet or exceed said criteria:
 Validation must be conducted in a manner free of bias, i.e. objective.
 Validation must replicate the conditions and methods of the study.
 Validators must possess education and experience in the field of study; i.e. a geologist would not perform an independent review of a purely biological study.
 Validation must meet the same rigorous standards of the study and be available for review and subsequent validation.
Validation of processes and data accuracy within a scientific system requires appropriate skills commensurate with the subject at hand. It would be more appropriate to submit the study to peer review and then publish the findings in the appropriate journal with a supplemental publication designed for public consumption and understanding to be openly available.
Response: This rule exclusively pertains to the internal practices of the EPA and does not regulate the conduct or determine the rights or obligations of any entity outside the federal government. It therefore does not direct any member of the public or EPA to validate data or methods. Instead, this rule establishes internal procedure for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information.
Comment: Commenter (6892) cites the article A Real, Not Faux, Transparency Proposal for Regulatory Science," by Professors Rena Steinzor and Wendy Wagner (August 1, 2018)[:]
      Of all the problems that plague EPA today, ensuring scientific transparency is not all that difficult. A real transparency proposal, as opposed to the Pruitt EPA's faux proposal, would closely track the standards developed by the scientific community and include the following five features. 
      1. "Rebuttable Presumption. A commitment to transparency must not compromise the agency's ability to use the very best information possible. ...
      2. "Broad Conception of Scientific Transparency. In contrast to the crabbed conception of transparency as applying only to "data" in the current, faux proposal, a real proposal embraces the conceptions of transparency developed by the premier scientific journals.
      3. "Prospective Application. In contrast to the faux proposal, real transparency requirements would not apply retrospectively to research already completed. ...
      4. "Coverage of All Research. Unpublished research submitted by industries in support of a license or permit is at least as susceptible to bias and error as prominent academic research published in the top journals. Yet the faux transparency proposal  --  which applies only to an ill-defined set of "pivotal" research  --  would likely only reach the latter and leave most industry research untouched. ...
      5. "Disclosure of Exclusions. When information is excluded from consideration in the agency's analysis, the public should be able to learn about those exclusions. ..." 
Response: The EPA disagrees with the commenter that the 2018 proposed rule promoted "faux" transparency, but also notes that through revisions in response to public comments, this final rule resolves the concerns noted. First, under this final rule the Agency will continue to consider all relevant scientific information in its significant regulatory actions and influential scientific information. Second, this rule is not in conflict with scientific journals, but rather reflects that the scientific community has also embraced greater data transparency, as evidenced by data sharing and availability requirements for many high-impact journals and the emergence of organizations, such as the Center for Open Science, and international initiatives like Findable, Accessible, Interoperable, and Reusable (FAIR) data principles; Facilitate Open Science Training in European Research (FOSTER); and Guidelines for Transparency and Openness Promotion (TOP) in Journal Policies and Practices that incentivize greater transparency in research[,]. Third, this final rule applies prospectively to significant regulatory actions and influential scientific information developed after the effective date of the final rule and has no retrospective effect on existing significant regulatory actions or influential scientific information. For future, significant regulatory actions and influential scientific information, the final rule applies equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. Fourth, the commenter is incorrect; this rule applies to all dose-response data, regardless of the source of funding or identity of the party who generated the data. Fifth, the Agency will identify the pivotal science that was given lesser consideration and provide a short description of why lesser consideration was given.
Comment: Commenter (6168) requests that EPA strengthen real transparency and accessibility by incorporating and contextualizing the full range of evidence, stating that transparency can only be achieved through privacy, accessibility, and scientific rigor, which in turn require contextualized and participatory knowledge making carried out in a timely manner. The commenter notes that the point of regulatory science is to provide the information necessary for making timely decisions, and regulators must be able to use a body of scientific evidence that is imperfect; they must be enabled to act even in the face of uncertainty. The commenter suggests that any rulemaking by EPA recognize that transparency does not mean certainty, which is a tool for regulatory delay; rather, transparency requires contextualizing uncertainty in order to move forward with environmental and human health protections.
Response: The EPA disagrees with the commenter that the requirements of this rule will result in any meaningful delay in promulgating regulations. While this final rule requires the Agency to evaluate the availability of dose-response data for pivotal science, the incremental burden to the Agency to carry out these requirements is expected to be small given (1) the extensive scientific review the EPA already conducts regularly and (2) that the requirement is limited to pivotal science (i.e., typically a small, though highly important, subset of the studies the EPA would review). Further, with this final rule, the EPA is maintaining language in 40 CFR 30.3 stating that the statutes that the EPA administers, or their implementing regulations, will control in the event of any conflicts with the requirements of this rule. The Agency will continue to comply with and abide by the requirements in those statutes and implementing regulations, including regulatory deadlines. The EPA does not represent transparency as a source of certainty in this final rule, but rather argues that transparency lends incremental improvement in the strength and clarity of EPA significant regulatory actions and influential scientific information.
Comment: Commenter (6937) proposes that EPA consider reward leaders and teams at EPA that do an exemplary job with transparency instead of proceeding with the proposed rule.
Response: The EPA is promulgating this rule in order to codify internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. Employee awards are outside of the scope of this rulemaking.
Comment: Commenter (8178) suggests that an alternative to the proposed rule would be to authorize the EPA inspector general to expeditiously review any claims of bias.
Response: The EPA is promulgating this rule in order to codify internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. By enabling greater transparency in regulatory decision-making to the public, this may reduce the instances of alleged bias in the EPA's regulatory decision-making.
Comment: Commenter (4541) suggests that the EPA assumes that association equals causation, administering massive doses of chemicals to laboratory animals predicts the human health impacts of much lower levels of exposure, natural causes of cancer and other diseases are nonexistent, current ambient levels of air pollution can cause disease or death without clinical evidence, and more. All of these assumptions violate the Bradford Hill Criteria and other requirements of the scientific method that the commenter supports (see section 10.3.2 of Chapter 10 for a list of criteria and a summary of the commenters' discussion on how EPA's analyses differ), rendering, in their opinion EPA's science an unreliable guide for researchers and policymakers.
Response: The comments are out of scope of the rule, and thus EPA is not responding to the comments.
18.5 Facilitate Data Publication/Accessibility 
18.5.1 Public Accessibility to Environmental Databases and Websites that are Easy to Navigate
Comment: Commenter (6168) states that industry proprietorship has been coddled by a government ensnared by business interests to such an extent that it has prevented government scientists from being able to efficiently analyze the health effects of particular compounds such as per- and polyfluoroalkyl substances (PFAS) and glyphosate (both receiving public attention currently), or to investigate water quality impacts from activities like hydraulic fracturing. The commenter asserts that people have the right to know about the status of and factors influencing their environment and its management; this includes uninterrupted, public access to environmental databases and websites that are easy to navigate, providing valid and contextualized information aimed towards laypersons. 
Response: The main benefits of this rule spring from greater transparency in significant regulatory actions and influential scientific information. By placing greater emphasis on the availability of dose-response data underlying pivotal science, the rule will allow for greater scientific scrutiny as EPA decision makers are developing significant regulatory actions and influential scientific information and increases the likelihood that any errors will be identified and corrected. Greater transparency is also inherently valuable as a principle of good government and provides benefits to the public at large, including reducing the risk of errors in EPA analyses and in the science such analyses rely upon. The ability for independent subject matter experts to validate pivotal science will facilitate more effective comment and engagement with the public during development of future significant regulatory actions and influential scientific information. While the EPA supports public access to environmental databases and websites, this final rule does not require the EPA or any other entity to disclose or host data.
18.5.2 Improve Public Accessibility
Comment: Commenters (2003, 2746, 4840, 6126, 6924) suggest that EPA should work to develop or improve public data repositories.
Commenter (2003) states that while there are many good reasons why some studies do not publish their full datasets online, such as protecting the identity of participants or shielding CBI from disclosure, many researchers do not publish their datasets online for the simple reason that there is not a convenient, widely-used scientific data repository. The commenter suggests that instead of proceeding with the proposed rule, EPA invest in developing a national data repository, looking to many existing data repositories that could serve as good models for a national data repository. The commenter acknowledges that a national data repository would still not solve the problem that not all data would end up being posted online; researchers from other countries, for example, may choose not to put their data in a U.S. repository. The commenter states that restricting EPA's access to foreign studies would not benefit public health and the environment, concluding that ultimately, if EPA wants to improve the transparency of scientific data, it should focus on efforts that will do so, without restricting its ability to use data that some consider not to be transparent as needed.
Commenters (2746, 6126, 6924) suggest that EPA should invest in research and development of systems and methods that would allow secure data sharing while still protecting the privacy of study participants.
Commenter (6126) states that government must do more to ensure that information about how it uses data is easy for the American public to access and understand. The commenter finds that EPA's proposed rule falls short of this call, and substantial improvements can be made to both current practices at EPA and the details of the proposed rule. The commenter offers to provide advice and consultation to improve EPA's use of science and evaluation at the agency moving forward.
Commenter (6126) elaborates that the challenge for EPA is that the infrastructure does not yet exist for EPA to collect, maintain, and analyze the data and models described by EPA as necessary for transparency in this regulation. The commenter states that EPA's experience implementing the e-Enterprise system over the last decade and of building the hazardous waste electronic manifest system have, if anything, demonstrated that ensuring the systems are in place can be time consuming; but perhaps more importantly, these systems have demonstrated that the investment of time can help ensure systems are useful, relevant, and secure.
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. However, the transparency provisions in this final rule are intended to build upon existing federal government efforts. The EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. For example, The EPA's 2016 Plan to Increase Access to Results of EPA-Funded Scientific Research noted that "transparency is a core EPA value" and that increased availability of research data would accelerate scientific breakthroughs that support the Agency's mission and policymaking efforts. The EPA's Open Government Plan 5.0 also details the EPA's progress in implementing the tenets of the numerous data transparency initiatives in the federal government. Together with these initiatives, this final rule will provide incremental progress toward the Agency's goal of greater transparency.
The Agency may continue to use foreign research studies regardless of whether the underlying data has been made available.
18.5.3 Make Use of Existing Data Repository (e.g., Rulemaking Docket)
Comment: Commenter (6360) suggests that for EPA applications of science, the data depository should be the public docket of a proposed rule or the accompanying administrative record associated with a proposed decision. The commenter further suggests that for sensitive information, EPA should adopt the data repository standards in common use by other federal agencies, journals, and EPA itself. Examples standards provided by the commenter include: 
 The Census Department has established stringent criteria for researchers seeking to review data that could allow individual responses to be identified. 
 Similar procedures are in place to protect data from new drug trials in journal repositories. 
 EPA has procedures for its contractors to review CBI  -  procedures that could be modified for non-government public commenters to use to review applied science that relies on sensitive data.
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. However, 40 CFR 30.4 does require the EPA to clearly identify all science that serves as the basis for informing a significant regulatory action, and make all such science that serves as the basis for informing a significant regulatory action publicly available to the extent permitted by law. The EPA clarifies that this requirement at 40 CFR 30.4 is equivalent to the Agency's current practice of including science in the public docket as part of a rulemaking.
18.5.4 Transparency of Published Data
Comments: Commenter (6168) suggests that metadata stored with published data should include the sample or data location, institution responsible for data collection and analysis, type of sampling conducted, time and frequency of sample or data collection, suite of parameters measured, basic assumptions, sources of error, and uncertainties. The commenter also suggests that the population and/or geography relevant to the study, why and how the study was executed, who funded the study, proposed studies that were rejected and the justifications, and potential values, biases, and uncertainties of the study should also be stored with the data. The commenter explains that proper contextualization of environmental data takes into account data provenance and the circumstances under which data has been collected and stored, which addresses the public's right to know and improves scientific rigor. 
Response: The final rule establishes that dose-response data is considered "publicly available in a manner sufficient for independent validation" when it includes the information necessary for the public to understand, assess, and reanalyze findings and may include, for example:
(1) Data (data would be made available subject to access and use restrictions);
(2) Associated protocols necessary to understand, assess, and extend conclusions;
(3) Computer codes and models involved in the creation and analysis of such information;
(4) Recorded factual materials; and
(5) Detailed descriptions of how to access and use such information.
The EPA is focusing on the underlying dose-response data for this rulemaking because of the influence these data have on particularly impactful decisions at the Agency. Risk assessments and regulations that target emissions and risk reduction of one or more pollutants, contaminants, or substances are integral to the Agency's mission and the underlying dose-response data that inform the quantitative value used to evaluate and mitigate potential risk are critical to understanding the assessment or regulatory action. In addition, the data underlying the dose-response assessment are more distinct than the broad range of data informing an entire risk assessment. Therefore, the EPA is concentrating its current efforts to increase transparency on a well-defined step in the quantitative assessment of risk supporting specific Agency actions. This final rule provides an important step in furthering the progress already being made toward maximizing transparency and will provide important insight for developing future statute-specific requirements.
18.6 Implement, Emulate or Work with Federal Programs
18.6.1 Environmental Protection Agency Policy and Guidance
Comments: Commenters (6125, 6126) suggest that instead of proceeding with the proposed rule, EPA update or improve enforcement of existing EPA guidelines.
Commenter (6125) suggests that if EPA has issues with its current risk assessment and benefit analysis procedures that currently follow NAS recommendations and its own existing guidelines, EPA should update the existing guidelines as needed, showing in far more detail than provided in the proposed rule the details of their reasoning, the conditions in which they should be employed, and the scientific bases for requiring consideration of alternative models in all risk assessments. The commenter continues that the draft guidelines should then undergo peer review by expert panels of the NAS and/or its own SAB and Science Advisory Panel. The commenter states that while guideline development is arduous and time consuming, it has several advantages over rulemaking, including 1) guidance is developed by experienced risk assessors; 2) it is a science-based process that involves consensus building across the EPA as well as with the broader science community; and, 3) critically, it maintains the separation between risk assessment and risk management so that EPA's decision makers benefit from the availability of science that is produced in an independent and objective fashion. The commenter provided background on EPA's current risk assessment and benefit analysis procedures:
      For assessing human health risk based on animal toxicology studies, EPA has developed a publicly-available, widely-used and accepted tool (the Benchmark Dose software (BMDS)) that can perform these calculations. BMDS currently contains thirty different models that are appropriate for the analysis of dichotomous (quantal) data, continuous data, nested developmental toxicology data, multiple tumor analysis, and concentration-time data. There also are many other open source and proprietary tools available to use as well. The agency has access to most of them. 
      Because air quality standards have been based in large measure on human clinical and community epidemiology studies, EPA adopted an "acceptable risk" interpretation of the requirement for "an adequate margin of safety" early in the initial reviews of the NAAQS.[,] The approach for conducting formal risk assessments for NAAQS criteria and standards reviews evolved over time, but has remained consistent with the principles and practices outlined above, namely that quantitative dose or concentration-response approach should flow from the available scientific data. As the NAS noted in 2009, the more recent human clinical and epidemiologic evidence available for fine particles and ozone risk analyses has involved fairly low-level exposures, and extrapolation below the level of observation to any great degree is less important than for compounds for which evidence is derived from animal bioassays or occupational (high dose) epidemiology.
      The most recent risk assessments for the ozone and fine particle NAAQS include examples of a non-linear threshold approach based on human clinical studies, and no-threshold concentration-response models based on short and long-term epidemiology studies of premature mortality. These risk assessments, which are peer reviewed by the Clean Air Scientific Advisory Committee, form the primary basis for benefits estimates for the NAAQS as well as for cost-benefit analyses of regulations that reduce NAAQS pollutants. The proposal would force both risk assessment and cost benefit analyses to include thresholds and/or other non-linear concentration-response functions, regardless of what the review of the scientific information concludes."
Commenter (6126) suggests that EPA could consider severing the connection between the availability of information to support replication and the use of scientific evidence in policymaking activities. As EPA notes in the proposed rule, "EPA has not consistently followed previous EPA policy" about scientific integrity. The commenter concludes that instead of seeking culture change through a regulatory mechanism that intends to address availability and use concurrently, EPA could instead seek to better, more consistently implement the existing policy the rule critiques.
Response: The requirements provide a workable framework for evaluating pivotal science in the context of the availability of its underlying dose-response data, while balancing important technical considerations in order to ensure the Agency maintains a strong scientific basis for its decision-making. This framework provides direction that is consistent with Agency guidelines. For example, when evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence (consistent with Agency guidelines on hazard identification and dose-response assessment) to support a relationship between exposure and effect, the EPA will identify a subset of those studies for use in characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect. This will be based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both will be identified as pivotal science, and trigger the requirements of this rule outlined in 40 CFR 30.5. The incremental progress made possible by this rule provides an important step towards prioritizing transparency in particularly impactful EPA rules and assessments and will inform future statute-specific rulemakings.
18.6.2 Collaborate with Other Agencies/Departments
Comment: Commenter (1008) suggests that the EPA should continue to work in concert with other federal research funding agencies to carefully develop policies and procedures for providing access to research data, noting that the EPA's efforts and challenges in furthering transparency to research results are not unique to the agency; `going it alone' will not serve the EPA or its core mission.
Response: The EPA disagrees with the commenter that this rule requires the Agency to "go it alone," and notes that the transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency. The EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. For example, in 2002 the OMB released its Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies, which includes discussion of the importance of the reproducibility of analyses underlying influential information. that "transparency is a core EPA value" and that increased availability of research data would accelerate scientific breakthroughs that support the Agency's mission and policymaking efforts. The EPA's Open Government Plan 5.0 also details the EPA's progress in implementing the tenets of the numerous data transparency initiatives in the federal government prior to 2018, including the OMB M-10-06, the Office of Science and Technology Policy Memorandum of February 22, 2013, and OMB M-13-13. In 2019, Congress passed the Foundations for Evidence-Based Policymaking Act of 2018 (or OPEN Government Data Act, Pub. L. 115-435) into law, which included requirements for federal agencies to prioritize making their data available to the public, and OMB has released additional guidance for implementing the Act.[.].
18.6.2.1 Office of Science and Technology Policy
Comment: Commenter (6924) suggests that EPA should focus on implementing existing initiatives and guidelines for improving data sharing and transparency at federal agencies, including ensuring that EPA is in full compliance with the provisions of a 2013 memo from the Office of Science and Technology Policy that discusses policy principles and the development of federal agency plans to increase public access to federally funded research. The objectives were developed in consultation with the National Science and Technology Council and with input from the public. To increase public access to science funded and used by the government, EPA should ensure it is in full compliance with the provisions of this memo.
Response: EPA is in compliance with the OSTP memo directing federal research agencies to develop public access plan. The EPA's 2016 Plan to Increase Access to Results of EPA-Funded Scientific Research noted that "transparency is a core EPA value" and that increased availability of research data would accelerate scientific breakthroughs that support the Agency's mission and policymaking efforts. While the EPA's Public Access Plan is distinct from this rulemaking, the EPA notes that the transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency.
18.6.2.2 Department of Energy (DOE)
Comment: Commenter (6182) suggests that EPA try to emulate the best practices of DOE's "Process Rule" while avoiding the lessons learned. The commenter also recommends that common rules and procedures (perhaps coordinated by OMB) be established lest counter-productive conflicts and confusion arise in areas with overlapping jurisdictional interest. The commenter concludes that, in other words, agencies should try to coordinate Process Rule best practices through OMB.
Response: The EPA notes that it does not have authority to direct OMB to coordinate Process Rule best practices and these comments are outside the scope of this rulemaking. 
The transparency provisions in this final rule are intended to build upon existing federal government efforts and provide incremental progress toward the Agency's goal of greater transparency. The EPA and the federal government have long encouraged open data initiatives, as the principle of transparency in regulatory decision-making and the other operations of government agencies is a fundamental behavior of good government that is inherently valuable to the public. For example, in 2002 the OMB released its Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies, which includes discussion of the importance of the reproducibility of analyses underlying influential information. The EPA's 2016 Plan to Increase Access to Results of EPA-Funded Scientific Research noted that "transparency is a core EPA value" and that increased availability of research data would accelerate scientific breakthroughs that support the Agency's mission and policymaking efforts. The EPA's Open Government Plan 5.0 also details the EPA's progress in implementing the tenets of the numerous data transparency initiatives in the federal government prior to 2018, including OMB M-10-06, the Office of Science and Technology Policy Memorandum of February 22, 2013, and OMB M-13-13. In 2019, Congress passed the Foundations for Evidence-Based Policymaking Act of 2018 (or OPEN Government Data Act, Pub. L. 115-435) into law, which included requirements for federal agencies to prioritize making their data available to the public, and OMB has released additional guidance for implementing the act.[.]
18.6.2.3 FDA
Comment: Commenter (5041) suggests that instead of requiring all scientific research used to make EPA policy, regulation, or guidance be based on publicly available datasets, EPA can easily adopt the long established and proven processes and procedures used by FDA. The commenter states that this ensures that privacy laws and concerns are met while having the most robust numbers of studies to ensure sound scientific policy/regulation/guidance making where the public may bear the cost or harm of said policy/regulation/guidance.
Response: The EPA disagrees with the commenter, as this final rule does not require all scientific research used to make EPA policy, regulation, or guidance be based on publicly available datasets. Instead, the purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information.
18.6.2.4 Department of Interior (DOI) Transparency and Reproducibility Standards 
Comment: Commenter (6161) recommends that EPA combine implementation of transparency and reproducibility standards with re-organized Information Quality and Scientific Integrity (SI) functions modeled on the U.S. DOI example within the Office of the Science Advisor, perhaps to be called the Office of Regulatory Science. The commenter states that this is because information quality complaints submitted to EPA overwhelmingly address science-based issues (including data transparency and reproducibility) and provides the following background on the DOI decision to consolidate the functions of the two programs in a single office.
Response: The organizational changes recommended by the commenter are outside of the scope of this rulemaking.

 Chapter 19: Comparison of Approach to Hazard, Exposure and Risk Assessment Guidance
The proposed rule preamble (83 FR 18771) solicited comment on all aspects of the proposed regulation (specifically, the notice of proposed rulemaking (NPRM)) and the bases provided in the preamble. This section provides comments received on hazard, exposure and risk assessment guidance. In this section these comments are organized as follows:
       Science and Judgement in Risk Assessment
       Request for Inclusion of Discussion on How EPA Proposes to Address Exposure Assessments and Risk Characterization Data and Models in the Future
       Request Guidance on How to Use Human Data in Risk Assessments Under the Proposed Rule
       Industry Toxicology Study and Epidemiological Study Review and Acceptance Differ
       Public's Interest Best Served When Science is Replicable, Transparent and Explainable with Other Toxicology Information
The specific comments provided in this Chapter reflect the comment letters that provide arguments supported by detailed rationale, examples, and references/documentation. The Docket ID numbers for these commenters are cited throughout this Chapter. In addition, some general comment letters may also include comments related to comments included and responded to in this Chapter. These comments are summarized in Appendix B. Note that points and issues raised in these general comment letters may also be raised by the more detailed comments summarized in this Chapter. The responses provided in this Chapter adequately address these more general comments.
19.1 Science and Judgement in Risk Assessment
Comment: Commenter (6155) asserts that the Executive Summary in Science and Judgment in Risk Assessment summarizes essential aspects of identifying and communicating uncertainty when doing dose-response assessments through the risk assessment process:
 EPA should conduct formal uncertainty analyses, which can show where additional research might resolve major uncertainties and where it might not.
 EPA should consider in its risk assessments the limits of scientific knowledge, the remaining uncertainties, and the desire to identify errors of either overestimation or underestimation.
 EPA should develop guidelines for quantifying and communicating uncertainty (e.g., for models and data sets) as it occurs into each step of the risk-assessment process.
 Despite the advantages of developing consistent risk assessments between agencies by using common assumptions..., EPA should indicate other methods, if any, that might be more accurate.
 When ranking risks, EPA should consider uncertainties in each estimate, rather than ranking solely on the basis of point estimate value. Risk managers should not be given only a single number or range of numbers. Rather, they should be given risk characterizations that are as robust (i.e., complete and accurate) as can be feasibly developed. 
 Communicating risk: [T]he process of assessing probabilities is difficult. Because substantial disagreement and misunderstanding concerning the reliability of single numbers or even a range of numbers can occur, the basis for the numbers should be set forth clearly and in detail.
 Risk assessment is a set of tools, not an end in itself. The limited resources available should be spent to generate information that helps risk managers to choose the best course of action among the available options.
The commenter adds that Science and Judgment in Risk Assessment also summarized the importance of recognizing uncertainty in the risk assessment process:
      The very heart of risk assessment is the responsibility to use whatever information is at hand or can be generated to produce a number, a range, a probability distribution -- whatever expresses the best present state of knowledge about the effects of some hazard in some specific setting. Simply to ignore the uncertainty in any process is almost sure to leave critical parts of the process incompletely examined, and hence to increase the probability of generating a risk estimate that is incorrect, incomplete, or misleading.
Response: The EPA agrees that identifying and communicating uncertainty is important for any scientific assessment, including dose-response assessments. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. The final rule requires the EPA to describe critical assumptions and methods used in its dose-response assessment and characterize the variability and uncertainty of the assessment. The final rule also requires that the EPA evaluate the appropriateness of using default assumptions on a case-by-case basis and to clearly explain the scientific basis for critical assumptions used in the dose-response assessment that the EPA relied on for the significant regulatory action or influential scientific information.
19.2 Request for Inclusion of Discussion on How EPA Proposes to Address Exposure Assessments and Risk Characterization Data and Models in the Future
Comment: Commenter (6362) requests that EPA include a discussion in the final rule regarding how it proposes to address exposure assessments and risk characterization data and models in the future extensions of related rules on Transparency in Regulatory Science. The commenter provides specific rule language suggestions in a redline version of the proposed rule regulatory text included in Appendix A of their comment letter for EPA to consider
Response: Based on the comments on the 2018 proposed rule and the 2020 SNPRM and as described more fully in the preamble to the final rule, the EPA determined that the Agency should pursue an incremental approach to maximizing transparency in the science that it relies upon by focusing the final rule requirements on dose-response data and, in particular, only those studies that are integral to characterizing dose-response relationships (e.g., identifying candidate PODs). This rule does not apply to exposure assessments and risk characterization data and models, but future rulemakings may be more expansive as the EPA continues to make incremental progress toward maximizing transparency. 
19.3 Request Guidance on How to Use Human Data in Risk Assessments Under the Proposed Rule
Comment: Commenter (6942) requests that EPA provide clarification and guidance for implementing the proposed regulation when human data are utilized for dose-response modeling and development of a chemical-specific cancer slope factor, reference dose, or reference concentration (e.g., risk assessments by the Integrated Risk Information System (IRIS)). 
Response: As detailed in the preamble to the final rule, the EPA is finalizing an approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. The EPA would like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. Therefore, any guidance developed by the EPA for implementing the rule would be intended for use by EPA staff. The EPA also intends for the final rule to be implemented in a transparent manner and will identify pivotal science in proposed significant regulatory actions or external review drafts of influential scientific information, which will allow the public, if they so choose, to independently validate the pivotal science and provide comment to the EPA.
19.4 Industry Toxicology Study and Epidemiological Study Review and Acceptance Differ
Comment: Commenter (6862) asserts that there is a great discrepancy in the review of the toxicology studies that industry is required to submit in order to register their chemical products. According to the commenter, these studies follow strict protocol guidelines and are conducted under stringent good laboratory practices (GLP), quality assurance (QA) and animal ethics reviews as well as rigid reporting standards. These studies in the Office of Pesticide Programs (OPP) have multiple layers of primary and secondary review most often by contractors at a very significant expenditure of taxpayer funds and program resources. The commenter notes that studies are essentially transcribed into Data Evaluation Records (DERs) that have to state what was done and what the results were since the reviewers cannot misrepresent what the study reported. According to the commenter, the reviewers, however, can indicate study deficiencies and identify responses not already indicated by the report. The commenter states that the reviewers responsible for the reviews are clearly identified and responsible for their decisions.
In marked contrast, the commenter (6862) states that epidemiological studies appearing in the open literature (or otherwise) follow a mixed bag of study designs, GLP, QA, ethics and reporting practices. According to the commenter, these studies are most often accepted at their face value without evidence of a critical review or identification of the responsible individuals.
The commenter (6862) contends that this practice is unfair to the public and that studies used for risk assessment should have the same level of critical review no matter what their source is.
Response: The EPA agrees that data transparency is vital for individuals who have not contributed to the study to be able to verify the quality and strength of published studies and that the opportunity to independently validate the pivotal science that the EPA relies upon is important in furthering scientific understanding and the Agency's mission. As discussed in the preamble to the final rule, peer review does not typically include reanalysis of the underlying data (i.e., the proper stage of data where the data that are ready to be analyzed to extract relevant information) and, thus, peer review is not considered a replacement for the data availability requirements of this rule. Further, the EPA determined that it cannot solely rely on data availability statements made in published research because initiatives toward greater data sharing and transparency amongst scientific journals and international organizations are still being implemented, are inconsistently enforced, and the true accessibility of data in a public repository is still limited. Therefore, the rule requirements for the EPA's independent evaluation of the availability of data are necessary and critical to prioritizing data transparency in the pivotal science underlying its significant regulatory actions and influential scientific information.
19.5 Public's Interest Best Served When Science is Replicable, Transparent and Explainable with Other Toxicology Information
Comment: Commenter (6105) notes that the publication of a series of studies on a single human group from exposure to a commonly used insecticide shows an unexpected effect at extremely low exposures (apparently an outlier amongst the pattern of available data). The commenter states that this finding has not been replicated in other human studies and is in contrast to extensive animal and other limited human studies that all point to changes in a blood enzyme as sentinel, at much higher exposures. The commenter provides that, in this case, EPA scientists asked the authors of studies from this single human group for the underlying data in order to confirm the results at the lower exposure. According to the commenter, the authors refused the release of data citing confidentiality concerns. As a result of this refusal, the commenter states that the EPA was not able to conduct its own analysis of all the study data. According to the commenter, since EPA had neither confirmatory studies, nor consistent information that made sense in light of the multitude of available studies of the chemical, nor the ability to conduct its own analysis, it chose not to use the incompletely published information from this single human group.
In one sense, the commenter contends that McCarthy and McCabe (2018) are spot on: the judgment should be left to risk scientists over which epidemiology and/or toxicology data to use for risk or safety assessment purposes. The commenter notes that from our perspective as risk scientists, EPA's decision not to use studies with incomplete information is correct. The commenter adds that the public's interest is best served when science is replicable, transparent, and explainable with other toxicology information (both experimental animal and human). The commenter states that when studies cannot be replicated or when such studies are not consistent or explainable with other information, using such studies then depends on having access to the underlying data for independent analysis. When the underlying data are not provided to EPA scientists, especially those adept in the receipt and protection of confidential information, the commenter contends that it is difficult to use these studies with confidence to make a credible risk judgment for public health, much less national rulemaking.
In short, the commenter states that the public is often worried about chemical exposure, as they should be, when the chemical exposure exceeds a safety level. The commenter states that the public's interest is best served by trusting EPA's experts dedicated to public health protection and withholding scientific data from EPA's independent analysis is not in the public's best interest.
Response: The EPA agrees that scientific openness benefits both scientific and government processes. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. Although this rulemaking is not intended to remedy a particular issue, it does provide important incremental progress toward the Agency's overarching goal of greater transparency. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for the EPA or independent subject matter experts to validate pivotal science, and as data are better understood through independent reanalysis the Agency can make stronger, data-driven decisions and the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
The EPA would like to emphasize that the final rule does not impose requirements on any entity outside of the EPA. This is a rule of internal procedures. The rule does not direct or require any outside entity or the EPA to establish data sharing mechanisms, nor does it require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science.
 Chapter 20: Chemical/Pollutant Specific Comments
As support for the proposal and need for transparency in regulatory science, several commenters provide chemical specific examples that they contend support the need for the proposal. These comments are organized as follows: 
 Chlorpyrifos
 Formaldehyde
 Lead
 Particulate Matter 
 Trichloroethylene
The specific comments provided in this Chapter reflect the comment letters that provide arguments supported by detailed rationale, examples, and references/documentation. The Docket ID numbers for these commenters are cited throughout this Chapter. In addition, some general comment letters may also include comments related to comments included and responded to in this Chapter. These comments are summarized in Appendix B. Note that points and issues raised in these general comment letters may also be raised by the more detailed comments summarized in this Chapter. The responses provided in this Chapter adequately address these more general comments.
20.1 Chlorpyrifos
Comment: Commenter (1974) supports the development of sound science standards in order to prevent what they believe to be "sham" science from affecting EPA rulings. As an example, the commenter cites EPA's proposal to ban chlorpyrifos. The commenter notes that due to a federal court order that EPA had not addressed the scientific bases for its action, the EPA's own science board found that the single study the Agency sought to rely on to justify a regulatory ban was "premature and possibly inappropriate." 
Commenter (6137) states that pesticide manufacturers, such as Dow Chemical Company, have long opposed the use of epidemiological studies that collect human health data that must be kept confidential. For example, the commenter provides that they vigorously challenged EPA's proposal to revoke tolerances for chlorpyrifos, one of the country's most widely used pesticides, which in effect would have prohibited the use of chlorpyrifos on food crops. The commenter notes that the EPA had found that it could not conclude that exposure to this pesticide in food and drinking water was safe based on a risk assessment that included a safety factor mandated under the Food Quality Protection Act to protect the health of infants and children whose developing bodies are uniquely vulnerable to toxic pesticides. See 80 Fed. Reg. 69,079, 69,090 (Nov. 6, 2015); see also NAS, Pesticides in the Diets of Infants and Children (1993). The commenter adds that the EPA retained the safety factor over industry objections because epidemiologic studies indicate that prenatal exposure to chlorpyrifos can harm the developing nervous system. So the commenter states that industry attacked those studies, claiming they needed to see the underlying medical information, even though the studies  -  conducted by highly reputable institutions including Columbia University, University of California-Berkeley, and Mt. Sinai School of Medicine  -  were all published peer-reviewed articles in scientific journals. While the Columbia scientists who authored the study have allowed EPA scientists to analyze the data in a secure setting on Columbia's campus, the commenter notes that they have refused to make the raw data publicly available in order to protect the privacy of the mothers and children who participated in the research. Not satisfied, the commenter states that the pesticide industry is pressing EPA to exclude the study so that it can continue to sell a pesticide known to cause severe harm to children and EPA is playing along.
Response: The final rule is not intended to remedy a particular instance in which the EPA allegedly relied on a poor-quality study or a study for which the conclusions could not be independently validated. Rather, the transparency provisions of the final rule are intended to build upon existing federal government efforts and provide incremental progress toward the EPA's goal of greater transparency. 
20.2 Formaldehyde
Comment: Commenters (6362, 6369) assert that the EPA's formaldehyde assessment is an example of the long-term problems where there is a lack of consistent, transparent application of modern scientific knowledge regarding chemical exposures and human health risk. Commenter (6362) asserts that, while knowledge regarding formaldehyde is much greater today, it does not appear that this new knowledge has been applied in the EPA's assessment of formaldehyde risk. The commenter notes that EPA continues to revise its assessment while not disclosing how emerging scientific evidence or modern risk assessment methods are being employed. 
Response: The final rule is not intended to remedy a particular instance in which the EPA allegedly relied on a poor-quality study or a study for which the conclusions could not be independently validated. Rather, the transparency provisions of the final rule are intended to build upon existing federal government efforts and provide incremental progress toward the EPA's goal of greater transparency.
20.3 Lead
Comment: Commenter (6365) states that the experience of the lead national ambient air quality standards (NAAQS) rulemaking demonstrates the salutary purpose that transparency would serve in the regulatory process. The commenter states that, in that rule, EPA relied heavily on the Lanphear study in its analysis of the health effects of lead exposure. The commenter states that the underlying data for the Lanphear study was not available to EPA or to stakeholders for replication and validation at the time of publication and it took stakeholders years, including litigation under the FOIA, to obtain that data. The commenter states that, due to the lack of availability, there was no complete reanalysis of that data for nearly a decade and EPA relied on the study in both the 2008 and 2016 lead NAAQS reviews. The commenter states that the lack of available data has been noted in the scientific literature as preventing alternative statistical analyses to be performed, which is particularly important given that key confounding factors such as parental education, intelligence, birthweight, smoking, and race were not controlled in several of the Lanphear cohorts and may contribute to an overestimation of health effects. The commenter provides that, when the data underlying Lanphear was reanalyzed, numerous errors that were not discovered in the peer-review process and that were not previously known to EPA were uncovered and analyzed. The commenter notes that, although EPA ultimately determined that these errors did not materially affect the NAAQS review documents that relied on the Lanphear study, the Agency considered the errors and their potential impact on EPA's regulatory efforts to be sufficiently substantial so as to include a memorandum in the subsequent NAAQS rulemaking docket acknowledging those errors and considering their impact on the regulatory process.
Response: The final rule is not intended to remedy a particular instance in which the EPA allegedly relied on a poor-quality study or a study for which the conclusions could not be independently validated. Rather, the transparency provisions of the final rule are intended to build upon existing federal government efforts and provide incremental progress toward the EPA's goal of greater transparency.
20.4 Particulate Matter
Comment: In support of the EPA's proposal for strengthening transparency in regulatory science, some commenters (6124, 6945) assert that certain studies supporting NAAQS found no significant relationship between PM2.5 and total mortality. Commenter (6945) notes that their reanalysis and the extensive null evidence on PM2.5 deaths support the importance of the EPA's "Strengthening Transparency in Regulatory Science" rule, which would make possible independent reanalysis of the "pivotal regulatory science" used as the primary basis for EPA regulations.
Commenter (6124) states that the commenter's null CPS II findings basically agree with the null findings in the April 2016 EHP Thurston article, which analyzed the National Institutes of Health (NIH)-AARP Diet and Health Cohort and found no significant relationship between PM2.5 and total mortality during 2000-2009. The commenter states that, since Thurston obtained these deidentified data from NIH, he should make his analytic data set available for additional analyses. The commenter states that Science Advisory Board (SAB) should request the publicly available Medicare data that was used by Schwartz for his recent NEJM and JAMA articles on PM2.5 deaths. 
Commenter (6124) states that, since repeated requests to Pope, ACS, HEI, and other CPS II investigators have been rejected, EPA formally should ask ACS to cooperate with transparent analyses of the CPS II data, such as, the analyses that the commenter has conducted and requested. The commenter states that, if ACS fully cooperates with EPA, then it might be useful to modify the EPA Transparency Rule to include a full cooperation option that does not require releasing actual data. The commenter states that, if ACS fails to cooperate with EPA and other legitimate investigators like myself, then their CPS II research results should not be used for EPA regulations. 
Commenter (3134) opposes the PM2.5 regulations and asserts they are based upon secret studies. The commenter provides a brief analysis and contends that the regulations prevent a public exposure of PM2.5 equivalent to five cigarettes spread over 80 years. The commenter asserts there have been zero serious medical investigations on PM2.5 air pollution in the United States because such a study would require at a minimum:
 A biologically plausible toxicity mechanism -- how does the toxin cause death or disease, and reliable data on exposure.
 An appropriate endpoint, which could be death if it is determined to be premature (impossible to do with observational population studies).
 An adequate size association to get past the confounders in an observational population study.
Response: The final rule is not intended to remedy a particular instance in which the EPA allegedly relied on a poor-quality study or a study for which the conclusions could not be independently validated. Rather, the transparency provisions of the final rule are intended to build upon existing federal government efforts and provide incremental progress toward the EPA's goal of greater transparency.
Comment: Other commenters emphasize the strength of the data used to support the development of particulate matter NAAQS. Commenters (5178, 6153, 6874, 8272) state that EPA's 2011 policy assessment for review the particulate NAAQS noted that in addition to epidemiologic studies, toxicological and controlled human exposure studies identify potential biological pathways for cardiovascular and respiratory effects associated with particulate exposures was sufficient and that the proposed rule is unnecessary. According to commenters, there is also evidence accumulating on the effects of PM2.5 exposure on low birthweight and infant mortality.
Commenters (5178, 6125, 6153, 6446, 6874, 6939, 8272) add that some of the most recent research adds to knowledge on susceptible populations to air pollution as well, including the analysis of 2000-2012 data on 61 million Medicare beneficiaries and zip-code level information on PM2.5 and ozone pollution. The commenters state that they found increased mortality risks  --  including higher risks for black beneficiaries and those eligible for Medicaid, who are disproportionately women with low incomes  --  at pollutant levels below the current limits. Commenter (6446) notes that, because these studies rely on confidential information about each person in the cohort, the proposed rule would either require publishing data so thoroughly redacted as to be effectively useless for validation, or preclude EPA from considering these studies and others like them in the NAAQS standard-setting process.
Commenters (5178, 6153, 6874, 8272) state that Mingyu Zhang and colleagues analyzed local air pollution data as well as data from the Boston Birth Cohort, which began recruiting pregnant women in 1998 and followed their children for several years after birth. The commenters provide that the study found that the children of women who were exposed to the highest PM2.5 levels during their third trimesters had a greater risk of elevated blood pressure at ages three to nine. The commenters state that both studies involve data that can be obtained by investigators who apply to use it and are approved by the relevant institutional review board (IRB) or Privacy Board, but their data sets are not publicly available. 
Response: The EPA clarified in the final rule how the Agency will evaluate scientific studies when promulgating significant regulatory actions and finalizing influential scientific information. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. 
As described in the preamble to the final rule, the transparency provisions of the final rule apply to dose-response data underlying pivotal science. Under this rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of personally identifiable information (PII), confidential business information (CBI), or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator under 40 CFR 30.7.
The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information. 
20.5 Trichloroethylene (TCE)
Comment: Commenter (1944) supports EPA's proposed rule and points to the use of the 2003 Johnson study in EPA's derivation of toxicological values for TCE as an example of why the rule is needed. The commenter asserts that, as noted in the proposed rule, both transparency and independent validation of key findings of a study (reproducibility) are necessary in EPA's scientific assessments to ensure "that the quality of published information meets the standards of the scientific and technical community." The commenter notes that, in 2011 EPA derived a reference concentration (RfC) and a reference dose RfD for TCE, based, in part, on a 2003 Johnson study. The commenter states that this assessment was subsequently adopted in the Toxic Substances Control Act (TSCA) Chemicals Work Plan Assessment for TCE. The commenter asserts that the Johnson et al. (2003) study should not be used to develop toxicological values that serve as the basis for regulation because the data records for Johnson et al. (2003) are inadequate or non-existent and Johnson et al. (2003) is not reproducible. The commenter provides detailed information regarding the TCE assessment, which is summarized below:
 Halogenated Solvents Industry Alliance (HSIA), the commenter, reports that attempts to see the raw data which formed the basis of the Johnson et al. (2003) study report have been unsuccessful. A direct appeal to Dr. Johnson failed to make the data available for public scrutiny. And a FOIA request pursuant to the Shelby Amendment was denied by the NIH.
 Information contained in Makris, et al. (2016) constitutes a transparency as well as a data quality concern sufficient to preclude Johnson et al. (2003) from being used as the basis for regulation. The transparency problem with Johnson et al. (2003) was pointed out by the peer review of the TSCA Chemicals Work Plan assessment for TCE. 
 The data reported in Johnson et al. (2003), even as amended in two subsequent errata, do not allow for data analysis generally accepted as essential to interpreting outcomes of developmental toxicity study findings. The lack of data availability and clarity sufficient to construct key analyses associated with a key study should disqualify the use of that study in important decisions such as RfC/RfD derivations used for regulatory purposes.
 At least two Good Laboratory Practice (GLP)-compliant studies conducted under both EPA and Organization for Economic Coordination and Development (OECD) guidelines have been unable to reproduce the effect seen by Johnson et al. (2003). In one of these studies, Carney et al. (2006), all stages of the testing, from development of the protocol to the final report, underwent extensive peer review by scientists from three separate governmental agencies (ASTDR, EPA, and the National Toxicology Program), as well as external experts. In addition, the protocol and study report (which includes the raw data) are available to the public. 
Response: The final rule is not intended to remedy a particular instance in which the EPA allegedly relied on a poor-quality study or a study for which the conclusions could not be independently validated. Rather, the transparency provisions of the final rule are intended to build upon existing federal government efforts and provide incremental progress toward the EPA's goal of greater transparency.
Comment: Commenters (4878, 9227) express concern that, under the proposal, EPA would exclude important studies regarding TCE exposure. 
Commenter (4878) states that, to exclude evidence that TCE exposure is a risk factor for Parkinson's disease is illogical and does not serve the best health interests of the American public. According to the commenter, the original recommendation was based on hundreds of studies, many of which would not be considered under the proposed rule due to data confidentiality.
Commenter (9227) states that EPA has proposed two regulations regarding TCE and the scientific basis for these proposed regulations is provided in the Agency's 2014 risk assessment: TSCA Work Plan Chemical Risk Assessment, Trichloroethylene: Degreasing, Spot Cleaning and Arts & Crafts Uses which drew heavily from the 2011 EPA IRIS Toxicological Review of TCE. The commenter states that development of TCE's hazard and dose-response assessments considered the principles set forth by the various risk assessment guidelines issued by the National Research Council and the U.S. EPA. According to the commenter, EPA found the TCE IRIS assessment to be scientifically rigorous. The commenter provides that EPA made this determination without the data underlying the key, peer-reviewed studies used in the assessment being publicly available.
Response: The EPA clarified in the final rule how the Agency will evaluate scientific studies when promulgating significant regulatory actions and finalizing influential scientific information. As described in the preamble to the final rule, the EPA is not categorically excluding any studies from consideration when promulgating significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. 
As described in the preamble to the final rule, the transparency provisions of the final rule apply to dose-response data underlying pivotal science. Under this rule, the EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of PII, CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. In cases where dose-response data underlying pivotal science are not available in a manner sufficient for independent validation, the EPA may still use the pivotal science after either giving it lesser consideration or receiving an exemption from the requirements of this rule from the Administrator under 40 CFR 30.7.
The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.

 Chapter 21: Miscellaneous and Out of Scope Comments
In addition to the topic-specific comments included in Chapters 1 through 21 of this document, several miscellaneous and out of scope comments were received. Comments provided in this Chapter include several of those comments. These comments are organized as follows: 
Section 21.1 Miscellaneous and Out of Scope Comments (NPRM)
       Miscellaneous Comments
       Out of Scope Comments
Section 21.2 Miscellaneous and Out of Scope Comments (SNPRM)
       Miscellaneous Comments
       Out of Scope Comments
The specific comments provided in this Chapter reflect the comment letters that provide arguments supported by detailed rationale, examples, and references/documentation. The Docket ID numbers for these commenters are cited throughout this Chapter. In addition, some general comment letters may also include comments related to comments included and responded to in this Chapter. These comments are summarized in Appendix B. Note that points and issues raised in these general comment letters may also be raised by the more detailed comments summarized in this Chapter. The responses provided in this Chapter adequately address these more general comments.
21.1 NPRM Miscellaneous Comments and Out of Scope Comments
Comments received that are ancillary to the Proposal, such as requests for comment period extensions/public hearings and comments that provide support for other commenters' comments; and comments received that are considered out of scope, are provided in this Chapter.
21.1.1 Miscellaneous Comments
21.1.1.1 Requests for Comment Period Extension
Comment: Commenters express concern that the initial 30-day comment period was too short. Some commenters simply request additional time to comment beyond the 30-day period. Other commenters request comment period extensions to a total comment period of 60, 90, 120, or 180 days. Commenters are generally concerned that EPA is soliciting comments on many complex scientific and technical issues that require careful and in-depth analysis, but the 30-day comment period does not provide sufficient time for the broad range of stakeholders to prepare thoughtful, detailed responses. Additional reasons commenters seek an extension include the following concerns:
 EPA has provided no analysis of the potential impacts of its proposal. 
 The mechanics of the proposed rule are not described, including cooperative agreements, technology platforms, and methods and technologies to protect confidential data  -  including identifiable and sensitive data such as individual health data.
 Implementation issues such as effective date, exemptions, retroactive applicability, impact on previous rulemaking records and studies are not addressed by the proposal.
Commenters (0557, 0564, 0598, 0570, 0573-0578, 0580-0583, 0585-0592, 0599, 0600, 0601, 0603, 0606, 0607, 0609-0613, 0616-0619, 0621, 0623-0626, 0628-0644, 0647, 0649-0654, 0657, 0658, 0661-0668, 0670, 0673-0678, 0681-0683, 0686, 0689, 0691, 0695-0702, 0704, 0706-0710, 0712, 0714-0716, 0778, 0780, 0781, 0786, 0788-0790, 0792-0794, 0796, 0798, 0800-0802, 0804, 0807, 0808-0811, 0993, 0995-1000, 1003, 1005, 1331, 1334, 1338, 1343, 1345, 1347, 1369, 1638, 1639-1642, 1779, 1797, 1942, 1948, 2006, 2048, 2067, 2485, 2561, 2737, 2739, 2983, 3053, 3074, 3079, 3081, 3100, 3115, 3229, 3273, 3276, 3279, 3282, 3283, 3292, 3300, 3302, 3323, 3341, 3346, 3350, 3353, 3355, 3356, 3358, 3364-3374, 3376-3381, 3564-3567, 3569, 3571, 3573, 3575, 3576, 3578, 3582-3586, 3589, 3590, 3592, 3594, 3599, 3600, 3608-3610, 3612, 3661-3668, 3675, 3679, 3682, 3685-3690, 3692-3697, 3834, 3972, 3974, 3976, 3977, 3979, 4120, 4124, 4394-4397, 4399, 4473, 4488, 4539, 4543-4545, 4770, 4867, 4868, 4875, 4980, 5129) request additional time to comment beyond the 30-day period.
Commenters (0042, 0058, 0546, 1340, 1380) request an additional 30-day extension. Commenter (0546) states that Executive Order 13563 recommends a comment period of at least 60 days to ensure adequate public participation in the regulatory process.
Commenters (0014, 0015, 0016, 0018, 0020, 0021, 0022, 0023, 0024, 0025, 0026, 0027, 0028, 0029, 0030, 0031, 0032, 0033, 0034, 0035, 0036, 0037, 0038, 0046, 0047, 0048, 0049, 0051, 0052, 0059, 0060, 0065, 0066, 0071, 0072, 0184, 0185, 0496, 0545, 0553, 0554, 0559, 0566, 0568, 0669, 1008, 1009, 1011, 1339, 1359, 1876, 2131, 3128, 3642, 3643, 4400) request an additional 60-day extension. Commenters (0026, 0051, 0545) assert that, while Executive Order 12866 indicates that a 60-day comment period is the minimum necessary to afford the public a meaningful opportunity to comment, in this proposal, where foundational and complex issues are at stake, a significantly longer comment period is appropriate.
Commenters (0565, 1948, 2014, 5009) request an additional 90-day extension. 
Commenter (5009) is concerned that the proposal considers fundamental changes in policy that could affect many of EPA's rulemaking activities and about the vagueness and potentially detrimental consequences of the proposal for those activities. The commenter states that, given the extremely broad scope and impact of this proposal, the 30 days allowed for public comment in the advance notice is insufficient to give the affected public adequate opportunity to participate in the rulemaking and comment on the proposal as required by the Administrative Procedure Act. 5 U.S.C. § 553(c). The commenter states that, under section (2)(b) of Executive Order 13563, a standard comment period should be at least 60 days, but this tremendously consequential proposal calls for an even more deliberate pace given the profound potential impacts on the regulatory processes for many of the statutes EPA implements and enforces. The commenter states that a full four-month comment period would be consistent with past practice for matters of similar importance and complexity, and is necessary to provide the public and other stakeholders a meaningful opportunity to evaluate the proposal and its implications for the agency's ability to meet its obligation to protect public health and the environment under federal environmental laws. 
Commenters (0057, 0063) request an additional 150-day extension. 
Commenter (0063) suggests that, in developing this rule, EPA should use a process more like the one employed in 2006, when the Office of Management and Budget (OMB) under President George W. Bush asked the National Academy of Sciences (NAS) for peer review of its proposed bulletin on risk assessment and allowed six months for submission of public comments. The commenter states that a rule with such substantial implications for public health merits considerable time for input from the public and the scientific community.
Response: The EPA extended the public comment period for the 2018 proposed rule until August 16, 2018. This constituted a 108-day comment period.
21.1.1.2 Requests for Public Hearing(s)
Comment: Commenters express concern that the proposal does not include a public hearing. Some commenters simply request the EPA to set at least one public hearing. Other commenters request multiple public hearings. Commenters are generally concerned that public input on this proposed rule, which could have significant consequences on the ability of EPA to adequately protect the public from the impacts of air pollution, unsafe chemicals, and other public health and environmental hazards, is of critical importance. Some commenters argue that the Clean Air Act (CAA) requires a public hearing for applicable rulemakings, and CAA rulemakings would be directly impacted by this proposal. Commenter (0046) states that among the CAA regulatory decisions subject to §307(d) that would be affected in this manner are the national ambient air quality standards (NAAQS) (§307 (d)(1)(A)), residual risk determinations for hazardous air pollutants (§307 (d) (1)(C)), standards for mobile source air toxics (§307 (d)(1)(K)), and residual risk standards for municipal solid waste combustors (§307 (d)(1)(D)). 
Commenters (0020, 0023, 0025, 0026, 0031, 0033, 0034, 0037, 0038, 0046, 0051, 0059, 0557, 0598, 0568, 0570, 0574-0578, 0581-0583, 0585-0592, 0599, 0600, 0601, 0603, 0606, 0607, 0609-0613, 0616-0619, 0621, 0629-0626, 0628-0645, 0647, 0649-0654, 0656, 0657, 0658, 0661-0668, 0673-0678, 0681-0683, 0686, 0689, 0691, 0695-0702, 0704, 0706-0710, 0712, 0714-0716, 0778, 0780, 0781, 0782, 0786, 0788-0790, 0792-0794, 0796, 0798, 0800-0802, 0804, 0807, 0809-0811, 0993, 0995-1000, 1003, 1005, 1011, 2902, 2983, 3053, 3074, 3081, 3100, 3642) request a public hearing. 
Commenters (0016, 0024, 0028, 0032, 0047, 0052, 0071, 0072, 2014, 3644) request multiple public hearings. Commenters (0014, 0022, 0030, 0035, 0048, 0060, 0554, 0566, 1009, 3128) request at least three public hearings. Commenters (0048, 0566) state that, because many scientists do not have experience with the rulemaking process and may not be prepared to submit written comments, EPA should supplement such comments with public hearings at locations across the country. 
Commenters (3644, 6123, 8281-PH82) express concern about the inadequate opportunity for public engagement on the public hearing due to EPA's refusal to accommodate remote participation. 
Commenter (3644) urges EPA to reverse its position and ensure that the public hearing provides an opportunity for testimony and other meaningful participation by all interested members of the public, including those unable to attend the hearing in person. The commenter states that EPA's refusal to allow for remote participation in the public hearing is inconsistent with the agency's recent practice. The commenter states that EPA's commitment to providing a verbatim transcript of the public hearing is an inadequate substitution for remote participation, for several reasons:
 Many people are more comfortable providing oral testimony than they are submitting written comments -- for example, because they are fluent in spoken but not written English. 
 Although EPA has promised to provide a transcript, it has not committed to do so by any particular date. Unless EPA makes the transcript available several weeks before the proposed rule's comment deadline, those who submit written comments will lack the opportunity to respond to opposing viewpoints expressed at the hearing. 
 Certain potential commenters who would otherwise fully participate in the public hearing, including some people with learning disabilities, may struggle to read, understand, or respond to a verbatim transcript.
Response: The EPA hosted a public hearing on the 2018 proposed rule on July 17, 2018.
21.1.1.3 Support for Other Commenters' Comments
Comment: The following commenters (5179, 6112, 6115, 6369, 6907, 6908, 6911) express support for other comments:
Commenter (5179) supports the comments submitted to date on this proposed rule by the Association of American Medical Colleges, the Association of American Universities, the Association of Public and Land-Grant Universities (plus, the Council on Government Relations, by joint letter dated July 11, 2018; attached to their comment letter), and, separately, Public Responsibility in Medicine and Research (PRIM&R) (by letter dated August 1, 2018; attached to their comment letter).
Commenters (6112, 6115) support the National Stone, Sand & Gravel Association's (NSSGA) comments on the Proposal.
Commenter (6369) supports the separate comments submitted by the American Chemistry Council on the Proposal.
Commenter (6907) supports the comments provided to this docket by the American Association of Pesticide Safety Educators (AAPSE), the National Academies of Sciences, Engineering and Medicine, Physicians for Social Responsibility, the Association of State Wetland Managers, the Vice Chancellors of Research at the University of California, the Entomological Society of America (ESA), and the joint comment made by the Association of American Medical Colleges (AAMC), Association of American Universities (AAU), Association of Public and Land-grant Universities (APLU), and Council on Governmental Relations (COGR).
Commenter (6908) incorporates by reference the comments submitted by CropLife America under Docket Number EPA-HQ-OA-2018-0259.
Commenter (6911) endorses the comments submitted by the NAAQS Implementation Coalition and the American Chemistry Council's Formaldehyde Panel.
Response: The EPA notes the commenters' support, and has addressed those comments elsewhere in this response document.
21.1.2 Out of Scope Comments
21.1.2.1 Use of the LNT Models and Radiation Studies
21.1.2.1.1 Oppose/Support Use of the LNT Model for Radiation
Oppose Use of LNT Model for Radiation
Comment: Commenters (0039, 0050, 0062, 4357, 6148) are opposed to continued use of the linear no-threshold (LNT) model for radiation. Commenters (0039, 0050, 4537, 6148) state that the LNT is based on a flawed scientific study or non-scientific assumptions. Commenters (0050, 0062) state that the LNT model is inappropriate for setting radiation protection rules and guidelines because low doses of radiation have been shown to be beneficial yet are not considered by the LNT model. 
Commenters (0039, 4537) state that the prominence of the LNT model stems from the June 29, 1956, Science paper "Genetic Effects of Atomic Radiation" authored by the NAS Committee on the Biological Effects of Atomic Radiation. The commenters continue that this paper is now widely questioned and has been seriously critiqued in many peer-reviewed publications, including two detailed 2015 papers, as summarized in a December 2, 2015, Wall Street Journal commentary. 
Commenter (0039) states that in December 2015, the National Association of Scholars called upon the NAS and the American Association for the Advancement of Science to revisit the 1956 Science paper; both declined. The commenter states that there are fairly well-known serious questions about ethical improprieties that may have contributed to flawed and overstated conclusions in the Science paper. The commenter states that due to the complacency of the scientific establishment and, no doubt, the prestige of some of the scientific bodies implicated in the cover-up, no one in a position or authority until now has been willing to challenge the linear no threshold dose-response orthodoxy.
Commenter (6148) states that restrictions imposed by LNT has left the U.S. behind in developing alternative uses for otherwise useful materials, primarily due to EPA's imposition of overly conservative safety factors by using the LNT, which in turn discourages continuing U.S. research, and in turn leaves U.S. standards stagnating at status quo. The commenter concludes that the use of scientifically developed linear dose response models, where appropriate, is needed to achieve transparency because they employ actual defensible data and risks, rather than non-scientific assumptions.
Commenter (0062) states that use of the LNT model as the default model for radiation-induced cancers by the EPA and other regulatory agencies worldwide during the past several decades, in spite of accumulating evidence supporting radiation hormesis, has not served the public well because of the unjustified fear of even small amounts of radiation it has generated, and the resulting expensive and harmful defensive actions. The commenter adds that it has increased regulatory burden on the enterprises that utilize radiation without any corresponding benefit of avoided diseases and that there are many adverse effects due to the use of the LNT model.
Commenter (0050) states that the LNT model is deeply flawed and scientifically worthless, and that there are no valid data supporting it. The commenter states that while there are many scientific papers purporting to support it, careful study shows that this so-called science is based upon flawed statistics (using only one-sided statistics to show harm, with no possibility of showing benefit), inappropriate controls, and poor experimental design. According to the commenter, the LNT model of radiation damage began as purposeful lies told by the genetics community in the 1950's, as a ploy to attract increased government funding for genetics research; it has spawned many useless (and dangerous) federal and state regulatory jobs, businesses, radiation damage lawsuits, and politicians who secure votes with an antinuclear and antiradiation agenda. The commenter continues, stating that the LNT radiation model is now more of a religion than a scientific theory, and getting rid of it is difficult. The American public has been lied to for over three generations, media assume the truth of the LNT model, and despite some 3000 scientifically valid papers showing that low levels of radiation are harmless or beneficial (hormesis), the EPA radiation scientists will not budge an inch, and are grasping the dead LNT myth with bloody fingernails; their jobs and programs are at risk, and they will never change their minds. 
Commenter (0050) states that low and moderately low radiation doses have been shown to be harmless or beneficial. The commenter explains that the reason is that these doses of radiation stimulate the production of enzymes involved in radiation repair; these enzymes, once stimulated, repair not only any damage caused by the radiation, but damage caused by other noxious agents, the most important of which is oxygen (the use of oxygen for energy metabolism generates damage that is qualitatively similar to the damage caused by ionizing radiation). The commenter adds that so far, over 150 genes have been identified that repair radiation and oxygen damage and the Nobel Prize in Chemistry in 2015 went to Paul Modrich, Tomas Lindahl, and Aziz Sancar "for mechanistic studies of DNA repair". The commenter states that an essential component of the LNT is that there is no such thing as radiation repair and concludes that this is absurd and fraudulent.
In support of the above comments, Commenter (0050) submitted their February 9, 2015, petition sent to Annette L. Vietti-Cook, Secretary, USNRC, Attention: Rulemakings and Adjudications Staff, U.S. Nuclear Regulatory Commission. The petition preface paragraph follows:
      I am submitting this petition for rulemaking pursuant to 10 CFR Part 2.802. The petitioner requests that the NRC amend 10 CFR Part 20, Standards for Protection Against Radiation, based on new science and evidence that contradicts the Linear No-Threshold (LNT) hypothesis, a model that has served as the basis for radiation protection regulations. I will present scientific data as reported in study after study to justify that safety regulations and policies should no longer be derived from the LNT model in order to ensure these requirements are more risk-informed. This ultra-simplistic concept assumes that all radiation absorbed doses, no matter how small, have a finite probability of causing a fatal cancer. The lower the quantity of radiation absorbed dose, the lower the probability of cancer induction, but the probability is never zero, let alone negative (i.e. beneficial or hormetic). The rate of radiation delivery is irrelevant, and all absorbed doses are additive; this is demonstrably false as evidenced by the practices of radiation oncology and of radionuclide therapy. Use of the LNT assumption enables regulators to feel justified in ratcheting down permissible worker and public radiation levels, either through actual dose limits or use of the "as low as reasonably achievable" (ALARA) principle, giving the illusion that they are making everyone safer (and creating ever-increasing workload for themselves and their licensees). There has never been scientifically valid support for this LNT hypothesis since its use was recommended by the U.S. National Academy of Sciences Committee on Biological Effects of Atomic Radiation (BEAR I)/Genetics Panel in 1956. The costs of complying with these LNT-based regulations are enormous. Prof. Dr. Gunnar Walinder has summed it up: "The LNT is the greatest scientific scandal of the 20th century."
Support Use of Linear Non-Threshold Model for Radiation
Conversely, commenters (0776, 2159, 2687, 6133, 6149, 6169, 6882, 6924, 6928) state that the NAS has repeatedly confirmed that there is no safe dose of radiation and that the LNT model is the best option for setting radiation protection standards. Commenter (6169) states that the NAS review constitutes peer-review of the LNT, and that the public was also included in the multi-year process. The commenter states that the public provided written and verbal comments and that at least two panelists were removed due to conflicts of interest that were pointed out by public comment.
Commenters (6133, 6169, 6928) elaborate that the NAS Biological Effects of Ionizing Radiation VII (BEIR VII) committee released a report in 2006 on the impact of low doses of radiation, Health Risks from Exposure to Low Levels of Ionizing Radiation: BEIR VII Phase 2, concluding that the LNT was the best model for protection against radiation. Commenters 6133 and 6149 cite the National Council on Radiation Protection and Measurements' (NCRP's) Commentary No. 27, "Implications of Recent Epidemiologic Studies for the Linear-Nonthreshold Model and Radiation Protection" (2018), which concluded that the LNT is the most practical model for implementing radiation protection for workers and the public.
Commenter (6169) states that more than 50 years of peer-reviewed, validated, and repeatable in vivo, not just laboratory studies, underlie the LNT model. The commenter adds that to its credit, despite pressure from special interests, EPA continues to recognize the model's pervasive scientific underpinnings, recently stating:
      Of all the agents demonstrated to be carcinogenic, the evidence for LNT is particularly strong for ionizing radiation...[g]iven the continuing wide consensus on the use of LNT for regulatory purposes as well as the increasing scientific confirmation of the LNT model, it would be unacceptable to the EPA to ignore the recommendations of the NAS and other authoritative sources on this issue. The EPA cannot endorse basing radiation protection on poorly supported and highly speculative proposals for dose thresholds or doubtful notions concerning protective effects from low-level ionizing radiation.
Commenter (6915) states that to "evaluate the appropriateness" of the linear, non-threshold approach for low-dose extrapolation by also considering non-linear and threshold models would provide no cognizable benefit in modeling accuracy or clarity, but instead could result in the manipulation of results, delay, and obfuscation.
Commenter (6133) provided a detailed argument for retaining the LNT model for radiation protection:
      As it does in every other instance and under every other environmental statute, EPA relies on independent, authoritative scientific bodies to provide analyses and evaluations of scientific evidence in support of its radiation standard-setting policies. EPA bases its regulatory limits and nonregulatory guidelines for population exposures to low-level ionizing radiation on the LNT dose-response model. EPA's radiation protection standards are based on the premise that any radiation dose carries some risk, and that risk increases directly with dose. This method of estimating risk is called the "linear no-threshold dose-response model (LNT)". 
      This longstanding and well-supported assumption presumes that the risk of cancer due to a low dose exposure is proportional to dose, with no threshold. For over 40 years the LNT dose-response model has been commonly utilized when developing practical and prudent guidance on ways to protect workers and members of the public from the potential for harmful effects from radiation in balance with the commercially justified and optimized uses of radiation. EPA derives the LNT model from reports by authoritative scientific bodies including the NAS, the NCRP, and the International Commission on Radiological Protection (ICRP). There is strong scientific consistency by these authoritative groups that an LNT model is the best at the current time (and has been for the past half century)., Indeed, EPA noted as recently as late 2015, "[o]ver the last half century, numerous authoritative national and international bodies have convened committees of experts to examine the issue of LNT as a tool for radiation regulation and risk assessment... Again and again, these bodies have endorsed LNT as a reasonable approach to regulating exposures to low dose radiation."
                  Studies in support of the LNT dose-response model
      The NAS BEIR VII committee has studied and published its report on risk models for estimating the relationship between exposure to low levels of ionizing radiation and harmful health effects. The data used in the BEIR VII study are: atomic bomb survivor studies, medical radiation studies, occupational radiation studies, and environmental radiation studies. The committee judged that the LNT model provided the most reasonable description of the relation between low dose exposure to ionizing radiation and the incidence of solid cancers that are induced by ionizing radiation. 
      The NCRP published its latest commentary on the LNT issue only months ago, in April 2018. The specific purpose of its commentary is to provide a review of recent epidemiologic data from studies with low doses or low dose rates and the Life Span Study (LSS) of atomic bomb survivors to determine whether these epidemiologic studies broadly support the LNT dose-response model as a reasonable basis for radiation protection. Epidemiologic studies of humans provide evidence that is critically important in establishing potentially causal associations of environmental factors with the disease. The studies were selected by a consensus of experts who have a broad purview of the recent radiation epidemiology literature, and they ensured that the largest and most important eligible studies were included.
      Examples of studies of radiation-exposed populations evaluated are:
 Japanese atomic-bomb survivors
            The LSS is a research program investigating life-long health effects based on epidemiologic studies. The study being conducted by the Radiation Effects Research Foundation (RERF) is used by standard-setting bodies in establishing a recommendation for radiation protection. The LSS cohort includes both a large proportion of survivors who were within 2.5 km of the hypocenters at the time of the bombings and a similar-sized sample of survivors who were between 3 and 10 km from the hypocenters whose radiation doses were negligible. 
            The major objective of the study is to investigate the long-term effects of atomic-bomb radiation on causes of death and incidence of cancer. The atomic-bomb survivors of Hiroshima and Nagasaki are subject to follow-up study,  for their remaining lives, starting from 1950. The LSS cohort of atomic-bomb survivors has provided important data because it is a large cohort (~87,000 survivors of all ages) with relatively accurate dosimetry, a wide dose range over 60 years of high-quality follow-up for mortality and over 50 years of follow-up for cancer incidence, and nearly 1,000 excess solid-cancer cases, besides excess leukemias. The study provides strong indirect support for the use of an LNT model. 
 Worker exposure studies 
            Radiation worker studies assess risks in worker groups exposed largely to many low doses received at a low dose rate, providing direct evidence regarding the validity of the LNT model. INWORKS is an example of these studies. INWORKS is the latest international collaboration for examining the health of workers in more than one country who were exposed occupationally to ionizing radiation. INWORKS included dosimetry for 20 different nuclear sites/organizations in three countries. Dosimetry was based on individual personal dosimeter readings at the start of the workers beginning their radiation work (at earliest, between 1944 and 1952) through 2005. The U.S. cohort of INWORKS consisted of 119,195 nuclear workers at four Department of Energy nuclear weapons facilities (Hanford site, Idaho National Laboratory, Oak Ridge National Laboratory, and Savannah River site) and at the Portsmouth Naval Shipyard. This large study provides one of the strongest pieces of epidemiologic evidence that the LNT quantitative model is useful for radiation protection. 
 Environmental exposure studies
            An example of environmental exposure studies for low doses and low dose rate is the Chernobyl resident cohorts.,  The 1986 accident at the Chornobyl nuclear power plant in northern Ukraine resulted in the exposure of substantial proportion of Belarus, Ukraine, and the Russian Federation to radioactive fallout. The most notable apparent health consequence of the accident has been the large increase in thyroid cancer among those exposed as children or teenagers starting 4-5 years after the accident. Studies of cohorts of children in Ukraine and Belarus who had thyroid measurements of iodine activity shortly after the Chernobyl accident and systematic thyroid screenings were conducted. The data on exposure to radioactive iodine have added considerable information relative to the dose-response relationship. The thyroid cancer experienced by children in exposed areas of the Ukraine, Belarus, and Russia conforms to the LNT model.
 Medical exposure studies 
            Patients treated with lung collapse for TB in the 1930s to 1960s are one of the few medically exposed populations that provide consistent evidence for dose-response relationships. Patients on average would receive on the order of 100 chest fluoroscopies over several years.

Since the 1970s, studies of TB patients who received repeated chest x-ray fluoroscopies to monitor lung collapse have provided valuable information relevant to the LNT hypothesis. The TB fluoroscopy studies provide strong support for the LNT model for breast cancer. 
      NCRP commentary in conclusion of its epidemiology studies states that, based on current epidemiologic data, the LNT model should continue to be used for radiation protection purposes, and "no alternative dose-response relationship appears more pragmatic or prudent for radiation protection purposes than the LNT model."
Response: These comments are outside the scope of this rule. This is a rule of internal agency procedure, and does not describe the Agency's position on scientific debate over the LNT model. Instead, under this rule the EPA continue to implement existing Agency practice and review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information. The EPA will continue to use the following, established factors to assess the quality of studies used to develop significant regulatory actions and influential scientific information: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review.[,] When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
21.1.2.1.2 Public Transparency of National Academy of Sciences (NAS) Process and Availability of Report/References Illustrated Greater Harm to Women
Comment: Commenter (6169) notes that, although EPA instituted a peer review for radiation models by tasking the NAS to conduct a review of radiation science, including the linear, no threshold, and other models, and it supported the use of the LNT, it failed to point out the increased harm to women and children in the data. The commenter notes that this is due to its averaging of damage across genders and ages. The commenter notes that the only reason to revisit is to un-average the results and provide additional protection for women and children. The commenter provides extensive arguments in support of the use of the LNT model for radiation.
Response: This comment is unrelated to the current rulemaking and therefore out of scope. This is a rule of internal agency procedure and does not relate to peer review for radiation models.
21.1.2.1.3 EPA Should Incorporate Uncertainty and Stochastic (Random) Impacts of Radiation
Comment: Commenter (6169) notes that one major recognized category of uncertainty in radiation is the stochastic (random) impacts of radiation, including the probability that random impacts will increase with an increasingly radiologically contaminated environment. The commenter notes that any model that incorporates uncertainty must incorporate our ever-increasing probability of exposure, in addition to the non-threshold nature of stochastic cancer and non-cancer impacts.
Response: This comment is unrelated to the current rulemaking and therefore out of scope. This is a rule of internal agency procedure and does not concern radiation studies.
21.1.2.1.4 EPA Needs to Rely on Radiation Health Studies that are Independent and Scientifically Substantiated
Comment: Commenter (6169) suggests that the EPA rely on radiation health studies that are independent and scientifically substantiated; and shun those that are not applicable to radiation protection, suffer from well-known and documented pseudo-scientific assumptions, or harbor conflicts of interest. The commenter notes that even report conclusions that are generally accepted, such as those found in NAS BEIR VII, can still reference, and be influenced by, inappropriate studies. The commenter contends that this is perhaps one of the reasons why women, children and pregnancy are not afforded the protection they require. 
Commenter (6169) states that improper hypothesis formation and research design have been baked into, and continue to plague, studies on radiation's impact on health. According to the commenter, this improper methodology has, unfortunately, evolved to be the only accepted way of examining health impacts. The commenter notes that this rigid study design presupposes a conclusion of no impact before any health data are actually examined, a huge scientific faux pas. The commenter explains that design does this by relying on error-ridden dose reconstruction, which almost always concludes that environmental exposure was too low to cause health impacts. Therefore, the commenter notes that even though this research often finds increase in disease, associating it with radiation exposure becomes impossible, not because radiation isn't the cause, but because it was falsely vindicated from the beginning. 
Further, commenter (6169) provides that there are a number of studies regarded as reliable by international radiation committees, such as ICRP or NCRP, which find no health impact, or claim no discernable impact. According to the commenter, many of these studies were actually funded or influenced by industry and often used in court to counter claims of health damage from radiation. Worse still, the commenter notes that these studies also received funding from Federal agencies, giving them the appearance of independence. The commenter contends that studies like these skew the overall balance of radiation damage assessment when committees, such as those from NAS, try to review the literature examining radiation's impact on health.
Response: These comments are outside the scope of this rule. This is a rule of internal agency procedure and does not describe the Agency's position on which radiation health studies to rely. Instead, under this rule the EPA continue to implement existing Agency practice and review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information. The EPA will continue to use the following, established factors to assess the quality of studies used to develop significant regulatory actions and influential scientific information: soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review.[,] When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
21.1.2.1.5 Radiation Protections Insufficient
Comment: Commenter (6882) states that the proposed rule would apply to radioactive substances, for which Environmental Protection Agency protection is already not protective enough and that the few protections that EPA regulations do offer must remain intact, or, better yet, strengthened. Commenter (6928) states that the EPA believes the official radiation models which underestimate harm.
Commenters (0776, 2159, 2678, 6882) note that radiation protections must be more protective for sensitive females since their cancer risk from radiation exposure is higher. Commenters (0776, 2159, 6882) provide that the EPA doesn't specifically protect against non-cancer impacts of radiation exposure and should establish a hazard index for radionuclides -- something the Environmental Protection Agency already has for toxic chemicals. 
Commenter (2687) expresses that radiation protections do not cover all of the various nuclides that are contaminating public and tribal water and are not strict enough considering that children and women are affected up to seven times more than the standard 35-year old white man.
Commenter (2159) adds that the EPA should also figure out how much radiation exposure is costing us, including loss of salary from impaired neural development and lower IQ, cost of treating radiation-induced cancers, and cost of accumulating radiation damage across generations.
Commenter (2159) provides that ICRP, the United Nations Scientific Committee on Effects of Atomic Radiation (UNSCEAR), the International Atomic Energy Agency (IAEA), the U.S. National Academies of Sciences, BEIR VII, and the UK National Radiological Protection Board (NRPB) are regularly quoted on radiation risk impacts on people. The commenter notes that they utilize a model of radiation risk assessment based on physics, calculating average energy impacts on a given mass or population, developed before the discovery of DNA and based on high-dose acute external irradiation by gamma rays of a large number of Japanese inhabitants of Hiroshima and Nagasaki. The commenter adds that these studies were begun 5 or more years after the atomic bombing and that the reference was to impact on a 25 year old, healthy white male. According to the commenter, left out of consideration are alpha and beta particle radiation which is not highly penetrating like gamma rays but, breathed in or ingested, become very damaging internal emitters. Also, the commenter notes that women are 45% more susceptible to radiation than men, children more than adults, girls more than boys and the younger the child, the greater the damage. Furthermore, the commenter notes that ionizing radiation is variable and individual susceptibility is as well.
Commenter (2159) provides that the European Committee on Radiation Risk (ECRR), in its 2010 Recommendations, considered methodology for evaluating radiation risk and presented the following: 
      The Committee concludes that the present cancer epidemic is a consequence of exposures to global atmospheric weapons fallout in the period 1959-63 and that more recent releases of radioisotopes to the environment from the operation of the nuclear fuel cycle will result in significant increases in cancer and other types of ill health. The ECRR model predicts 61,600,000 deaths from cancer, 1,600,000 infant deaths and 1,900,000 foetal deaths. In addition, the ECRR predicts a 10% loss of life quality integrated over all diseases and conditions in those who were exposed over the period of global weapons fallout. The total maximum permissible dose to members of the public arising from all human practices should not be more than 0.1mSv, with a value of 2mSv for nuclear workers. This would severely curtail the operation of nuclear power stations and reprocessing plants, and this reflects the Committees belief that nuclear power is a costly way of producing energy when human health deficits are included in the overall assessment.
Response: These comments are out of scope, as the current rulemaking does not establish or modify radiation protections.
21.1.2.2 EPA Climate Change Model Forecasting Flawed
Comment: Commenter (4541) states that EPA has tried to develop elaborate models forecasting the effect of increasing atmospheric carbon dioxide (CO2) on global temperatures 100 years into the future, but the commenter notes that those projections have already departed far from reality. According to the commenter, the models upon which EPA relies have often hidden stubborn falsehoods within their underlying structure. The commenter contends that increased model complexity has allowed modelers to hide problems where no one can find them. The commenter states that we are expected to believe the models because they are "state of the art," but even with a large amount of funding it would be impossible to test all their assumptions. The commenter contends that there needs to be a focus on simpler models with traceable and reproduceable assumptions and results. And, when the models disagree with observations (evidence), they need to be corrected. 
Commenter (4541) asserts that the EPA's scientific foundation for potentially catastrophic anthropogenic global warming is based on the temperature projections of dozens of the supposedly more sophisticated global climate models. According to the commenter, those climate models are not sound science and are merely speculative scenarios about climate, none of which has been validated by the historical temperature record. The commenter states that scientific method involves testing a falsifiable hypothesis with experiments and evidence and climate model projections do not involve any such falsifiable hypothesis, so they are not an exercise of the scientific method.
Commenter (4541) notes that even the modelers themselves recognize and admit their models are not designed to produce predictions of future temperatures, only "what if" scenarios of results assuming the accuracy of unproven assumptions. The commenter provides that the Summary for Policymakers of Climate Change Reconsidered II: Physical Science states, "The science literature is replete with admissions by leading climate modelers that forcings and feedback are not sufficiently well understood, that data are insufficient or too unreliable, and that computer power is insufficient to resolve important climate processes."
Moreover, according to the commenter (4541), none of the models adequately accounts for the Pacific and Atlantic Ocean temperature cycles (PDO and AMO). The commenter notes that none takes into account solar activity cycles indicated by variations in the number and size of sunspots, variations in solar magnetic fields, or cosmic rays flux, all of which are known to significantly affect climate. The commenter states that natural cycles have produced major climate changes in the past, such as the Minoan (peak at 1300 BC), Roman (peak at 100 BC) Medieval Warm Period (about AD 950 to 1250  -  during which "global temperatures" were higher than they are today), and the early twentieth century warm period from 1915 to 1945.
Commenter (4541) states that design flaws explain why the projections of all climate models have now diverged dramatically from the actual temperatures experienced over the past two decades. The commenter provides a figure in their commenter letter (Figure 1) that illustrates that there has been no global warming for nearly 20 years. The commenter adds that projections of the models, and their increasing divergence from real-world temperature observations are illustrated in a figure in their comment letter (Figure 2). According to the commenter, the graph presented in Figure 2 was created by NASA scientist John Christy, Ph.D., who, with his colleagues at the University of Alabama in Huntsville, monitors atmospheric temperatures as computed from the data collected by U.S. satellites. The commenter further points out that Figure 2 illustrates the actual atmospheric temperatures recorded by U.S. weather satellites and weather balloons as compared to the average of the climate models, where the average projection is well above the observed real-world temperatures, with the divergence growing over time.
Commenter (4541) asserts that this growing divergence of the models from reality invalidates and falsifies them, as does:
 the inability of the models to account for the natural global temperature cycles and changes of the past century;
 the inability of the models to account for the Little Ice Age and Medieval Warm Period over the past 1,000 plus years; and
 the projection by all models of a tropical hotspot produced by CO2 that doesn't exist in the real world.
According to the commenter, what is most shocking is how weak the models are as any sort of "evidence" at all for the idea of catastrophic anthropogenic global warming. 
Response: These comments are outside the scope of this rulemaking. This is a rule of internal agency procedure and does not pertain to climate change models.
21.1.2.3 Particulate Matter NAAQS Supporting Information
Comment: Several commenters contend that the supporting data for particulate matter NAAQS is strong, results have been replicated, and increasing supporting information adds to the supporting data for the NAAQS. 
Commenters (5178, 6874, 8272) state that the ACS Study linked air pollution data from 151 U.S. metropolitan areas to data on risk factors and mortality in 552,138 adults residing in these areas who enrolled in 1982 and were followed through 1989. The commenters state that the authors found increased mortality to be associated with sulfate and fine particulate air pollution "at levels commonly found in U.S. cities."
Commenters (1791, 5178, 6109, 6114, 6125, 6874, 6895, 6939, 8272) assert that the results of the Six Cities Study and the ACS Study have stood up to extensive subsequent analysis, including the "Reanalysis of the Harvard Six Cities Study and the American Cancer Society Study of Particulate Air Pollution and Mortality," which highlights the strength of such research. 
Commenter (5015) provides a detailed synopsis of the Particle Epidemiology Reanalysis Project (which is not repeated here). The commenter states, in conclusion, the Reanalysis Team interpreted their findings to suggest that increased relative risk of "mortality may be attributed to more than one component of the complex mix of ambient air pollutants in urban areas in the United States". In the alternative analyses of the ACS Study cohort data, the commenter states that the Reanalysis Team identified relatively robust associations of mortality with fine particles, sulfate, and sulfur dioxide, and they tested these associations in nearly every possible manner within the limitations of the datasets. 
Response: These comments are outside the scope of this rulemaking. In this final rule the EPA is not categorically or specifically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. This includes those studies supporting the NAAQS cited by the commenter. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
21.1.2.4 National Primary Drinking Water Regulations Arsenic Standard 
Comment: Commenter (6172) asserts that EPA should identify public health levels for all National Primary Drinking Water Regulations (NPDWR), including arsenic. The commenter states that levels of regulated substances that are unsafe in drinking water as referenced in the Safe Drinking Water Act (SDWA) are: an "unreasonable risk to health," section 1415 and "protective of public health," section 1412. The commenter states that, while the EPA established a standard of 10 parts per billion (ppb) for arsenic in drinking water, in 2002, EPA did not find that arsenic concentrations above their standard necessarily present an "unreasonable risk to health." The commenter states that, in their reply to a Congressional inquiry, EPA stated that it is "determining what does not pose an unreasonable risk to health with respect to arsenic, rather than address the much more complex issue of what does constitute an unreasonable risk to health." 
Commenter (6172) urges EPA to release all data used to craft SDWA policy including risk assessments of inorganic arsenic. The commenter states that this would allow for further assessment and demonstrate the repeatability of conclusions and comparisons to similar studies. The commenter states that a regulation that strengthens the transparency of science utilized by EPA will result in improved federal public policy and SDWA implementation. The commenter states that, for example, a recent study could influence the current Integrated Risk Information System (IRIS) program that is developing an updated assessment of inorganic arsenic, however, the study includes some data that is available to government agencies but not to the general public.
Response: The comments are outside the scope of this rulemaking. This is a rule of internal agency procedure, and does not pertain to national primary drinking water regulations. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule.
21.1.2.5 Oppose EPA's Lack of Regulation of Perchlorate
Comment: Commenter (6172) states that the lack of a clear principle or interpretation of SDWA section 1412 allowed EPA to decide not to regulate perchlorate on October 10, 2008 and a subsequent affirmative EPA decision on February 11, 2011, that perchlorate meets the section 1412 criteria for regulation as a contaminant. The commenter asserts that such ambiguity in decision-making does not provide for regulatory certainty or persuasive reasoning for the local governments. The commenter notes that regulations should result in public disclosure of a clear principle for implementing the discretion provided to the Administrator for selecting new contaminants for regulations under SDWA Section 1412. 
Response: The comments are outside the scope of this rulemaking. This is a rule of internal agency procedure, and does not describe the Agency's position on perchlorate. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers.
21.1.2.6 1,2,3-Trimethylbenzene (TMB) IRIS Assessment
Comment: Commenter (6362) states that EPA's 2016 IRIS assessment of 1,2,3-Trimethylbenzene (TMBs) does not accurately represent the health effects associated with exposure to TMBs because it fails to utilize a consistent and transparent data evaluation procedure for evaluating and weighing the full body of evidence.
Response: This comment is outside the scope of this rulemaking. This is a rule of internal agency procedure and does not pertain to the TMB IRIS Assessment .
21.1.2.7 Cost of Exposure to Radioactivity
Comment: Commenter (6169) suggests EPA should account for the true costs of exposure to radioactivity. The commenter states that internal costs to the nuclear industry are not the only consideration. The commenter states that allowing more radiation exposure externalizes costs to other industries, to individual members of the public, and to the population experiencing collective effects of radiation from the nuclear power and weapons industries. The commenter states that like non-cancer health impacts of radioactivity, EPA does not account for the economic impact of radiation exposures. The commenter states that any benefit to the nuclear industry of noncompliance or non-protection is a cost borne by the exposed public, other industries, and current and future inhabitants (human and non-human species and gene pools) of contaminated environments.
Response: This comment is outside the scope of this rulemaking. This is a rule of internal agency procedure and does not pertain to radioactivity.
21.1.2.8 Interagency Review
Comment: Commenters (8281-PH39, 9227, 8272, 6125, 6126, 6194, 6144, 6916, 9227) questioned the Executive Order 12866 interagency review process for the proposed rule. 
Response: Comments on deliberative processes, including interagency review and Executive Order 12866, are out of the scope of this rulemaking.
21.1.2.9 Federal Guidelines
Comment: Commenter (6915) provided interpretations of OMB Memorandum 13-13.
Response: OMB documents are not open to public comments and are out of the scope of this rulemaking. EPA maintains that the rule is designed to build upon federal guidelines.
Comment: Commenter (6133) asserts that greater clarity is needed in the process for determining which actions are identified as "significant regulatory actions" under Executive Order 12866.
Response: The determination of significant regulatory actions under Executive Order 12866 is not open to public comments and is outside the scope of this rulemaking.
21.1.2.10 Industry Should Be Required to Disclose Data
Comment: Commenter (5290) states that in the world of industrial chemicals, EPA should stop allowing corporations to self-report and start requiring them to disclose internal studies regarding the tens of thousands of existing and proposed chemicals that assault our health and ecosystem. For reference, the commenter suggests that EPA review an article discussing the discrepancy between DuPont's public statements about the safety of its Perfluorooctanoic Acid (PFOA) chemicals with the conclusions of its own confidential health studies, which included decades of evidence pointing to harm to public health (see https://www.nytimes.com/2016/01/10/magazine/the-lawyer-who-became-duponts-worst-nightmare.html)
Response: This comment relates to practices outside the scope of this rulemaking. This rule does not impose requirements on outside parties. This rule describes how the EPA will determine the consideration to afford pivotal science in its final significant regulatory actions and influential scientific information for which the conclusions are driven by the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect based on the availability of the underlying dose-response data, in addition to other factors.
21.2 SNPRM Miscellaneous Comments and Out of Scope Comments
21.2.1 Miscellaneous Comments
21.2.1.1 General Opposition/Support
Comment: While several commenters provided comments that detailed specific support and/or opposition to the rule with detailed comments on specific aspects of the initial proposal and supplemental proposal, several thousand commenters (including, but not limited to 10045, 10048, 10866, 10974, 11176, 11182, 11183, 11184, 11192, 11243, 11403, 11408, 11479, 11484, 11894, 11918, 12403, 12412, 12415, 12423, 12427, 12430, 12450, 12456, 12458, 12464, 12474, 12540, 12715, 12719, 12727, 13549, 14397) provided general opposition to the proposal. Some examples of general statements of opposition include the following:
 Commenter expresses strong opposition to the criteria proposed in the Supplemental Notice of Proposed Rulemaking (SNPRM, or supplemental proposal) put forth by the EPA relative to its rule, "Strengthening Transparency in Regulatory Science." (10866)
 The main provisions of this proposed rule are either unnecessary, duplicative, burdensome, or likely to be confusing in their application. The proposed rule will not deliver significant improvements in EPA's current regulatory processes or improve the public's confidence in science. In fact, the proposed rule is likely to slow the advancement of science in the U.S. and increase political interference in the conduct of science. (11182)
 One of the stated purposes of this SNPRM is to clarify vague, inaccurate, or inconsistent language in the 2018 proposed rule. The "clarifications" offered in the SNPRM demonstrate the intention to undermine science through the façade of increasing transparency. While the EPA frames the rule as reflecting "recent innovations and policies surrounding information access," and there is indeed a widespread movement for open data and open science (which the commenter supports), this proposed rule and its modifications in the attendant SNPRM are disingenuous to that spirit. (11192)
 While the commenter supports transparency pursuant to data and information accessibility and expanding public knowledge, the proposed rule would likely restrict public knowledge by dismissing and devaluing scientific input to the EPA's decisions. Neither the 2018 proposed rule nor this SNPRM offer potentially meaningful advances in the EPA's decision-making or the public's engagement with the decision-making process. (11361)
 The Agency uses best practices that allow for a weight of the evidence approach, and this rule would not only change the process but codify it, making it much harder to reverse. Part of that codification would be to add the nonscientific criterion "data availability" to the list of assessment factors that the EPA will use to determine the quality of a study. These factors have been guiding EPA's data quality procedures since 2003 and include soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. They help the Agency to assess the context surrounding the objectiveness and integrity of scientific information and to determine how to use it in its weight of the evidence approach. (11479)
 The EPA received thousands of comments on the initial rule in 2018, did not respond to those comments, nor hold any sort of engagement with scientists, community members, nor organizations who submitted comments is reflected in the Strengthening Transparency in Regulatory Science SNPRM. In fact, the Supplemental Notice does not reflect any of the technical or legal comments submitted in 2018. (12540)
 The EPA's proposed rule and supplemental proposal attempts to implement a mechanism to shortcut its own guidance, guidance reports from the National Academy of Sciences, and guidance reports mandated by congressional actions for the implementation of national environmental statutes for the past 50 years. (12403)
 The Transparency Rule and the modifications included in the SNPRM will negatively impact important public health research and endanger public health, limiting the use of critical research that has formed the basis for improved understanding of risks, particularly risks associated with environmental exposure to toxic substances. (11918)
Conversely, there were also commenters (10973, 11358, 11388, 11899, 11915, 12612, 12692) that provided general statements of support for the initial proposal/supplemental proposal. Some examples of these statements include the following:
 Commenter supports transparency in all EPA actions (both regulatory and non-regulatory), including making the data and information used to inform EPA decision-making public in most situations. In taking steps to improve the transparency of current EPA rulemaking processes, it is important that the Agency appropriately balance three major considerations: EPA's statutory obligations, transparency of influential (pivotal) decision-making processes, and the public and business communities' rights to privacy. (10973)
 Commenter appreciates EPA's work to clarify the proposed rulemaking, as the Agency's initial proposal was too vague and missing key components, which would have made it difficult to understand and to put into practice. The commenter would like to thank the Agency for adding and clarifying many definitions for key terms within the supplemental notice. These modifications to the proposal are critical in assuring any final rule is implementable and defendable. (11358)
 Commenter supports transparency in both regulatory and non-regulatory EPA actions, including the data and information used to inform EPA decision-making; and appreciates the additional information and definitions provided in the supplemental notice, as this information is critical to understanding the scope and meaning of the rule. (11388)
 Full transparency, as proposed by EPA and in accordance with their further recommendations, would go a long way toward eliminating incompetence, fraud and politics substituting for science  -  and thereby reducing the number of costly, burdensome regulations that have severely impacted American businesses, communities and families, for few benefits. The commenter asserts that, in reality, public health and safety depend on ensuring that research and data purportedly supporting it are made public and carefully reviewed by multiple experts, to ensure accuracy, replicability and integrity. The commenter concludes that EPA will take full advantage of all available research that passes these tests. (11899)
 The rule is necessary to encourage and enforce the use of the scientific method among the investigators who are conducting research, particularly epidemiologic research, that is relevant to EPA regulatory policy. (12612)
Response: The EPA is providing response to the comments received for both the 2018 notice of proposed rulemaking and 2020 supplemental notice of proposed rulemaking as part of the submission of the final rule. The 2020 SNPRM was developed to clarify issues noted by comments on the 2018 NPRM.
The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
The EPA considered these comments when finalizing this rulemaking, and the EPA does not agree that its approach is "unnecessary, duplicative, burdensome, or likely to be confusing." As described in the rule and preamble, the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. When characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect, the EPA will identify pivotal science and give greater consideration to pivotal science for which the underlying dose-response data are available in a manner sufficient for independent validation. Including this review of dose-response data availability for pivotal science is critical to the EPA's progress toward increased transparency and providing increased opportunity for scientific reanalysis and review by independent third parties. This approach will result in significant regulatory actions and influential scientific information that are based on the highest quality, most relevant studies and ensure adequate human health and environmental protections, while maximizing transparency and maintaining consistency with the statutes the EPA administers.
21.2.1.2 Cannot Make Claim That Comments Are Out of Scope
Comment: Commenters (11403, 12427) assert that EPA cannot consider comments involving the 2018 proposed rule out of scope. The reasoning includes:
Commenter (11403) states that because EPA failed to address so many of the problems with the 2018 proposed rule, the commenter has readdressed several in this comment. The commenter notes that EPA has suggested in this proposed rule that comments "that address aspects of the 2018 proposed rulemaking that are not addressed, altered, or replaced by this SNPRM will be deemed outside the scope of this supplemental action." Rather than risk EPA deeming any comment "outside the scope," the commenter reminds EPA that when reopening a rule for public comment, the rule is "challengeable anew," and therefore nothing from the 2018 Proposal may be deemed outside the scope.
Commenter (12427) provides that, although EPA is requesting comments now only on the supplemental notice, the overall effect of the SNPRM cannot be disentangled from that of the proposed rule, since one builds on the other. Thus, the commenter contends that it will be unreasonable, arbitrary, and capricious for EPA to refuse now to accept comments that pertain to the joint interactive effect of the Proposed Rule and SNPRM.
Response: Responses to comments have been provided for both the comments on the 2018 NPRM and the 2020 SNPRM. The only comments found to be out of scope are those found not to be relevant to the rule.
21.2.1.3 Alternative Options for Achieving Transparency
Comment: Commenter (10039) contends that EPA may find the Administrative Conference of the United States (ACUS) Recommendation 2013-3, Science in the Administrative Process, 78 Fed. Reg. 41,357 (July 10, 2013), especially relevant to this proposed rulemaking. The commenter notes that recommendation 2013-3 highlights innovative Agency practices to "enhance the transparency of their scientific decision-making practices" and offers "proposals to bring greater congruity to the treatment of publicly and privately funded scientific research." The commenter provides that all ACUS recommendations and their accompanying reports are available online at www.acus.gov.
Response: Thank you for the recommendation. The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. Where underlying dose-response data in pivotal science are available, subject matter experts could independently reanalyze the data to affirm original research conclusions, check for errors, test alternative assumptions, and better understand and evaluate the implications of the uncertainty. Such independent reanalyses will subsequently enable the EPA to make stronger, data-driven decisions in future rulemakings or in revisions to existing rules or influential scientific information. This could occur through standard cyclical reviews (e.g., revisions to NAAQS, risk and technology reviews, national primary drinking water regulations), ad hoc revisions, or revisions through the information quality guidelines or other petition processes. Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information.
Comment: Commenter (10740) reiterates Steinzor and Wagner's proposal for transparent science in that:
   Transparent science should make publicly available
          a conflict of interest disclosure statement if the study was privately sponsored, as well as the underlying contract governing that research, in order to ensure that researchers' independence to determine study design and report results was preserved;
          a clear statement of the methods for data collection and analysis used in the study to allow for scrutiny and even replication of the study; and
          all of the underlying data, presumably in digital form (e.g., not original specimens, etc.).
   A proposal for transparent science should
          apply the same standards to all scientific research and analyses used by the Agency, particularly research that is not published and that has escaped rigorous peer review;
          require that a list of all excluded research be shared with the public as the decisions are made. Such disclosure could be accomplished by listing excluded -- or presumptively excluded -- information on a dedicated website in the course of a rulemaking or Agency decision.
          be applied to all technical analyses prepared by the Agency.
According to the commenter, this proposed rule neither conforms with the principles above nor meets the requirements of a proposal for real transparent science.
Response: The EPA disagrees with the commenter. The application of the principles discussed in their comments are a decision made by a researcher. This final rule does not regulate any entity outside the EPA. Rather, in this action, the EPA is finalizing a rule of internal agency procedures, including how the Agency will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information. In doing so, the final rule provides greater transparency on the consideration the EPA will give pivotal science where the underlying dose-response data are or are not available for independent validation.
Consistent with existing Agency practice, the EPA will review and evaluate all relevant scientific studies when developing significant regulatory actions and influential scientific information. The EPA will continue to use the following, established factors to assess the quality of studies used to develop significant regulatory actions and influential scientific information: 
    Soundness  -  The extent to which the scientific and technical procedures, measures, methods or models employed to generate the information are reasonable for, and consistent with, the intended application. 
    Applicability and Utility  -  The extent to which the information is relevant for the Agency's intended use.
    Clarity and Completeness  -  The degree of clarity and completeness with which the data, assumptions, methods, quality assurance, sponsoring organizations and analyses employed to generate the information are documented.
    Uncertainty and Variability  -  The extent to which the variability and uncertainty (quantitative and qualitative) in the information or in the procedures, measures, methods or models are evaluated and characterized.
    Evaluation and Review  -  The extent of independent verification, validation and peer review of the information or of the procedures, measures, methods or models.
When evaluating potential links between exposure to a pollutant, contaminant, or substance and effects and the nature of the dose-response relationship, the EPA will follow best practices and rely on the highest quality, most relevant studies in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well-substantiated evidence (consistent with Agency guidelines on hazard identification and dose-response assessment) to support a relationship between exposure and effect, the EPA will identify a subset of those studies for use in characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect. This will be based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements, quantitative analyses, or both will be identified as pivotal science. Once the EPA has identified pivotal science -- for either significant regulatory actions or influential scientific information -- the EPA will then evaluate if the underlying dose-response data are available in a manner sufficient for independent validation. The EPA will give greater consideration to pivotal science for which the underlying dose-response data are either publicly available in a manner sufficient for independent validation or, in the case of personally identifiable information (PII), CBI, or proprietary data, available through restricted access that affords privacy in a manner sufficient for independent validation. 
The EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
Finally, Part 30.4 of the rule states, "The EPA shall clearly identify all science that serves as the basis for informing a significant regulatory action. The EPA shall make all such science that serves as the basis for informing a significant regulatory action publicly available to the extent permitted by law." 
Comment: Commenter (10974) notes that improvements to developing health-based benchmark values used in state air toxics and permitting reviews can be made with certain data practices, but EPA would be better served supporting state efforts to make existing data easier to understand, to help better communicate research results and conclusions, and to help states develop third-party partnerships.
Response: This final rule does not regulate any entity outside the EPA. Rather, in this action, the EPA is finalizing a rule of internal agency procedures, including how the Agency will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information. In doing so, the final rule provides greater transparency on the consideration the EPA will give pivotal science where the underlying dose-response data are or are not available for independent validation. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
Comment: Commenter (11192) states that while they support transparency pursuant to data and information accessibility and expanding public knowledge, the proposed strengthening transparency in regulatory science (STRS) rule would likely restrict public knowledge by dismissing and devaluing scientific input to the EPA's decisions. The commenter asserts that it is important to note that there are several ways the EPA could actually make its regulatory decision-making more clear to the public and support public access to data, including: (1) funding researchers to de-identify data, (2) creating a user-friendly data repository with de-identified data, (3) conducting scientific assessments for all proposed regulations, and (4) providing syntheses and summaries of science assessments at multiple levels of detail to be comprehensible by various interested parties. According to the commenter, neither the 2018 Proposed Rule nor this SNPRM offer potentially meaningful advances in the EPA's decision-making or the public's engagement with the decision-making process.
Response: The EPA disagrees with the commenter, the Agency believes that this final rule will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule. 
The EPA is not categorically excluding any studies from consideration - even studies where the underlying dose-response data are not available for independent validation - when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
Comment: Commenter (12427) contends that the proposed rule and SNPRM fail to address a more pressing issue, publication ethics. The commenter notes that in the context of publication ethics, "Transparency" has a different meaning. According to the commenter, transparency means that sources of funding for research or publication should always be disclosed. The commenter states that authors should disclose role of the funding source in designing the research, recruiting investigators/authors, collecting the data, analyzing the data, preparing the manuscript, or controlling the publication process; and that, in fact, many journals require this. Furthermore, the commenter provides that conflicts of interest should be disclosed by authors and evaluated by reviewers. The commenter notes such disclosures should include "information about financial (for example, patent ownership, stock ownership, consultancies, speaker's fees), personal, political, intellectual, or religious interests relevant to the area of research or discussion. According to the commenter, publication ethics include respecting confidentiality: "In the majority of cases, editors should only consider publishing information and images from individual participants/subjects or patients where the authors have obtained the individual's explicit consent.".
Response: This final rule does not regulate any entity outside the EPA. Rather, in this action, the EPA is finalizing a rule of internal agency procedures, including how the Agency will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information. As stated in Part 30.3, "The provisions of this part apply to science that serves as the basis for informing a significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science."
Comment: Commenter (12546) states that the 2016 Plan to increase access to results of EPA-funded scientific research. (2016 Plan) is scientifically and technically sound; thus, EPA should focus on implementing the 2016 EPA Plan instead of the proposed rule and associated SNPRM.
Response: EPA has implemented the public access requirements for EPA researchers, grantees and contractors as required by the February 22, 2013, Office of Science and Technology Policy (OSTP) memorandum "Increasing Access to the Results of Federally Funded Scientific Research" and the 2016 EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research." EPA researchers, grantees and contractors are required to make peer-reviewed journal articles and the underlying data and models resulting from EPA-funded research accessible to the public. The EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research" requires the data underlying a journal article resulting from research funded by EPA be made publicly accessible. 
However, this final rule, as stated in Part 30.3, applies "to science that serves as the basis for informing a significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science."
It should also be noted that the EPA's 2016 Plan to Increase Access to Results of EPA-Funded Scientific Research noted that "transparency is a core EPA value" and that increased availability of research data would accelerate scientific breakthroughs that support the Agency's mission and policymaking efforts.
Comment: Commenter (12422) states that many in the scientific and regulatory communities recognize the importance of transparency and access to the data underlying important studies. The commenter asserts that addressing these issues can be accomplished without injuring the scientific enterprise or discouraging scientists from pursuing pressing scientific questions. According to the commenter, mechanisms to accomplish this that do not require new regulation include development of public access databases, use of the Freedom of Information Act (FOIA), and reanalysis by third parties. The commenter provides that certain journals, for example, require authors to agree to make their data available to editors and others upon request.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions.
Many scientific publications do require authors to make a data availability or data access statement, which discloses where and under what conditions the underlying study data are available. Yet the EPA cannot solely rely on data availability statements made in published research because initiatives toward greater data sharing and transparency amongst scientific journals and international organizations are still being implemented, are inconsistently enforced, and the true accessibility of data in a public repository is still limited. For example, Christensen et al. (2019) evaluated 1,072 peer-reviewed articles and "found that rates of data availability for empirical articles published after journals adopted data-sharing policies differ widely between journals, from 0 percent to 83 percent, with a mean of 35 percent."
Comment: Commenter (11921) notes that while access to underlying data for a particular study can be valuable, the systematic assessment of all of the available scientific evidence is the most important component of environmental policy making. The commenter provides that evidence integration (NRC 2014) is used by a wide variety of organizations when evaluating the information for the potential harm caused by environmental agents. Given the diverse nature of disciplines and methods that can shed light on health effects, the commenter states that most data synthesis procedures evaluate the findings from myriad studies using different approaches, even if they might place different weights on them. Ideally, according to the commenter, the degree of confidence in the findings of an effect increases along a spectrum of results that progressively converge with one other, beginning with the technical and growing to a more conceptual level.
Commenter (11921) states that recognizing that each discipline, method and study has its own limitations, a key step in verification of any particular finding is through "triangulation" (Cite: Lawlor, 2016, Pearce, 2019), that is, addressing the same question using multiple lines of evidence from disparate approaches to compensate for the inherent limitations in any one approach -- the more different such approaches, the better. The commenter notes that a different population, also possibly with a different design or strategy (e.g., examination of the impact of air pollution reduction vs. observations based on existing patterns alone) is more persuasive than solely testing if the results of the original study were valid. The commenter adds that the convergence of findings from experimental exposure of human subjects and epidemiologic findings can provide confirmation. The commenter further adds that mechanistic insights, based on animal, cellular and in vitro studies, can provide the biological underpinnings and plausibility of clinical or epidemiological observations.
Commenter (11921) states that the EPA approach to evidence synthesis - before the current proposal - has usefully relied on a form of evidence integration provided by examining all evidence from a variety of sources to reach final determination of exposure and health effects (US EPA 2015). The commenter adds that other federal agencies, such as the Office of Health Assessment and Translation within National Institute of Environmental Health Sciences (NIEHS) (OHAT, 2015), have also applied similarly comprehensive and systematic approaches.
There are, of course, according to the commenter (11921), opportunities to improve the systematic review processes at EPA and other agencies, and there is active discussion about the best way to conduct systematic reviews. The commenter notes that the use of evidence from the widest range of study types and designs is by far the most effective and widely accepted way to draw scientific conclusions. The commenter contends that while reproduction of the results of single studies is sometimes helpful in such systematic reviews, limiting evidence synthesis and assessment to only studies that have accessible data would seriously circumscribe the Agency's decision making and the robustness of such decisions.
Commenter (11176) states that a broader approach to ensuring valid findings, especially in complex areas like regulatory environmental science, is needed. The commenter notes that rather than focusing on single studies, evidence experts look at bodies of evidence and seek triangulation of data from a diverse set of research approaches to evaluate consistency of results and enhance confidence in findings above that of any individual study. The commenter states that as a regulatory Agency, EPA work is constantly under scrutiny, putting its scientific justifications under a microscope with legal challenges an ever-present possibility. The commenter states that as a result, EPA, with the expertise of its scientific and other professional staff and with the consultation of its scientific advisory panels, has an effective and advanced system of scientific checks and considerations that have evolved and improved over the decades. The commenter suggests that as an example of EPA's rigorous scientific processes, consider the rigor, detail, and transparency of the EPA program for managing the quality of its environmental data. The commenter suggests also considering the EPA Integrated Science Assessments which rate evidence for certain hypotheses on a causal spectrum and carefully consider a body of evidence without undue emphasis on any one study. The commenter states that this rule reverses that progress by having it revert to a more narrow check and also freezes EPA's scientific validation process around this narrow check. The commenter concludes that the rule should be withdrawn so that EPA scientists can use the most recent and rigorous methods for determining the best science to use in supporting the EPA's mission to protect the environment and the health of the U.S. population.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions.
The Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. The EPA will continue to use established guidelines for identifying and integrating evidence.
The EPA continues to believe that the opportunity to independently validate the pivotal science that the EPA relies upon is important in furthering scientific understanding and the Agency's mission. A presenter in a 2016 NAS workshop on Principles and Obstacles for Sharing Data from Environmental Health Research stated more directly that "for environmental policy making to be legitimate, the scientific reasoning behind a given decision -- including the data supporting it -- must be transparent." When data are widely available, researchers can validate research results and help identify and correct unintended errors, as well as reanalyze the data for new and different purposes, examine novel questions, provide new scientific insights, and improve model development.
Comment: Commenter (11103) recommends the EPA invest in the research and development of open-source, usable tools, and infrastructure that produce statistically sound, reproducible, and verifiable results. The commenter suggests the EPA consider the entire body of cumulative evidence on any given subject and implement policies that aid in overcoming technological barriers to reproducibility rather than excluding studies. The commenter states that scientific research today is largely computationally heavy and follows a workflow, such as the intricate numerical modeling of climate and weather. The commenter states that scientists need standards-based workflow systems that better document, track, and detail their research and enable them to map the complex process from data to results. The commenter states that this will make it easier for other researchers to independently verify any claims. The commenter states that, instead of enforcing strict transparency rules on the underlying data -- which cannot always be transparent either to protect the privacy of identifiable data or to protect the intellectual property of proprietary data -- the EPA should encourage transparency through initiatives aimed at methodology.
Commenter (11103) supports the recommendation from the National Academic of Sciences, Engineering, and Medicine (NASEM) that the National Science Foundation (NSF) consider creating code and data repositories for the long-term archiving of digital artifacts and believe the EPA should endorse this. The commenter states that not being able to access legacy data in a usable format presents an obstacle for reproducibility. The commenter states that what is needed are data archives that provide long term stewardship such as the U.S Department of Energy's (DOE) Environmental Systems Science Data Infrastructure for a Virtual Ecosystem (ESSDIVE). The commenter states that this data repository stores, expands access to, and improves usability of data from DOE's research in terrestrial and subsurface environments. The commenter states that researchers in all fields of environmental studies could benefit from such resources and EPA could benefit from the increased openness of their data.
Response: This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenter (10945) asserts that the proposal would fail to assure that the best possible science is available for regulatory decision making. The commenter suggests that the American Economic Association's Data and Code Availability Policy demonstrates an alternative that assures scientific credibility without restricting good science because it is conducted with data that cannot, for proprietary, statistical, or legislative purposes, be widely distributed. The commenter states that the importance of sharing data (and computational instructions, "code") for the purpose of transparency and reproducibility of science is paramount to the American Economic Association (AEA) and for science in general. The commenter states that the AEA's Data and Code Availability Policy emphasizes this importance, but also exhibits flexibility by explicitly recognizing the scientific credibility of research performed with restricted data, and prescribing alternative approaches to replicability in that case:
      If data are subject to any access restriction that prevents authors from depositing the files in an openly accessible data repository, additional information is required. Authors shall provide detailed information on how, where, and under what conditions an independent researcher can access the data. This information shall be provided to the editors upon submission... and the AEA Data Editor will verify the information, and may contact the data providers identified by the authors. 
Commenter (10945) states that, if data contain confidential information - identifiable person or firm/establishment information  -  de-identification and anonymization are required both for ethical as well as legal reasons Health Insurance Portability and Accountability Act (HIPAA). The commenter states that such data can and regularly are made available at archives that support relevant access controls. The commenter states that a prime example is the government's own Federal Statistical Research Data Center (FSRDC) system, which currently houses confidential data from five agencies, is used at any time by several hundred researchers (as of the end of 2018, 760 researchers were active), on hundreds of approved projects (as of the end of 2018, there were 315 active projects). The commenter states that using this highly restricted, confidential data on businesses, people, their health, their taxes, researchers each year publish in top journals. The commenter states that, in 2018, the latest available numbers show at least 82 publications, most of those in peer-reviewed journals, often in the top journals in economics and other social disciplines. The commenter states that many of those researchers and those publications use the same underlying data repeatedly, demonstrating that an active scientific discussion can happen in the presence of confidential data, in an ethical and privacy-preserving fashion. The commenter states that, while data is restricted from publication, it is available to a very large community of researchers under proper safeguards. The commenter states that the FSRDC system is but one of many systematic, objective, and non-discriminatory systems of accessing confidential health, business, and demographic data in the United States. The commenter states that the Foundations for Evidence-Based Policymaking Act of 2018 (P.L. 115-435) explicitly calls for making data available "while protecting such assets from inappropriate access and use", and mandates OMB to "establish a process" by which a large and diverse cadre of stakeholders can access the data under those conditions, with explicit mention of researchers.
Commenter (10945) states that, at the AEA, if access to the data used is restricted, the Editor needs to be alerted to such situations at the time of submission. The commenter states that, in all such cases, the AEA policy still requests provision of the code, and a description of how the data could be accessed or reproduced by another researcher. The commenter states that the Data Editor may test the access protocol prior to acceptance, and may ask for support by third parties in running the provided code. The commenter states that it is likely that this will not always be possible, and the ultimate decision about whether to go forward with the submission, and whether to accept an article that uses restricted data rests with the journal's Editor or Co-Editor. The commenter states that the AEA has been successful in asking independent third-parties with access to the same data to conduct verification and reproducibility tests, subject to a stringent protocol, and to the satisfaction of the Data Editor. The commenter states that the AEA is firmly of the opinion that reliable, verifiable, high-quality science can be conducted when using confidential data. The commenter states that reliable, reproducible, credible science can be done and has long been done, using non-public data. The commenter states that, because non-public data will remain a feature over the future, for ethical and competitive reasons, policy decisions should be based on the conclusions of knowledge arising from its use. The commenter states that the proposal's rigid and restrictive interpretation of transparency  -  that anyone in the public requesting the data can obtain it  -  is unnecessary when restricted access facilities of trusted intermediaries and associated protocols are in place, widely used, and scientifically acceptable. The commenter states that the EPA's policy needs to recognize and accommodate these contemporary approaches.
Response: The EPA has finalized the approach that gives greater consideration to pivotal science whose underlying data or models are publicly available or available through restricted access. Restricted or tiered access in this rule means that the underlying data or models are be available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
Comment: Commenter (11176) suggests that, for the EPA to continue improving access to its data and the data it uses, it may be helpful to consult the insightful, constructive and comprehensive 2019 Bipartisan Policy Center technical paper, Meaningful Transparency at EPA: A Framework for Rationalizing Approaches to Promote Open Science and Data Sharing for Evidence-Based Policymaking.
Response: The EPA supports making data and model publicly accessible to allow for reanalysis for independent validation. EPA is also committed to protecting CBI and PII according to all existing rules and regulations. The Agency supports the use of restricted or tiered access for data and models that have CBI, proprietary data, or PII. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule.
Comment: Commenter (12731) states that study evaluation criteria used in leading systematic review frameworks that have been developed for environmental health, such as the University of California San Francisco Navigation Guide and the U.S. Toxicology Program Office of Health Assessment and Training systematic review handbook, have been adapted from prominent systematic review methods in medicine, namely the Cochrane Reviews. The commenter states that Cochrane Review study evaluation criteria have been developed and refined over decades, and are supported by empirical evidence and experience in application. The commenter states that, in contrast, the EPA's failure to provide any factual support for assigning the availability of underlying data controlling weight in its assessment of studies renders the proposal arbitrary.
Response: The EPA disagrees with the commenters statement that the Agency has failed to "provide any factual support . . ." The SNPRM asked for comment on this issue and the final rule addresses these concerns.
 The EPA agrees that it is important not to categorically exclude any study because the data underlying a study at the stage required by this rulemaking may not be available for independent validation. Such an approach could have resulted in the systematic exclusion of valid studies. Therefore, the EPA is not finalizing the primary proposal in the 2020 SNPRM that would have categorically required that for studies to be considered pivotal science the underlying data would need to be available for independent validation. However, given that transparency is an important aspect of EPA's regulatory actions and assessments, it should be an important consideration in how the Agency considers pivotal science. As described in 40 CFR 30.5 of the final rule, the EPA will rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. Where there is convincing and well substantiated evidence to support a relationship between exposure and effect, the EPA will identify those studies --  based on the exposure situation being addressed, the quality of the studies, the reporting adequacy, and the relevance of the endpoints -- that would inform a dose-response assessment for those effect endpoints. From that subset, the specific dose-response studies or analyses that drive the requirements or quantitative analyses of an EPA significant regulatory action or influential scientific information will be identified as pivotal science.
In the final rule the EPA also provides additional clarification regarding the other scientific factors that the EPA will consider. Although the EPA is prioritizing transparency in pivotal science, the Agency also recognizes that there will be instances where the underlying dose-response data of pivotal science is unavailable for independent validation. In order to ensure that the Agency maintains a strong scientific basis for its decision-making, the availability of underlying dose-response data should be considered as long as other significant technical considerations can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability, even in the absence of maximum transparency. Though not exhaustive or exclusive, the EPA has identified several factors that balance some of the important technical considerations the EPA will consider in addition to data availability and that are particularly relevant to the stage of the analysis where dose-response data are used. The EPA will thus consider the following factors:
 the quality of the study relative to other studies for which the dose-response data are available; 
 the extent to which there are other studies for which the dose-response data are available;
 the sensitivity of the conclusions in the significant regulatory action or influential scientific information based on the use of the study;
 the extent to which the study is fit for the purpose or intended use relative to other studies for which the dose-response data are available;
 the use of exposures or doses in a range and duration that is relevant for the intended use and that minimizes the need for extrapolations;
 the extent to which the study is supported by other scientific evidence;
 the extent to which the study accounts for unique scientific considerations;
 the extent to which the study minimizes the use of defaults and assumptions, uses appropriate and strong statistical methods, and includes a robust representation of uncertainty and confidence intervals; and
 the study's consideration of a broad range of parametric dose-response or concentration-response models, a robust set of potential confounding variables, nonparametric models that incorporate fewer assumptions, various threshold models across the dose or exposure range, and models that investigate factors that might account for spatial heterogeneity.
These factors are intended to assist the EPA in determining the consideration to afford to pivotal science with underlying dose-response data that are not available for independent validation. The final rule requirements and the consideration of these factors apply to any data used in characterizing the relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect, regardless of the direction of that effect. Because study quality factors (including soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review) would have already been evaluated at an earlier stage in the assessment process, the studies evaluated with these additional factors are presumed to be of the highest quality available.
 Some of the factors described above are intended to be evaluated for pivotal science with underlying data that are not available for independent validation relative to pivotal science with underlying data that are available for independent validation. For example, when assessing studies, the EPA may determine that greater consideration should be given to a study with underlying data that are unavailable for independent validation when that study is of higher quality compared to a medium-quality study with underlying data that are available for independent validation (factor 1), the conclusions of the significant regulatory action or influential scientific information are or are not highly sensitive to the exclusion of the study for which the underlying data are not available for independent validation (factor 3), the study with data unavailable for independent validation was better fit for the purpose of the EPA assessment (factor 4), or the results of the study for which the underlying data are not available are supported by other scientific evidence, such as mechanistic data (factor 6).
Importantly, these factors are do not apply to other stages in the assessment process. For example, the consideration for exposures that were conducted at more environmentally relevant exposure concentrations (factor 5) does not suggest that epidemiological studies will automatically be given greater weight than laboratory studies. The EPA will continue to use established guidelines for identifying and integrating evidence and will only use the factors in 40 CFR 30.5 studies in evaluation of the data availability requirements of this rule. In addition, not all of these factors will be applicable to all studies or assessments. For example, some pollutants, chemicals, or substances may have unique scientific considerations (factor 7), such as the valence state of a metal compound or endogenous contributions to internal concentrations, that may not be relevant for other pollutants, chemicals, or substances. Therefore, the weight afforded to each factor by the EPA may vary by assessment, and how those factors were considered will be documented in the assessment. If two studies, one with and one without available data and are relatively equal with respect to the study quality factors in 30.5(b), the study where the underlying data is available will be given greater consideration and the weight of the other study will be based on an assessment of the factors in 30.5(d). In this way, the EPA will balance the importance of transparency with the need to maintain a strong scientific basis for its assessments.
This final rule requires the consideration of these factors in assessing pivotal studies for which the dose-response data are not available for independent validation. The EPA may adapt these factors in upcoming statute-specific rulemakings, as appropriate, for significant regulatory actions under the different environmental statutes that the EPA administers. How scientific information is to be considered varies among the different environmental statutes and sometimes within an individual statute. Interpretation of the assessment factors will be tailored to the specific circumstances and the specific environmental statutes.
Comment: Commenter (11490) states that various systems have been devised to rate the quality of scientific evidence, perhaps the most widely accepted of which is the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) system. The commenter states that GRADE assesses the quality of a study based on study design (with randomized controlled trials starting as high quality and observation studies as low quality), and then considers various factors that may lower the quality of evidence (e.g., risk of bias, inconsistency, indirectness, imprecision, publication bias) or increase the quality of evidence (e.g., large effect size, dose-response relationship). The commenter suggests the EPA should rely on these factors, rather than just one aspect -- availability of raw data -- to assess to what extent a study should inform policy decisions. 
Response: The EPA is not relying solely on data availability. Part 30.5 of the final rule states, "When determining the consideration to afford to pivotal science that lacks available data and models, EPA shall consider factors including soundness, applicability and utility, clarity and completeness, uncertainty and variability, and evaluation and review. The Agency will identify the pivotal science that was given lesser consideration and provide a short description of why lesser consideration was given."
Comment: Commenter (10048) states that the systematic review process in place for IRIS and other review processes already includes consideration of the quality of a research study and methods. The commenter states that best professional judgement of data quality should be left to subject matter experts and not assigned solely on the basis of data availability. 
Response: The EPA is not categorically excluding any studies from consideration - even studies where the underlying dose-response data are not available for independent validation - when promulgating final significant regulatory actions or developing influential scientific information. Rather, as discussed in Part 30.5 of the final rule, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. The EPA will continue to use established guidelines for identifying and integrating evidence.
The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions.
Implementation guidelines and statute-specific transparency regulations or programmatic regulations are forthcoming and may be more expansive than the procedural rule. 
Comment: Commenter (10339) recommends that the EPA consider journals published by scientific societies that have established guidelines and standards for the submission of datasets to repositories should be considered trusted sources for EPA scientific reviews. Commenter (10865) contends that the research on which the EPA relies on to make most of its policy determinations is already transparent in most cases given that many scientific journals and research agencies now have policies governing the sharing of data among researchers and with appropriate access by the public at large.
Commenter (12405) states that, as overall justification for the proposed rule, the EPA claims it is following the accepted practice of many science organizations including many scientific journals. The commenter asserts that this is misleading. The commenter states that major journals in the field only require data be made confidentially available to other researchers for the purposes of reproducing or extending analysis. No major journal requires scientists to publish raw data to the public in all cases. 
Response: Many scientific publications have guidelines that require authors to make a data availability or data access statement, which discloses where and under what conditions the underlying study data are available. Yet the EPA cannot solely rely on data availability statements made in published research because initiatives toward greater data sharing and transparency amongst scientific journals and international organizations are still being implemented, are inconsistently enforced, and the true accessibility of data in a public repository is still limited. For example, Christensen et al. (2019) evaluated 1,072 peer-reviewed articles and "found that rates of data availability for empirical articles published after journals adopted data-sharing policies differ widely between journals, from 0 percent to 83 percent, with a mean of 35 percent."
The EPA is finalizing the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
This final rule does not require scientists to publish their "raw data" nor does it regulate any entity outside the EPA. Rather, in this action, the EPA is finalizing a rule of internal agency procedures, including how the Agency will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information.
Comment: Commenter (11400) asserts that independent validation of public health studies is not the only means of determining credibility of studies; the National Academy of Sciences also lists other accepted methods of improving the rigor and credibility of studies, such as peer review, systematic reviews, and meta-analyses. The commenter states that the scientific merit of studies is what should be given greatest consideration, and scientific value should be evaluated by scientists.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions.
The EPA is a leader in the evaluation of human and ecological impacts of environmental pollution and methods to cost-effectively mitigate these impacts. The selection and use of quality, peer-reviewed literature is a critical part of this work. However, the strength of the review conducted as part of the study's acceptance and publication in a scientific journal has received increased attention in the scientific community due to the lack of transparency and consistency, potential for bias, and the difficulty that independent researchers have had in reproducing published results.[,][,][,][,][,] Jamieson et al. (2019) state, "Journals play a key role in ascertaining the quality of the evidence in work submitted for publication through peer review. However, that practice is neither uniform nor free of exploitation. While it can select for trustable results, its existence alone is not sufficient to verify the quality of the evidence used to back claims." Because of the potential impact of significant regulatory actions, independent validation of studies that are pivotal to the EPA's significant regulatory actions and influential scientific information may increase public confidence in the findings and conclusions presented in those studies. As Jamieson et al. (2019) states, publication alone (including the typical peer review necessary for publication) is insufficient to verify a study's quality, and additional confidence could be gained from third-party reanalyses that independently validate the study results.
Finally, as described in the rule and preamble, the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. When characterizing the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect, the EPA will identify pivotal science and give greater consideration to pivotal science for which the underlying dose-response data are available in a manner sufficient for independent validation. Including this review of dose-response data availability for pivotal science is critical to the EPA's progress toward increased transparency and providing increased opportunity for scientific reanalysis and review by independent third parties. This approach will result in significant regulatory actions and influential scientific information that are based on the highest quality, most relevant studies and ensure adequate human health and environmental protections, while maximizing transparency and maintaining consistency with the statutes the EPA administers.
Comment: Commenter (10339) states that the proposal and SNPRM fail to explain how publicly funded, peer-reviewed studies that comply with National Institutes of Health (NIH) data deposition and access policies will be considered. Commenter (10339) urges the EPA to instead adopt policies governing data sharing and access that are consistent with those of other research agencies such as the NIH. Commenter (11408) suggests that rather than making all raw data available, the Agency should instead establish a database that "registers studies and obtains systematic sets of parameters and results", such as the NIH's clinical trials database. 
Response: As described in the rule and preamble, the EPA is not categorically excluding any studies from consideration when promulgating final significant regulatory actions or developing influential scientific information. Rather, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance.
EPA has implemented the public access requirements for EPA researchers, grantees and contractors as required by the February 22, 2013, OSTP memorandum "Increasing Access to the Results of Federally Funded Scientific Research" and the 2016 EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research." EPA researchers, grantees and contractors are required to make peer-reviewed journal articles and the underlying data and models resulting from EPA-funded research accessible to the public. The EPA "Plan to Increase Access to Results of EPA-Funded Scientific Research" requires the data underlying a journal article resulting from research funded by EPA be made publicly accessible. 
However, this final rule, as stated in Part 30.3, applies "to science that serves as the basis for informing a significant regulatory action or influential scientific information, as well as to dose-response data underlying pivotal science, regardless of the source of funding or identity of the party conducting the science."
Comment: Commenter (11890) states that the Science Advisory Board (SAB) identified reasonable approaches the EPA can take to improve transparency without violating privacy rights of study participants under federal and state law. The commenter states that these included use of data sharing agreements, use of trusted third parties (e.g., Federal Statistical Research Data Centers, Health Effects Institute (HEI)), and working in concert with the National Academies of Science and other federal partners to adopt evolving but achievable transparency standards in health research going forward. The commenter states that all these are preferable to the current proposal.
Response: The EPA notes that this final requires EPA to make underlying public data available, if EPA chooses to do so, in a fashion that is consistent with the law and retains required protections. The EPA has considered the SAB's September 30, 2019 comments and finalized the approach that gives greater consideration to pivotal science whose underlying dose-response data are publicly available or available through restricted access. Restricted or tiered access in this final rule means that the underlying dose-response data are available through a data sharing mechanism, such as through an agreement with the originating author or institution, access to a refined or redacted dataset that anonymizes the more sensitive portions of the analyzable dataset, a restricted access data repository or secure data enclave, or some other mechanism (e.g., Data Use Agreements) that allows a qualified subject matter expert access to enough data to support independent validation while still protecting sensitive information.
21.2.1.4 Consultation with the Scientific Community
21.2.1.4.1 Importance of Consultation
Comment: Commenters (10041, 11490, 12472) highlight the importance of consultation in the decision-making process, including the development of this rule. Specific comments include:
Commenter (10041) provides that in July 2018, they sent a comprehensive comment letter in response to the previous version of this proposed rule. In the letter the commenter stated, that the "best available evidence" should be considered when developing regulations specifically aimed at protecting human health. The commenter asserts that the current COVID-19 pandemic is showing us in real time the necessity of scientific expertise in decision-making.
Commenter (11490) asserts that in sum, open data is a positive trend in science and EPA is right in seeking to support and encourage this trend, but this proposed rule serves to undermine science-based policymaking processes rather than to elevate the worthy cause of increased transparency. The commenter notes that in this SNPRM, EPA solicited input on how to ensure that, over time, more of the data and models underlying the science that informs significant regulatory decisions and influential scientific information are available to the public for independent validation. The commenter contends this is accomplished by working together with stakeholders in the scientific community to build infrastructure for data sharing, not by ignoring all science -- archival and forthcoming --  that has not achieved these new and ambitious (and in many cases, unreachable) transparency aspirations.
Commenter (12472) states that when finalizing this proposed rulemaking, the EPA should consider comments by states, scientific advisors, and other experts so that access to data and models used by the EPA is ensured while preserving CBI, PII, and proprietary data.
Response: In development of this final rule, EPA considered the public comments on the 2018 NPRM, the 2020 SNPRM, and the advice of the SAB.
21.2.1.4.2 Utilization of the EPA's SAB
Comment: Commenters (11183, 11479, 11894, 11919, 12427, 12464, 12720, 12727, 13788) contend that the SAB was not properly consulted in the development of the proposed rulemaking. Specific concerns include:
Commenter (11183) states that given the importance of this rule, EPA should have also consulted with scientific experts via EPA's SAB, which issued a report earlier this year finding "key considerations that should inform the proposed rule have been omitted from the proposal or presented without analysis." The commenter notes that even though they have identified important flaws in the proposal the SAB has also not been consulted.
Commenter (11402) notes that with the original rule, this proposed supplemental rule was not reviewed by the EPA's own SAB before its release for public comment. The commenter states that a minimum, the Agency should consult with its own scientific experts regarding the proposed supplemental rule's potential short and long-term human and environmental health effects.
Commenter (11479) provides that SAB wrote a memo recommending that the SAB review the merits of the rule because "it deals with a myriad of scientific issues for which the Agency should seek expert advice from the Science Advisory Board." According to the commenter, some of the areas flagged by the memo include the lack of assessment of the impact of data restrictions on current or future rulemaking, the fact that EPA did not solicit input from the scientific community, and that the rule does not acknowledge the strides in transparency that have already been made by the epidemiologic science community. The commenter notes that after a near-unanimous vote concurring with the memo, the SAB wrote in a June 28, 2018 letter to former administrator Scott Pruitt that "[the] SAB urges the Agency to ... request, receive, and review scientific advice from the SAB before revising the proposed rule." The commenter states that instead of tasking the SAB with review of the full rule, Administrator Wheeler waited nearly a year and finally asked that the SAB look into a narrow slice of the rule, CBI and PII in April 2019. The commenter contends, because the SAB was concerned with many other aspects of the rule and what its implementation would mean for EPA's ability to use the best available science in decision-making, it also reviewed the full rule as a body.
Commenter (11894) states that 42 U.S.C. § 4365(b) requires EPA to submit this SNPRM to EPA's SAB for review at the time it is sent to OMB for inter-agency review. The commenter notes that EPA ignored this requirement twice over: for both the initial proposal and the SNPRM. The commenter asserts that the error is especially troubling given the Agency's pledge to provide the SAB with ample notice, an evidently hollow pledge made after the Agency signally failed to comply with these statutory requirements with respect to the initial proposal. The commenter provides that, deprived of proper notice, the SAB never met publicly regarding the supplemental proposal, and necessarily excluded the public from its undocumented deliberations. In spite of these limitations, the commenter recommends the Agency should take heed of the SAB's warning: 
      There appears to be consistency among analyses of how to address transparency that are orthogonal to the Proposed Rule. There is minimal justification provided in the Proposed Rule for why existing procedures and norms utilized across the U.S. scientific community, including the federal government, are inadequate, and how the Proposed Rule will improve transparency and the scientific integrity of the regulatory outcomes in an effective and efficient manner. It is plausible that in some situations, the Proposed Rule will decrease efficiency and reduce scientific integrity, determining if in fact that will be the case requires a thorough and thoughtful examination that is currently absent in the Proposed Rule. Moving forward with altered transparency requirements beyond those already in use, in the absence of such a robust analysis, risks serious and perverse outcomes.
Commenter (12427) queries why EPA has failed to involve EPA career scientific staff in the development of the proposed rule and its supplemental notice and why EPA has failed to meaningfully engage external experts via existing EPA scientific Federal advisory committees.
Commenter (12464) asserts that EPA's consultation with the SAB has also been inadequate, in violation of Environmental Research, Development and Demonstration Authorization Act (ERDDAA). The commenter notes that EPA is required to consult with the SAB on the Proposal, a fact that EPA has now acknowledged. The commenter states that even though the SAB has now been able to offer a belated report on the Proposal -- something it was not able to do before the initial notice in 2018 -- EPA's consultation with the SAB is nevertheless inadequate. First, the commenter contends that EPA has not allowed adequate time for the SAB to review the SNPRM. According to the commenter, the SAB last met as a group to discuss the Proposal in a public teleconference on January 21, 2020, when it reviewed a draft report that commented on the initial version of the proposal from 2018. The commenter notes that the SNPRM was not issued until March 3, 2020, after this meeting. As such, the commenter provides that, although the SAB subsequently issued its final report on April 24, 2020, it had no opportunity to hold a meeting to discuss the supplemental proposal. The commenter states that therefore, it is not clear that the SAB was genuinely able to consult as a body on the contents of the supplemental proposal. In addition, the commenter notes that even the SAB Report itself indicates only that it addresses certain "aspects of the supplemental notice." In other words, the commenter provides that the SAB indirectly acknowledges that it was not able to provide input on all parts of the supplemental proposal. Additionally, the commenter contends that the multiple gaps and ambiguities in the Proposal, as described above, also made it impossible for the SAB to weigh in adequately. The commenter notes that the SAB itself indicated, "[g]iven the relatively skeletal nature of the Proposed Rule, it is not possible to define the implications of the rule with confidence."
Commenter (12720) states that in a major deviation from standard process under ERDDAA, EPA did not seek consultation from the SAB on the full 2018 rule proposal or supplement. According to the commenter, the SNPRM does not address numerous critical issues identified by the SAB including:
  the purpose of the rule;
  the costs and resources required to implement the proposal;
  the recommendation that application should not be retroactive;
  the need for tailored approaches to different types of data;
  and the protection of CBI and PII.
The commenter (12720) asserts that EPA did not consult with the SAB prior to publication of the original rule in 2018 nor the current supplement, despite the SAB's urging in June 2018 that EPA "request, receive, and review scientific advice from the SAB before revising the proposed rule." The commenter contends that this failure to consult is emblematic of a larger disregard for SAB input at EPA. The commenter also notes that to date, EPA has only asked the SAB for input on one issue related to this proposal-existing mechanisms for secure access to CBI and PII. The commenter provides that further, even on this narrow issue, EPA did not seek a consensus report but instead asked only for individual comments from members. Additionally, the commenter contends that because of the significant scientific and technical issues raised in the rule, in June 2019 SAB identified the rule as a planned action that merited its review and elected to review the full proposed rule beyond EPA's narrow request on CBI and PII.
Overall, the commenter (12720) contends that EPA's SAB consultation on the transparency proposal and supplement is notable in its departure from standard ERDDAA process in that:
  EPA did not request SAB review of the full original rule proposal or supplement; and
  EPA did not request a consensus SAB review.
Furthermore, the commenter (12720) states that in response to its request for consultation on existing mechanisms for secure access to CBI and PII, EPA received comments from 22 individual SAB members in Sept 2019. Of these, seven (30%) noted serious concerns with this process. The commenter asserts that these comments raise significant concerns about the consultation process for the original proposal, yet EPA made no changes in the process with this supplement. The commenter notes that EPA has not requested SAB review on the supplement. Furthermore, the commenter provides that the SAB's reports, received by EPA in September 2019 and October 2019, identified critical issues which the supplemental proposal does not address.
Commenter (12727) notes that as with the original proposal, EPA failed to submit the SNPRM to the SAB. The commenter states that this failure is particularly troubling given that the SAB requested review of the original proposal, noting that "the precise design of the rule appears to have been developed without a public process for soliciting input from the scientific community." According to the commenter, the failure to timely consult with the SAB is contrary to statute. The commenter provides that EPA must submit its Proposal to the SAB pursuant to the requirements of the Environmental Research, Development, and Demonstration ERDDAA. Additionally, the commenter states that ERDDAA requires the Administrator to submit to the SAB any "proposed criteria document, standard, limitation, or regulation, together with relevant scientific and technical information in the possession of the [EPA] on which the proposed action is based" at the time it provides that proposal to another Agency of the government for formal review. The commenter contends that not only does EPA's repeated disregard of ERDDAA requirements evince troubling contempt for congressionally mandated procedures; it is also consequential. The commenter notes that although the Final SAB Report mentions the SNPRM, it does so only fleetingly. Moreover, the commenter provides that there was no opportunity for actual SAB discussion of the Supplemental Notice, and there was consequently no public input into the SAB process, since there was no process -- be it a public meeting or any other type of face-to-face deliberation of the SAB. According to the commenter, the SAB pre-proposal review requirement in fact exists to forestall proposals as deficient as the SNPRM. As such, the commenter asserts that EPA's arbitrary failure to adhere to its legal obligation under ERDDAA renders the Supplemental Notice unlawful.
Commenter (13788) states that while the SAB conducted a review of the 2018 Proposal that includes some discussion of the SNPRM, the board did not receive the text in time to have a meeting to fully deliberate on the SNPRM. The commenter asserts that substantive deliberations for a review of the SNPRM would have required an open meeting under the Federal Advisory Committee Act. Also, the commenter notes that under ERDDAA, the Administrator "shall" make any regulation under the CAA available to the SAB when it is provided to any other Agency for formal review and comment. The commenter contends that to the extent this proposal is meant to impact any CAA rule outcomes, EPA therefore was required to provide the SAB information when the SNPRM was sent to the OMB but it did not.
Response: The EPA has consulted with and reviewed the SAB advice three different times during the development of this final rule. 
 In 2018, as part of its statutory duties, the SAB considered the science supporting the proposed action. The SAB met once and issued a final report on April 28, 2018.
 In 2019, at the request of the Administrator and EPA's Office of Research and Development the SAP discussed existing mechanisms for secure access to PII and CBI as discussed in the proposed rule. The SAB met once and issued a final report on September 30, 2019
 In 2019/2020, as part of its statutory duties, the SAB reviewed the science supporting the proposed rule. The SAB met four times and issued a final report on April 24, 2020.
Comment: Commenter (12465) notes the recently completed "SAB Consideration of the Scientific and Technical Basis of EPA's Proposed Rule Titled Strengthening Transparency in Regulatory Science" and that the recommendations therein should be addressed before proceeding with the rule making.
Response: In development of this final rule, EPA considered the public comments on the 2018 NPRM, the 2020 SNPRM, and the advice of the SAB. Numerous scientific organizations, health organizations, and other interested stakeholders provided comment on the NPRM and SNPRM during the public comment periods.
The EPA received comments from the SAB on the 2018 NPRM and 2020 SNPRM on April 24, 2020. The EPA considered these comments in development of the final rule. In brief, the SAB consensus was that it supported the precept of enhancing public access to scientific data and analytical methods to ensure scientific integrity, consistency, and robust analysis and their consensus recommendations were that the EPA develop additional policy and/or guidance documents; further describe how pivotal science would be selected, made available, and peer reviewed; consider establishing an office on data sharing with the support of a peer review panel; provide greater clarity in the definitions of key terms, including data, independent validation, and publicly available; develop specific criteria for the Administrator's exemptions under the rule; consider applying rule requirements only to information developed after the effective date of the rule; seeking input from experts to identify best practices to ensure efficiency, utility, and protection of data that are made available; and consider funding reanalysis work. 
As noted in the preamble for the final rule, this rule sets the overarching structure and principles for transparency of pivotal science in significant regulatory actions and influential scientific information and technical considerations, including those noted by the SAB, could be addressed in subsequent implementation guidelines and/or statute-specific rulemakings. Further, in response to comments from SAB and the public, the EPA clarified and modified several key definitions, including data, independent validation, and publicly available, in the final rule. The EPA also provided criteria for the Administrator's exemption in 40 CFR 30.7 of the final rule. The EPA would like to clarify that this rule does not direct or require any outside entity or the EPA to establish data sharing mechanisms and does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science. Comments related to making data available and future funding opportunities are not within the scope of this rulemaking. Finally, the EPA is maintaining that the rule provisions apply equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. However, in consideration of instances where data cannot be made available, the EPA amended the rule to provide technical considerations (40 CFR 30.5(d)) that can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability in the absence of maximum transparency and included an exemption criterion -- that the development of the dose-response data was completed or updated before the effective date of the rule -- to 40 CFR 30.7 to account for the concerns raised by the SAB and other public commenters.
Comment: Commenter (12720) states that the SAB recommends that EPA consult with library science and data management experts, as well as consider establishing a data sharing office to assist with rule implementation. The supplemental proposal makes no mention of cost or resource issues, leaving the SAB's concerns of unreasonable burdens on researchers, and subsequent exclusion of their data, unaddressed. The commenter contends that EPA did not consult with data management or library science experts, and in fact to date has not engaged with the research, scientific or medical community for input on the rule or supplement. 
Furthermore, the commenter (12720) provides that the SAB letter notes that "[a]ssumptions are often made about (a) error distributions for exposure estimates (most commonly, that they can be ignored); (b) model specification errors and uncertainties; and (c) causal interpretations of modeling results. These assumptions should be explicitly stated, and results of tests presented. The commenter states that epidemiologists (e.g., Sander Greenland), statisticians, and risk analysts have written at length over several decades about how to test, validate, and document assumptions and methods for dose-response modeling and uncertainty and variability characterization, and these modern methods should be applied to make the factual and assumption-based foundations of pivotal studies as clear as possible. However, the SAB finds that there are scientific and technical challenges to be overcome and provides suggestions to implement this requirement." The commenter contends that again, the SAB confines its comment to environmental epidemiology studies, despite the significant expansion of scope within the Supplemental Proposal. According to the commenter, no consideration for other types of "data and models" outside of that field is made in the letter, reflective of EPA's Supplemental Proposal that does not substantively engage with the scope and complexity of scientific information that would be ensnared by the SNPRM. The commenter provides that the "scientific and technical challenges" that are highlighted here are fundamental to the Supplemental Proposal's impracticability and arbitrary nature, especially in light of the fact that EPA has not offered any convincing rationale for the Supplemental Proposal nor the original Proposal. The commenter asserts that as the SAB notes, the specific assumptions and methods utilized in high quality studies are already included in published, peer-reviewed studies. The commenter contends that EPA has not given any rationale for this requirement in the proposal, nor has it identified any systematic deficiency in the peer-reviewed literature that prevents a reasonable ascertainment of study assumptions and methods.
Response: In development of this final rule, EPA considered the public comments on the 2018 NPRM, the 2020 SNPRM, and the advice of the SAB. Numerous scientific organizations, health organizations, and other interested stakeholders provided comment on the NPRM and SNPRM during the public comment periods.
The EPA received comments from the SAB on the 2018 NPRM and 2020 SNPRM on April 24, 2020. The EPA considered these comments in development of the final rule. In brief, the SAB consensus was that it supported the precept of enhancing public access to scientific data and analytical methods to ensure scientific integrity, consistency, and robust analysis and their consensus recommendations were that the EPA develop additional policy and/or guidance documents; further describe how pivotal science would be selected, made available, and peer reviewed; consider establishing an office on data sharing with the support of a peer review panel; provide greater clarity in the definitions of key terms, including data, independent validation, and publicly available; develop specific criteria for the Administrator's exemptions under the rule; consider applying rule requirements only to information developed after the effective date of the rule; seeking input from experts to identify best practices to ensure efficiency, utility, and protection of data that are made available; and consider funding reanalysis work. 
As noted in the preamble for the final rule, this rule sets the overarching structure and principles for transparency of pivotal science in significant regulatory actions and influential scientific information and technical considerations, including those noted by the SAB, could be addressed in subsequent implementation guidelines and/or statute-specific rulemakings. Further, in response to comments from SAB and the public, the EPA clarified and modified several key definitions, including data, independent validation, and publicly available, in the final rule. The EPA also provided criteria for the Administrator's exemption in 40 CFR 30.7 of the final rule. The EPA would like to clarify that this rule does not direct or require any outside entity or the EPA to establish data sharing mechanisms and does not require the EPA to collect, store, or publicly disseminate dose-response data underlying pivotal science. Comments related to making data available and future funding opportunities are not within the scope of this rulemaking. Finally, the EPA is maintaining that the rule provisions apply equally to all dose-response data underlying studies used as pivotal science, regardless of when the study or the data was created. Scientific transparency is important regardless of the age of the study or the dose-response data. However, in consideration of instances where data cannot be made available, the EPA amended the rule to provide technical considerations (40 CFR 30.5(d)) that can provide some level of certainty or confirmation of a study's conclusions, importance, and applicability in the absence of maximum transparency and included an exemption criterion -- that the development of the dose-response data was completed or updated before the effective date of the rule -- to 40 CFR 30.7 to account for the concerns raised by the SAB and other public commenters.
This final rule does not require the EPA to disclose or host data, but to determine if dose-response data are available and to give greater consideration to those studies for which such data are available. Hence, this rule does not impose costs on the EPA or any other party to make data available, including costs to ensure that PII and CBI are not disclosed. The Agency may opt, at its discretion, to incur the costs associated with making data available when it is in the public interest to do so, but that will be decided on a case-by-case basis and is not a requirement of the final rule. 
As Jamieson et al. (2019) state, "Journals play a key role in ascertaining the quality of the evidence in work submitted for publication through peer review. However, that practice is neither uniform nor free of exploitation. While it can select for trustable results, its existence alone is not sufficient to verify the quality of the evidence used to back claims." Because of the potential impact of significant regulatory actions, independent validation of studies that are pivotal to the EPA's significant regulatory actions and influential scientific information may increase public confidence in the findings and conclusions presented in those studies. As Jamieson et al. (2019) states, publication alone (including the typical peer review necessary for publication) is insufficient to verify a study's quality, and additional confidence could be gained from third-party reanalyses that independently validate the study results.
Comment: Commenter (11919) refers to an April 24 letter to the Administrator in which the SAB noted its "concerns about the scientific and technical challenges of implementing some requirements of the Proposed Rule." The commenter states that the Board listed several factors that it says the EPA needs to resolve in the proposed rule to protect privacy, including addressing the legal, ethical, professional, and financial reasons for research confidentiality. The commenter encourages the EPA to address the Board's input before finalizing this rule given that the Board is charged with providing advice and recommendations on "the adequacy and scientific basis of any proposed criteria document, standard, limitation, or regulation."
Response:: In development of this final rule, EPA considered the public comments on the 2018 NPRM, the 2020 SNPRM, and the advice of the SAB. Numerous scientific organizations, health organizations, and other interested stakeholders provided comment on the NPRM and SNPRM during the public comment periods.
21.2.1.4.3 Consultation with Major Research Organizations/Scientists
Comment: Commenters (10974, 11183, 11479, 11890, 12465, 12645) contend that EPA has inadequately consulting with various major research organizations and scientific societies, including the NIH, American Association for the Advancement of Science), the National Academies of Sciences, Engineering, and Medicine (NASEM), and several premier scientific journals. Specific concerns include:
Commenter (10974) contends that the supplemental proposal does not integrate, nor does it provide solutions for or improvements to, the current work on data reproducibility within the scientific community. The commenter provides that, similar to how the supplemental proposal does not address an extant need for rulemaking regarding data transparency, the supplemental proposal (and 2018 proposal) do not support nor adequately address the existing efforts by all types of institutions to improve data transparency and availability issues.
Commenter (12645) suggests that, before potentially excluding from consideration a large percent of relevant scientific literature, the EPA could have surveyed these non-US scientists to find out whether they would have cooperated with their new rule or its extension in the SNPRM. The commenter states that they could have done a far more intensive analysis of the sources of the studies that they cite in existing regulations, or have responded to my and others previously expressed concerns. For example, the commenter notes that the International Society of Environmental Epidemiology has made this point very strongly. EPA's failure to do any such analysis, or to submit consideration of this new rule to the NAS or even their own SAB, is a dereliction of their duty to the American public and to Congressional mandates for basing environmental regulations on the best available science. 
Commenter (11183) states that EPA has still not consulted with critical stakeholders, including scientists, health professionals and communities. In the 2 years since the 2018 Proposal, the commenter states that EPA still has not consulted with the premier federal Agency for health research  -  the NIH  -  "the nation's medical research agency" "the largest source of funding for medical research in the world", the American Association for the Advancement of Science (AAAS), the world's largest multidisciplinary scientific society and the largest scientific organization in the US, and also a leading publisher of cutting-edge research through its Science family of journals, or the National Academies of Science or Medicine (NAS or NASM), "private, nonprofit institutions that work outside of government to provide objective independent, evidence-based scientific advice on matters of science, technology, and health". In particular, the commenter contends that EPA should have consulted with the NIH as this rule has implications for the research that is funded by NIH, numerous studies considered by EPA in all of their regulations that are funded by NIH and NIH has stringent rules about participant confidentiality.
Commenter (11479) contends that it is clear from the number of questions posed by the Agency in its 2018 Proposal and SNPRM about how the rule might be implemented, and from the outcry and alarm from the scientific community, that the Agency did not consult with scientists or respected national scientist organizations as it crafted the rule. The commenter also asserts that EPA incorrectly claims that the rule follows accepted practices on open science that major science journals and the NAS have adopted. The commenter notes that the editors-in-chief of Science, Nature, PLoS, Proceedings of the National Academy of Sciences (PNAS), Cell, and The Lancet issued a statement in 2018 and again in 2019 that this policy would exclude important studies from consideration in the rulemaking process, thus adversely impacting the decision-making process. Furthermore, the commenter contends that among those not consulted in the crafting of this rule was NASEM, which has held several committee meetings and carried out a series of reports detailing how scientific literature can be evaluated transparently without the full disclosure of underlying datasets. The commenter notes that in a comment on the rule, NASEM urged EPA to seek objective and expert guidance on the rule and its implementation and offered itself as an independent review body. The commenter provides that the Bipartisan Policy Center (BPC) was also cited in EPA's proposed rulemaking and had to clarify in a comment to the Agency that "the proposed rule is not consistent" with its report on the use of science in policymaking that EPA cited in "substance or intent." The commenter states that while BPC supports enhanced transparency, "the report never suggested excluding studies from consideration in developing regulation if data from those studies were not publicly available."
Commenter (11890) contends that EPA should first consult with NAS and other independent scientists and science organizations in a meaningful manner before deciding whether any changes to EPA's current use of scientific evidence is needed. Additionally, the commenter notes that this would include a full evaluation by the EPA SAB once a complete proposal has been developed.
Commenter (12546) contends that throughout this process, EPA has not consulted with critical stakeholders, including scientists, health professional and communities. The commenter has grave concerns with the fact that since the 2018 proposal, EPA still has not consulted with the stakeholders or organizations facing serious, long-term implications from this rule: scientists; medical researchers and health professionals; universities; hospitals; peer-reviewed journals/publishers; and communities who participate in research studies. Further, the commenter notes that there are numerous studies considered by EPA that are funded by NIH, which has stringent rules around participant confidentiality. Therefore, the commenter asserts that EPA should have consulted with the NIH. Additionally, the commenter states that given the importance of this rule, EPA should have also consulted with scientific experts via the NASEM. The commenter states that if EPA has consulted with these critical stakeholders, it should publicly release a report akin to an Interagency Scientific Assessment regarding how it addressed any comments or concerns around the impacts of this rule.
Commenter (12645) notes that EPA's heavy reliance on the NAS is ironic in that the current EPA leadership has steadfastly refused to consult the Academies on any of the issues related to its attempts to change the scientific processes at EPA. The commenter contends that similarly, in violation of congressional mandates, it has not involved its external scientific committees in considering the rationale for any of its major proposed changes in EPA's scientific processes.
Response: In development of this final rule, EPA considered the public comments on the 2018 NPRM, the 2020 SNPRM, and the advice of the SAB. Numerous scientific organizations, health organizations, and other interested stakeholders provided comment on the NPRM and SNPRM during the public comment periods. 
21.2.1.4.4 EPA Should Seek Comment from Various Advisory Panels
Comment: Commenter (12464) contends that the SNPRM is inconsistent with the requirements of Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA). The commenter notes that the SNPRM continues to rely on the rulemaking provision of FIFRA as a source of authority for the Proposal. The commenter provides that this provision mandates that EPA seek comments from the Secretary of Agriculture and the FIFRA Scientific Advisory Panel on draft regulations, yet there is no evidence in the rulemaking docket that EPA has sought such input.
Response: In this action, the EPA is finalizing a rule of internal agency procedures, including how the Agency will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information for independent validation. The EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers (including FIFRA). If implementing this procedural rule would result in conflicts with existing environmental statutes, the procedural rule will yield to the existing statutes. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. Some of these subsequent actions will be substantive rules issued under the associated environmental statutes and will be subject to any applicable procedural and analytical requirements in those statutes and judicial review.
Comment: Commenter (12727) states that it was arbitrary under the CAA not to consult with the Clean Air Scientific Advisory Committee (CASAC). The commenter notes that CASAC is charged with reviewing NAAQS and recommending any new standards. The commenter provides that in turn, the NAAQS are to be based on criteria that "accurately reflect the latest scientific knowledge useful in indicating the kind and extent of all identifiable effects on public health or welfare." To this end, the commenter asserts that CASAC's task of reviewing the NAAQS would be complicated and perhaps rendered impossible by the SNPRM. According to the commenter, CASAC could be required to recommend a change to the NAAQS based on their evaluation of valid research that EPA now seems to find objectionable, which suggests that this enterprise is not what Congress intended. Further, the commenter notes that because the SNPRM would apply to influential scientific information, such as integrated science assessments prepared by Agency staff, EPA might itself deprive CASAC of the information it needs to fulfill its statutory duty. The commenter contends that EPA should have considered these complexities in its rulemaking and consulted with CASAC on its implications for CASAC's duties. For these reasons as well, the commenter states that finalizing the proposed rule would be arbitrary and unlawful under the CAA.
Response: In this action, the EPA is finalizing a rule of internal agency procedures, including how the Agency will consider the availability of dose-response data underlying pivotal science used in its significant regulatory actions and influential scientific information for independent validation. The EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers (including the CAA). If implementing this procedural rule would result in conflicts with existing environmental statutes, the procedural rule will yield to the existing statutes. The EPA will undertake such efforts in forthcoming actions, which will be either statute-specific science transparency regulations or programmatic regulations implementing this procedural rule. Some of these subsequent actions will be substantive rules issued under the associated environmental statutes and will be subject to any applicable procedural and analytical requirements in those statutes and judicial review. 
Comment: Commenter (10974) states that EPA's revisions do not clarify how this proposed rule enhances the public's ability to understand or meaningfully participate in the regulatory process. The commenter suggests that although EPA purports that the 2018 NPRM and the SNPRM would benefit "the public", EPA is actually demonstrating that "the public" is not the intended audience for this rulemaking, further confusing the issue. The commenter states that although the new definition of the term "independent validation" at 40 CFR § 30.2 and the context provided in the supplemental proposal purports to uphold the validation of data and models by "subject matter experts", the supplemental proposal contains references to "the public" as the primary entity that would be interested in independently validating research. The commenter states that it is unclear who represents "the public" in these statements. In fact, by defining "subject matter experts", EPA seems to be backtracking on the idea that increased transparency of underlying data and models will have any impact at all on the ability of "the public" to either understand or meaningfully participate in EPA's regulatory process. The commenter notes that the proposed definition at 40 CFR 30.2 in the supplemental proposal limits the reanalysis to individuals with expertise (industry, academia, or government organizations)  -  not "the public" as a whole. The commenter states that this clarification makes sense in the context of EPA's attempts to regulate its judgement of scientific information, but it contradicts EPA's broader argument that this proposed rule would benefit "the public's" understanding of science and EPA's deliberative and rulemaking processes.
Commenter (10974) states that proposing rulemaking to allow "subject matter experts" to reanalyze a specific study or data set  -  based solely on the premise that those "experts" disagree with the results of a given study  -  is redundant of the scientific peer review process. The commenter notes that published studies are often reviewed by public agencies and the data revisited to understand statistical power, relevance to the policy decision, and data quality. The commenter notes that, even in these instances, the underlying data are not made public, but are instead provided at the discretion (and legal authority) of the institution that owns the data. The commenter notes that individual researchers do not typically own the data they collect, nor the results of their research. The commenter states that institutions that conduct research and subsequently own data sets and models (e.g. federal, state, local, and tribal governments, universities, or private industry) are frequently bound by their own rules and guidelines for conducting that research. The commenter suggests that it is governmental overstep that EPA holds the authority to influence how these entities collect and share their data. The commenter asserts that it is unfathomable that EPA overturn privacy agreements and protections between study participants, as originators of a given data set, studies, or models. The commenter finds that the proposed changes to 40 CFR 30.2 are not effective at addressing their concerns, and are, in fact, contradictory to the communicated motivation of this rule to help the public understand, and participate in, the rulemaking process.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions.
 Implementation of this rule will more effectively share pivotal science for external consideration and increase the opportunity for independent validation of pivotal science by subject matter experts. As data are better understood through independent reanalysis, the public will, if they so choose, be able to more effectively comment, engage, and hold the EPA accountable during the development of future significant regulatory actions and influential scientific information. 
The EPA expects to identify pivotal science, and the consideration afforded to pivotal science, in proposed significant regulatory actions and external review drafts of influential scientific information, which will allow the subject matter experts, if they so choose, to independently validate the pivotal science and provide comment to the EPA. The EPA believes that this approach will allow the public to more effectively comment, engage, and hold the EPA accountable during the future development of specific significant regulatory actions and influential scientific information.
Given their expertise, the use of subject matter experts is key to effectively determine if the results of a study can be produced when the study data are reanalyzed. Because this rule is about scientific data, the EPA finds it unlikely that without the necessary expertise, one could reasonably reanalyze the dose-response data underlying pivotal science. This final rule does not preclude the public from engaging subject matter experts to determine whether a study can be independently validated. Also, the definition cannot be considered solely in isolation. The regulatory text in which the term is used informs the extent of the availability of dose-response data underlying studies. Specifically, 40 CFR 30.5 requires, in part, that the dose-response data underlying studies that the EPA will consider as pivotal science be available in a manner sufficient for independent validation. Scientific information is considered available in a manner sufficient for independent validation when it includes the information necessary to understand, assess, and reanalyze findings. The efficacy of the reanalysis will depend on the expertise of the person conducting the reanalysis.
Comment: Commenter (12720) contends that retroactive application of new data release requirements would newly allow industry lobbyists to petition the Agency for rulemakings under EPA's IQGs to review or refine rules on grounds that some subset of studies do not meet the supplemental proposal's public data release requirements.
Commenter (11190) asserts the rule would open the door for special interests (polluters) to legally challenge regulations protecting the public from dangerous pollutants.
Response: The rulemaking applies prospectively to significant regulatory actions and influential scientific information. The rulemaking has no retrospective effect on either significant regulatory actions or influential scientific information. For future final significant regulatory actions and influential scientific information, it applies equally to all the data and models underlying studies used as pivotal science. Scientific transparency is important regardless of the age of the study or the dose-response data.
21.2.1.5 Request for Extension of Comment Period/Public Hearing
Comment: Several commenters (including, but not limited to, 9336, 9337, 9338, 9339, 9359, 9455, 9493, 10041, 10048, 10742, 10752, 10861, 10969, 10972, 11184, 11186, 11402, 11576, 12715, 12727) requested an extension of the comment period for the supplemental proposal, asserting that the 30-day comment period as inadequate given the complexity of the supplemental proposal and the ongoing COVID-19 pandemic. Most requests wanted a minimum of an additional 30 to 90 days to comment beyond the initial 30-day comment period. Support provided for extension requests included:
 Executive Order 12866, which provides regulatory planning and review guidance for federal agencies, states that the public's opportunity to comment on any proposed regulation, "in most cases should include a comment period of not less than 60 days." (9338, 9359, 9493, 10048, 11184, 11186)
 The Administrative Procedure Act (APA) "requires that the public have a meaningful opportunity to submit data and written analysis regarding a proposed rulemaking." The purposes of the APA's notice and comment requirements are "(1) to ensure that agency regulations are tested via exposure to diverse public comment, (2) to ensure fairness to affected parties, and (3) to give affected parties an opportunity to develop evidence in the record to support their objections to the rule and thereby enhance the quality of judicial review." (11576, 12727)
 The APA requires agencies to "give interested persons an opportunity to participate in the rule making" through the submission of written comments or oral presentation. 5 U.S.C. § 553(c). The importance of this public comment process -- the purposes of which include ensuring informed Agency decision-making, encouraging public participation in the administrative process, and ensuring that agencies keep an open mind towards their rules -- "cannot be overstated." N.C. Growers' Ass'n v. United Farm Workers, 702 F.3d 755, 763 (4th Cir. 2012). In order to achieve these purposes, "the opportunity to comment `must be a meaningful opportunity.'" Id. (quoting Prometheus Radio Project v. FCC, 652 F.3d 431, 450 (3d Cir. 2011). (12715)
 The CAA requires that the public be permitted to meaningfully comment on EPA's proposed rulemakings. (12727)
 There are numerous reasons beyond the COVID-19 emergency for granting an extension: the uncertain meaning or implications of new provisions in the supplemental notice, how the new and older provisions will interact, and the unprecedented proposed reliance on the federal Housekeeping statute (5 U.S.C. § 301) as legal authority for limiting the use of science in rulemaking (and in an unspecified range of other circumstances) under eight EPA statutes. (9359)
 Under these circumstances [COVID-19 pandemic], a thirty-day comment period would not fulfill the requirements of the Administrative Procedure Act -- which requires agencies to "give interested persons an opportunity to participate in ... [a] rule making through [the] submission" of comments -- insofar as the national emergency and its consequences would prevent critically impacted constituencies, such as scientists and health professionals, from participating and being heard. (9493)
 It is absurd to set a comment deadline while citizens are subject to local, state and federal states of emergency and/or safer at home orders that have been adopted during the COVID-19 crisis. See, e.g., President Trump Declares State of Emergency for COVID-19 (discussing the March 13, 2020 emergency declaration). At a bare minimum, the public comment period for the SNPRM should not begin until after the COVID-19 crisis has ended. (10752)
Commenter (10752) requested more than a 30- or 90-day extension to the initial 30-day comment period. The commenter states that since the EPA improperly claims that separate comments on the SNPRM are necessary, it is necessary for EPA to reopen comments on the SNPRM for a minimum of 360 days. A 360-day review period, according to the commenter, would give the public a chance to review the hundreds of thousands of public comments and how they apply to and/or are affected by the SNPRM.
Commenter (9336) requested that EPA suspend the public comment period while President Trump's national emergency declaration remains in effect and provide at least 90 days for comment once the national emergency is lifted. It is essential that EPA afford the public adequate time to thoughtfully consider the effects of this supplemental proposal and an opportunity to be publicly heard on key issues.
Response: The comment period on the 2020 SNPRM was extended thirty days to May 18, 2020, through EPA-HQ-OA-2018-0259-10743. In total, the comment period on the SNPRM, including prepublication availability, lasted 76 days.
Comment: Several commenters (including, but not limited to, 9336, 9338, 9359, 10084, 12727) also requested that EPA conduct a public hearing or several hearings to provide opportunity for the public to provide oral testimony on the supplemental proposal. Support provided for the need for a public hearing included: 
 The scope of the rule has been expanded from the original rule, and the SNPRM would impact rulemaking under several of the EPA's major environmental acts. (10084)
 The SNPRM cites the CAA as a source of authority for this proposal. We note that the SNPRM would pertain to several of the rulemakings listed in CAA section 307(d)(1). CAA section 307(d)(5) provides that "In promulgating a rule to which this subsection applies...the Administrator shall give interested persons an opportunity for the oral presentation of data, views, or arguments, in addition to an opportunity to make written submissions." (10084)
 Section 307(d) of the CAA requires EPA to hold a public hearing on this Supplemental Notice, and to hold the record open for at least 30 days after the hearing. Because the Supplemental Notice -- like the original proposal -- would "pertain[] to" many EPA rulemakings enumerated in section 307(d)(1), this proposal is clearly subject to the procedural requirements of section 307(d). That EPA relied on the CAA as a source of authority for the original proposal, and indicates in the Supplemental Notice that it is still considering that possibility, only reinforces EPA's obligation to comply with the public hearing requirements of section 307(d). EPA's failure to do so thus far is unlawful, and wrongly denies the public an important and legally-required opportunity to weigh in on this sweeping and harmful proposal. (12727)
 EPA must hold public hearings on the supplemental proposal. At this time, virtual hearings are appropriate. These virtual hearings should include both the option for video conferencing and call in. This technology is readily available to ensure the public has the same opportunity as in an in-person hearing to testify face-to-face with EPA's experts and "see" the other witnesses offering testimony. Even as we ask for virtual hearings for this proposal, however, virtual hearings are not a replacement for in-person hearings and should not become the norm for EPA rulemakings. (9338)
 Requested that EPA provide three public hearings once the national emergency is lifted. It is essential that EPA afford the public adequate time to thoughtfully consider the effects of this supplemental proposal and an opportunity to be publicly heard on key issues. (9336)
 The SNPRM also cites the Clean Water Act (CWA), the Resource Conservation and Recovery Act (RCRA), and the SDWA as sources of authority. 40 CFR Part 25 sets forth requirements for public participation under CWA, RCRA, and SWDA, the scope of which includes EPA rulemaking, such as the SNPRM. 40 CFR 25.3(c) includes the following objectives of EPA: 
            (1) To assure that the public has the opportunity to understand official programs and proposed actions, and that the government fully considers the public's concerns;
(2) To assure that the government does not make any significant decision on any activity covered by this part without consulting interested and affected segments of the public;
(3) To assure that government action is as responsive as possible to public concerns;
(4) To encourage public involvement in implementing environmental laws;
(5) To keep the public informed about significant issues and proposed project or program changes as they arise;
(6) To foster a spirit of openness and mutual trust among EPA, States, substate agencies and the public; and
(7) To use all feasible means to create opportunities for public participation, and to stimulate and support participation. (10084)
Response: A public hearing was not hosted for the 2020 SNPRM, but a public hearing was hosted to gather public comment on the 2018 NRPM. Besides this public hearing, interested stakeholders had a total of 185 days to comment on both the April 2018 NPRM and the 2020 SNPRM. The EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural final rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers.
Comment: Commenters (including, but not limited to, 11354, 11393, 11407, 11479, 11496, 11894, 12412, 12423, 12427, 12435, 12442, 12464, 12465, 12474, 12701, 12715, 12720, 13549, 14397) believe that the public comment period for the supplemental rulemaking, as extended to 60 days from 30 days, is still inadequate to provide opportunity for the public to meaningfully comment on the supplemental proposal. Several of the commenters cite the complexity and magnitude of changes set out in the supplemental proposal in the midst of the COVID-19 pandemic as reasoning for the need for additional time. Examples of reasons for the need for additional time to comment include:
 Many of the public health and medical experts whose research is most affected by the proposal and who are best positioned to comment on it are on the front lines of the response to the COVID-19 pandemic. The commenter states that given their expertise in public health, the signatories (medical experts) are uniquely positioned to guide government offices and advise the public on how to respond to this unprecedented pandemic and implies that their full attention should be on responding to the pandemic. Public health experts whose input is essential to the evaluation of this proposal are the same experts responding to the public health emergency. (11479, 12456)
 Even if now were an appropriate time to hold a comment period, Executive Order 12866 suggests that a 60-day comment period is the minimum necessary to afford the public a meaningful opportunity to comment during normal times. This is especially true for a rule that would significantly impact nearly every aspect of EPA's scientific and regulatory work. The supplemental notice is substantially different from the 2018 draft rule. For example, by expanding the rule's applicability from only dose-response research to all research, and from only "pivotal regulatory science" to all "influential science," the supplemental notice dramatically widens the scope of the rule and the research affected by it. Further, the draft rule was vaguely written, limiting the ability of commenters to assess and articulate impacts of such a policy change. (11479)
 More time is needed because the administrative record for this rulemaking fails to address impacts of the proposed rule as modified by the supplemental notice, including the types and number of studies that would be affected, the costs that the proposed rule would impose on the Agency and individual researchers, and how the rule would impact the Agency's mission to protect public health and safety. Despite this dearth of analysis, the Administration has deemed this proposal not economically significant. Thus, additional time and more avenues for public comment, including public hearings, are crucial to provide the public and experts with the opportunity to assess the rule's impacts, since the Agency has failed to do so. (11479)
 The SNPRM would significantly impact every facet of EPA's work. EPA is required to provide the public with notice of a proposed rule, followed by a meaningful opportunity to comment on the rule's content. 5 U.S.C. § 553 (b)-(c). Executive Order 12,866 directs EPA to "afford the public a meaningful opportunity to comment on any proposed regulation, which in most cases should include a comment period of not less than 60 days." The opportunity to comment must be meaningful under the APA, which "means enough time with enough information to comment." Prometheus Radio Project v. FCC, 652 F.3d 431, 450 (3d Cir. 2011) (citing Rural Cellular Ass'n v. FCC, 588 F.3d 1095, 1101 (D.C. Cir. 2009)). (11393, 11407, 12412, 12474, 12715, 13549)
 At a normal time, 60 days would be insufficient for the public to meaningfully review and comment on the SNRPRM with such wide-ranging proposed changes and impacts. However, this is not a normal time: The national emergency due to the COVID-19 pandemic necessitates additional time for the commenter to review and evaluate the potential impacts of the SNPRM. (11393)
Response: The comment period on the SNPRM was extended 30 days to a total of 60 days. The Agency believes the current 60 day period was sufficient time to submit comments. This fact is confirmed by the many comments already received by the Agency. Interested stakeholders had a total of 185 days to comment on both the April 2018 NPRM and the 2020 SNPRM.
Comment: Commenter (12720) states that the EPA summarily denied stakeholders' requests for a virtual public hearing, despite the immense public interest the supplemental proposal has garnered. To-date, the commenter notes that thousands of public comments have already been submitted regarding the supplemental proposal in the short 60-day comment period the Agency has allowed. The commenter notes that their members will submit over 114,000 comments expressing deep concern about the Proposal, and its harmful effects on science and the regulatory process. By the close of the comment period, the commenter states that the Agency will receive hundreds of thousands of comments from environmental advocates, scientists, and professors, alike, asking that the Agency reconsider the proposal and supplemental proposal. Yet, the commenter states that instead of recognizing this substantial public interest, the Agency has refused to schedule a virtual public hearing regarding the supplemental proposal, completely ignoring the public's requests that it do so. See, e.g., Email correspondence, C. Hawkins to A. Mesnikoff, Mar. 23, 2020.
According to the commenter (12720), a virtual public hearing is necessary to allow those who would be affected by the supplemental proposal, were it to become final, to explain the basis of their objections to EPA's experts and policy makers. Likewise, the commenter asserts that a hearing would also allow opponents of the supplemental proposal to experience other witnesses offering testimony, consider their positions, and respond. The commenter adds that a virtual hearing will ensure that all parties have an opportunity to learn about the supplemental proposal, consider its impacts, and provide meaning comments to assist the Agency in the rulemaking process. 
The commenter (12720) contends that the need for a virtual public hearing is particularly critical now, when many who would ordinarily comment on the supplemental proposal are facing unprecedented limitations, due to the coronavirus pandemic. The commenter states that the entire country -- including scientists and researchers who would be directly affected by the Supplemental Proposal -- has had their way of life uprooted and is still adjusting. The commenter notes that without a virtual public hearing, the voices of scientists, professors, and regular people who are deeply concerned about the effects of this Supplemental Proposal, will not be adequately heard.
The commenter (12720) states that the EPA's refusal to hold a virtual public hearing regarding the proposal is puzzling, in light of its willingness to do so for other critical proposed rules. For example, the commenter notes that the Agency has announced plans to hold two virtual public hearings for a proposed action titled "Review of National Ambient Air Quality Standards for Particulate Matter." The commenter provides that the virtual hearings were announced on May 5, 2020, and are set to occur later this week, on May 20, 21 and 22, 2020, proving that the Agency can act relatively quickly to plan to hold virtual hearings as part of the rulemaking process. Clearly, according to the commenter, the Agency has the technological wherewithal, organizational experience, and staffing capacity to hold virtual public hearings on major proposed rules. The commenter states, however, that the EPA has not explained why it cannot do so here.
Commenter (12720) adds, the Agency is well-aware that public hearings are an important tool in making the public's voice heard regarding significant proposed rulemakings, like the supplemental proposal. For example, the commenter notes that when the original proposed rule was issued in 2018, EPA not only held a public hearing, but extended the comment period to 90 days. The commenter states that the hearing on the 2018 Proposal was critical because it allowed the public to express their concerns about the limitations of the proposed rule. The commenter notes that the Agency made sweeping changes to its proposal, in part, as a result of those concerns. Nonetheless, the commenter states that the EPA refuses to explain why the original proposed rule -- part of the very same rulemaking proceeding -- required a public hearing, but the instant, much more expansive proposal merits none at all. Instead, the commenter complains that the EPA has summarily denied stakeholders' request for a hearing.
Response: A public hearing was not hosted for the 2020 SNPRM, but a public hearing was hosted to gather public comment on the 2018 NRPM. The Agency believes the 60-day period was sufficient time to submit comments. Interested stakeholders had a total of 185 days to comment on both the April 2018 NPRM and the 2020 SNPRM.
21.2.1.6 Other Miscellaneous Comments
21.2.1.6.1 Scope
State Activities
Comment: Commenters (12472, 11892) state that it is unclear whether the rule applies to state programs that are authorized or delegated by EPA, and that it should be clarified that this will not be the case. 
Response: This final rule describes how the EPA will determine the consideration to afford pivotal science of the EPA's final significant regulatory actions and influential scientific information for which the conclusions are driven by the quantitative relationship between the amount of dose or exposure to a pollutant, contaminant, or substance and an effect based on the availability of the underlying dose-response data and other applicable factors. This rule exclusively pertains to the internal practices of the EPA and does not regulate the conduct or determine the rights or obligations of any entity outside the federal government. If implementing this procedural rule would result in conflicts with existing environmental statutes, the procedural rule will yield to the existing statutes. Implementation guidelines and statute-specific transparency regulations or programmatic regulations are forthcoming.Attainment Demonstrations.
Comment: Commenter (12472) express that the rule should also not apply to "other regulatory decisions, such as attainment demonstrations," as this would cause difficulties with compliance.
Response: Transparency is important in ensuring that the decisions the EPA makes are based on sound science. The EPA is finalizing the applicability of this rulemaking to significant regulatory actions and influential scientific information because of the potential broad impact of these actions and assessments on American lives and livelihoods. The EPA is not applying this rulemaking to permit proceedings, site-specific actions, or enforcement actions because these actions are typically focused on individual regulated entities.
EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. The EPA will undertake such efforts in forthcoming actions, which will be either.
Individual Permits
Comment: Commenter (11892) notes that "the statement in Section §30.5 `except where explicitly stated otherwise that provisions of this subpart do not apply to any other type of Agency action, including individual party adjudications, enforcement activities or permit proceedings' should be interpreted to mean that that individual permits are not covered by this rule" because there are already "adequate appeal processes and public involvement steps in place for challenging the validity of an agency's decision without adding the additional steps and criteria in the proposed rule."
Response: EPA is relying exclusively on its housekeeping authority to promulgate this purely procedural rule. In this final procedural rule, the EPA does not interpret or apply provisions of a particular statute or statutes that it administers. 
The EPA is finalizing the applicability of the rulemaking to final significant regulatory actions and influential scientific information because of the broad impact of these actions and assessments. The EPA will not expand the applicability of this rulemaking beyond these categories of regulations and assessments to permit proceedings, site-specific actions, and enforcement actions because of the focused nature of these actions.
As stated in the final rule, Part 30.4. "The provisions of this part do not apply to physical objects (like laboratory samples), drafts, and preliminary analyses, and influential scientific information or pivotal science that meet one or more of the exemptions identified in Section IX of the OMB Final Information Quality Bulletin for Peer Review. In the event the procedures outlined in this part conflict with statutes the EPA administers, or their implementing regulations, the statutes and regulations will control. Except where explicitly stated otherwise, the provisions of this part do not apply to any other type of Agency action, including individual party adjudications, enforcement activities, site-specific actions, or permit proceedings."
21.2.1.6.2 Public Trust in Science
Comment: Commenter (11403) expresses that:
      Critical analysis does not require a re-analysis of raw data every time a study is relied upon because scientists correctly know that the scientific process, through peer review and objectivity, creates knowledge that can then be interpreted using explanations that build on previous science. Part of that knowledge is that peer review is an ethical component of science that is centered on truthful reporting of methods and outcomes. If violations occur, the scientists responsible are censured by their peers, and other studies will reveal the errors. 
The commenter adds that the proposed rule would undermine public trust in the scientific process by "implying that the thorough scientific process including peer review does not provide information for the public to trust and understand."
Response: This final rule does not require the EPA or third parties to reanalyze, rather, the purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. 
As Jamieson et al. (2019) state, "Journals play a key role in ascertaining the quality of the evidence in work submitted for publication through peer review. However, that practice is neither uniform nor free of exploitation. While it can select for trustable results, its existence alone is not sufficient to verify the quality of the evidence used to back claims." Because of the potential impact of significant regulatory actions, independent validation of studies that are pivotal to the EPA's significant regulatory actions and influential scientific information may increase public confidence in the findings and conclusions presented in those studies. As Jamieson et al. (2019) states, publication alone (including the typical peer review necessary for publication) is insufficient to verify a study's quality, and additional confidence could be gained from third-party reanalyses that independently validate the study results.
Comment: Several commenters (11497, 11498, 12458) express concern that the supplemental proposal's expansion to cover influential scientific information will undermine public health and confidence in the EPA. Comments include the following:
 Commenter (11497) states that the widened scope of the rule would further erode public confidence in EPA decisions and deeply undermine the ability of EPA to use all the best available science in its regulatory decision, which, in turn, will undermine public health.
Commenter (11498) provides that the fact that EPA would risk prohibiting or severely limiting important studies related to contaminants, exposure, and protections of human health sends a chilling message to the scientific community and risks breaching the confidence of the American public on whether they can trust EPA decisions to protect their health.
Commenter (12458) states that influential scientific information could include scientific outreach and educational information produced by the Agency, as well as guidance and Agency statements of scientific fact that could influence public and private action. According to the commenter, excluding or minimizing the role of certain kinds of research from Agency activities will almost inevitably result in a divergence between EPA's public health recommendations and those of some segments of the scientific community, potentially causing public confusion and delegitimization of the Agency.
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions.
The EPA is not categorically excluding any studies from consideration - even studies where the underlying dose-response data are not available for independent validation - when promulgating final significant regulatory actions or developing influential scientific information. Rather, as discussed in Part 30.5 of the final rule, the Agency will continue to evaluate the quality of all relevant studies, consistent with the intended use of the information. The EPA will also continue to rely on the highest quality, most relevant studies available in determining the potential for hazard due to exposure to a pollutant, contaminant, or substance. The EPA will continue to use established guidelines for identifying and integrating evidence
21.2.1.6.3 Lab Practices
Comment: Commenter (12399) provides that: 
      Section 30.3 explicitly states that the rule does not apply to physical objects such as lab samples. This... raises some issues about how the rule might include provisions to promote sound research lab practices. One approach might be to require that the agency place greater weight on studies that can demonstrate the application of `Good Laboratory Practices' (GLP). 
The commenter (12399) suggests that "greater transparency related to the handling and management of lab samples should be included as part of the transparency rule."
Response: The Agency has considered the commenters suggestion and does not believe that lab samples and other physical objects are appropriate for the scope of this final rule. Availability of such physical objects requires a different set of considerations, including shelf-life, preservation and accessibility. The final rules states, at Part 30.3 that "The provisions of this section do not apply to physical objects (like laboratory samples) . . ."
21.2.1.6.4 Rule is Inconsistent with Scientific and Public Health Practice
Comment: Commenter (12435) states that demonstrable environmental public health protections have been achieved by EPA for decades utilizing scientific studies submitted and reviewed without the proposed stultifying disclosure requirements. Moreover, the commenter notes that other federal regulatory agencies, including but not limited to the Food and Drug Administration, the Occupational Safety and Health Administration, the Department of Housing and Urban Development, the Consumer Product Safety Commission, and the Department of Agriculture, have likewise accepted and used for regulatory purposes human health studies that have been generated, published, and submitted without meeting the proposed criteria. The commenter adds that the same practice applies to nonregulatory, albeit authoritative, guidelines issued by the World Health Organization, the Centers for Disease Control and Prevention, the US Preventive Services Task Force, the American Heart Association, and the American Cancer Society. The commenter states that these agencies will continue to accept for consideration scientific findings that have fulfilled consensus standards of peer review that already enjoy widespread acceptance by the academic, scientific, public health, and medical communities. If the proposed supplemental rule were adopted, the commenter asserts that the EPA would operate outside the mainstream of scientific and public health practice."
Response: The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions.
The EPA is a leader in the evaluation of human and ecological impacts of environmental pollution and methods to cost-effectively mitigate these impacts. The selection and use of quality, peer-reviewed literature is a critical part of this work. However, the strength of the review conducted as part of the study's acceptance and publication in a scientific journal has received increased attention in the scientific community due to the lack of transparency and consistency, potential for bias, and the difficulty that independent researchers have had in reproducing published results.[,][,][,][,][,] Jamieson et al. (2019) state, "Journals play a key role in ascertaining the quality of the evidence in work submitted for publication through peer review. However, that practice is neither uniform nor free of exploitation. While it can select for trustable results, its existence alone is not sufficient to verify the quality of the evidence used to back claims." Because of the potential impact of significant regulatory actions, independent validation of studies that are pivotal to the EPA's significant regulatory actions and influential scientific information may increase public confidence in the findings and conclusions presented in those studies. As Jamieson et al. (2019) states, publication alone (including the typical peer review necessary for publication) is insufficient to verify a study's quality, and additional confidence could be gained from third-party reanalyses that independently validate the study results.
21.2.1.6.5 Regulatory Language Correction
Comment: Commenter (12710) writes that the proposal repeatedly uses the word "section" when the word "part" should be used. For example, the commenter states that section 30.3 begins, "[t]he provisions of this part apply..." but in the next sentence states, "The provisions of this section do not apply..." and concludes by referencing the "provisions of this part." The commenter adds that this same mistake is made in both versions of section 30.5. 
Response: The final rule uses the word "part" rather than "section."
21.2.1.6.6 Request for Redline Strikeout Document
Comment: Commenter (10084) notes that federal agencies, including EPA, normally include a redline strikeout document that represents the proposed changes to a rule. The commenter notes that a redline document is not included in the SNPRM docket folder, which makes an expedited review even more difficult. The commenter suggests that a redline document would be very useful for review and documentation purposes and the commenter requests a redline document showing the changes made to the SNPRM.
Response: The preambles to the NPRM, SNPRM and final rule are so significantly different that a redline document would not be very useful. A redline document of the regulatory text can be provided.
Comment: Commenter (12410) asserts that the rules set forth should be dependable and give the EPA good cause for making decisions that do not overreach regulatory authority or give way to obstructionist looking to delay development. The commenter provides that the rules should be the rules and not subject to further challenge on those not acting in good faith or wishing to contemplate insufficient data sets with unreasonable uncertainty, i.e. reanalyze data not capable of being substantially reproduced.
Response: If implementing this procedural rule would result in conflicts with existing environmental statutes, the procedural rule will yield to the existing statutes.
Comment: Commenter (11899) asserts that critical issues such as temperature and climate fluctuations, threats to plant and animal species from fossil fuels versus from wind turbines, solar panels and other replacement energy sources, the enormous benefits of using fossil fuels  -  and much more  -  could and should have been addressed in open, robust discussions at EPA and throughout America, before EPA's Endangerment Finding was made. The commenter contends that would have happened at an Agency governed by transparency rules of the kind proposed by the Agency's "Strengthening Transparency in Regulatory Science" formulation. The commenter states that America's industries, communities and families would all be better off today if it had been in force when the Endangerment, Clean Power, Particulate Matter (PM) 2.5um and other EPA policy decisions were being made.
Response: The rulemaking applies prospectively to final significant regulatory actions and influential scientific information. This rulemaking has no retrospective effect on either final significant regulatory actions or influential scientific information. For future final significant regulatory actions and influential scientific information, it applies equally to all the data and models.
21.2.1.6.7 Science Community Bias 
Comment: Commenter (10834) suggests bias in the science community and contends that there is substantial evidence that shows that organizations, like the American Association for the Advancement of Science (AAAS) and the Science editors have repeatedly rejected and refused to publish articles in support of the proposed rule and instead, continue to publish (prefer) articles that speak out in opposition to the rule. The commenter provides a brief description of the process they underwent in an attempt to publish their own manuscript through Science. The manuscript entitled, "The EPA Transparency Rule is Scientifically Justified and Necessary" was submitted as part of the commenter's official comments on the supplemental proposal, along with copies of email exchanges between him/her and the journal editors, as support. In general, the manuscript argues in support of the 2018 proposed rule and takes the position that the rule is completely consistent with publication policies set by science journals, despite the science community's ongoing opposition to the rule. The commenter provides examples where science-based communities/organizations (AAAS and Science) have openly criticized and aggressively opposed the rule. The commenter concludes that the rejection of their manuscript, coupled with these organizations' statements is evidence of partiality within the scientific community and urges the EPA to consider these predispositions when weighing decisions. 
Response: EPA appreciates the commenter's support of this rule. EPA agrees that this rule is a necessary incremental step in the path to increased transparency.
21.2.2 Out of Scope Comments
21.2.2.1 Critiques of EPA Regulations and Science 
Comment: Commenter (11899) suggests that epidemiological studies are faulty by design and incapable of accurately assessing (or reliably identifying) the impacts (causes and effects) of air pollution by specific source(s) on human health. The commenter indicates that the Agency has historically supported, preferred (or selected) epidemiologic studies with outcomes that back pre-convinced, faulty epidemiological assertions. As an example, the commenter provides a brief discussion on the assessment of impacts on public health associated with particulate matter (PM). In this case, the commenter argues that the Agency failed to present a plausible biological explanation for why or how super-tiny particles can cause multiple diseases and deaths simply by getting into lungs or bloodstreams. The commenter notes that many of the studies used to assess the impacts of particulates illegally and unethically involved human test subjects, which according to the commenter, violates U.S. laws  -  e.g., the Nuremberg Code, the Helsinki Accords and EPA Rule 1000.17  -  all of which make it unethical or illegal to conduct toxicity experiments on humans. The commenter claims that during these studies EPA-contracted researchers failed to disclose to volunteers that they (the Agency) were adamant in the assertion that the polluted air the test subjects were about to breathe was toxic, carcinogenic, and deadly. Instead, according to the commenter, volunteers were led to believe that the they would face only "minimal risks,"  - i.e., the level of risk ordinarily encounter in daily life, in performing routine physical activities. While other volunteers were told they might experience claustrophobia in the small study chambers, or some minor degree of airway irritation, shortness of breath, coughing or wheezing. The commenter also highlights the point that volunteers recruited to participate in these particulate matter/toxicity experiments were also the demographic deemed most at-risk by the EPA  -  i.e., the elderly, asthmatics, diabetics, people with heart disease, children. In addition, the commenter notes that study participants was often exposed to eight, thirty or sometimes sixty times more particulates per volume  -  for up to two hours  -  than they would normally breathe outdoor. 
The commenter lists the following issues/questions that they believe were not adequately addressed by the EPA regarding the PM/toxicity experiments:
 How can it be that PM2.5 are dangerous or lethal for Americans in general, every time they step outside  -  but harmless to human test subjects who were intentionally administered pollution dozens of times worse than what they would encounter outdoors?
 How can it be, as EPA-funded researchers asserted at the time, that "acute, transient responses seen in clinical studies cannot necessarily be used to predict health effects of chronic or repeated exposure"  -  when that is precisely what EPA was claiming the studies can and do show, in order to justify its expensive, punitive PM2.5 regulations?
  If PM2.5 is lethal and there is no safe threshold, weren't EPA officials and its hired researchers deliberately misleading volunteers in order to persuade them to breathe deadly, carcinogenic poisons? 
 Weren't the EPA-contracted researchers violating laws and ethical guidelines when they experimented on children, in violation of EPA's own rules banning such experiments  -  and later deleting evidence describing those tests?
 If no one died, doesn't that mean EPA falsely claimed that there is no safe level, that all PM2.5 particulates are toxic, that its regulations would save countless lives, and that the direct and ancillary benefits vastly outweigh their multi-billion-dollar annual costs of its particulate regulations? If that is the case, didn't EPA impose those costs not only for no real benefits, but at great harm to the overall health and welfare of Americans?
 Doesn't all this mean there really are safe levels for airborne soot  -  and PM2.5 particles are not really toxic or lethal?
 Doesn't it mean EPA's draconian standards should be significantly modified, and companies, communities and consumers should be compensated for their costs in complying with excessive, unjustified particulate regulations?
The commenter concludes that the results/outcomes of these studies could not be used to lead to an "informed consent," and that the EPA should rescind PM2.5 requirements and instead develop an entirely new rulemaking process under EPA's new transparency regulations, with full disclosure and scrutiny of all relevant data, models, claims, analyses and conclusions.
Commenter (4541) notes that the Bradford Hill Criteria are endorsed by the Federal Judicial Center (FJC), an education and research agency of the United States federal courts established by an Act of Congress (28 U.S.C. §§ 620 - 629) in 1967, at the recommendation of the Judicial Conference of the United States. The commenter states that FJC's reference manual for judges, titled the Reference Manual on Scientific Evidence, provides expert advice for determining admissibility of scientific evidence in U.S. federal courts, and advises federal judges and lawyers practicing in federal courts to adhere to that advice in complying with the rules of evidence. The commenter adds that the latest (third) edition is co-published by the National Research Council of the National Academies.
Commenter (4541) asserts that existing EPA science violates BHC:
 EPA assumes that association equals causation, administering massive doses of chemicals to laboratory animals predicts the human health impacts of much lower levels of exposure, natural causes of cancer and other diseases are nonexistent, current ambient levels of air pollution can cause disease or death without clinical evidence, and more. All these assumptions violate the BHC and other requirements of the scientific method, rendering EPA's science an unreliable guide for researchers and policymakers.
 A major shortcoming of observational studies is the lack of control that results in their failure to replicate results, a violation of BHC #2 requiring consistency of the observed association. Observational studies cited by EPA and other air quality regulators are not designed to test hypotheses and cannot establish causation. Most observational studies cannot be replicated, and in nearly one case in ten, efforts to replicate results find benefits where the previous study found harms or vice versa. 
 EPA assumes without providing clinical evidence that exposure to ambient levels of air pollution causes disease and mortality. This violates BHC #6, requiring biological plausibility, and #8, requiring experimental evidence. Another violation of the BHC is EPA's reliance on animal experiments in which mice and rats are exposed to near-toxic doses of toxins. EPA assumes such experiments produce reliable evidence of the risk to humans of exposure to far lower levels of those toxins in daily life. But this assumption is contradicted by current toxicological knowledge (BHC #7).
Response: This internal procedural rule governs EPA procedures for considering dose-response data underlying pivotal science to be used in EPA's significant regulatory decisions and influential scientific information. It does not pertain to individual EPA studies. Therefore, the comment is out of scope of the rule.
Comment: Commenter (12645) stresses that biological plausibility is not a one-way street leading to regulation. The commenter states, that in the case of ascribing causality to the epidemiologically, as an example, observed linkage between sulfur dioxide and respiratory health effects needed to be reconsidered when it was found that sulfur dioxide was usually absorbed in the upper airways rather than penetrating to the lung. 
Response: This internal procedural rule governs EPA procedures for considering dose-response data underlying pivotal science to be used in EPA's significant regulatory decisions and influential scientific information. It does not pertain to biological plausibility. Therefore, the comment is out of scope of the rule.
21.2.2.2 Politicization of EPA 
Comment: Commenter (11894) argues that there is a strong disconnect/disparity between the "transparency" requirements proposed by EPA (e.g., when data and models are used as influential scientific information that might affect policy or decisions by the private sector) and methods used by the White House (current Administration), for example, when presenting estimates of the potential number of U.S. deaths from COVID-19. The commenter claims that these disparities illustrate the hypocrisy of the decision processes and attitudes toward science. As support, the commenter cites an excerpt from an April 2nd article from The Washington Post as support, which states that, "A White House representative said the task force has not publicly released the models it drew from out of respect for the confidentiality of the modelers, many of whom approached the White House unsolicited and simply want to continue their work without publicity." 
Response: These comments are outside the scope of this rule. The purpose of this action is to increase transparency by codifying internal procedural requirements for how the EPA will consider the availability of the underlying dose-response data that it relies upon to promulgate significant regulatory actions and develop influential scientific information. These requirements build upon open data initiatives in the federal government and scientific community and advance the EPA's mission and commitment to the public by prioritizing transparency of the underlying dose-response data in pivotal science for the most impactful of EPA's assessments and regulatory actions. 
Comment: Commenter (11903) urges the EPA to consider how the Agency evaluates comments received as it proceeds to finalize the proposed rule.
Reliance on Appointed Managers 
Commenter (11903) expresses concerns over the Agency reliance on managers (who are often appointed) to assess the merits of the proposal and stresses the need for the EPA to appropriately evaluate those managers' past performance when making predictions. The commenter underscores the point that since the Trump Administration took office in January 2017, Federal Courts have found, repeatedly, that the EPA has violated a statutory duty, failed to meet a statutory deadline, or otherwise acted unreasonably. The commenter maintains that this trend is largely due to the change in administration and the change in direction of environmental policy, and warns the EPA to address these influences, if the notice of proposed rulemaking (NPRM) and the supplemental notice of proposed rulemaking (SNPRM or supplemental proposal) are going to have a different fate than previous rules.
The commenter (11903) maintains that each of the managers tasked to this rulemaking has some responsibility to ensure that its outcome protects public health and the environment. To meet that responsibility, the commenter believes that each must estimate the likelihood that EPA  -  due to its reliance on poor predictions, improper assumptions, or misinterpretation of the law  -  is pursuing a policy failure. To do so, the commenter believes that each should remain mindful that, since January 2017, courts have repeatedly ruled against the EPA. The commenter references an attached document entitled, "Cases Finding that the Trump Administration's EPA Violated a Statutory Duty, Failed to Meet a Statutory Deadline, or Otherwise Acted Unreasonably," as support and notes that the cases described in the attachment illustrate a dismal record that the commenter urges the EPA, and any reviewing Court, to consider moving forward. In addition to the list of cases discussed in the attached document, the commenter also provides a number of examples where courts issued a decision to vacate EPA's rule  -  e.g., EPA's April 2018 rule seeking to suspending the entire list of hydrofluorocarbons deemed unsafe substitutes for ozone depleting substances. According to the commenter, the lessons learned from these cases should include: EPA has become poor at promulgating rules, EPA undervalues analyses showing that proposes polices are suboptimal, unreasonable or otherwise unlawful, and predicting the future is difficult, in particular, the courts' likely response to rulemaking efforts.
In light of litigation outcomes referenced above, as EPA moves forward, the commenter (11903) strongly recommends that the Agency should give greater weight to the opinions of individuals (managers) who argued against EPA's proposed rules (later deemed unlawful actions), and lesser weight to individuals that were proponents of those actions. The commenter recommends that the EPA evaluate remarks supporting its proposal, as well as the comments opposing its proposal, in light of past performance and challenges the Agency to change its culture of decision-making. The commenter also requests that the Agency consider and make part of this rulemaking record the article, Tollefson J, Science under siege, Nature 559:316-319 (July 19, 2018) when deliberating.
How to Perceive Comments Received from Red States vs Blue State 
Commenter (11903) warns the EPA that remarks received on the proposed rule may present conflicting predictions concerning the rule's merits and associated consequences (i.e., offering predictions regarding how best to advance the public interest vs. predictions for the purpose of influencing governmental decisions). The commenter states that conflicting predictions about regulatory activity are weighed differently by officials in "Red States" vs. "Blue States," specifically in the area of environmental protection and public health intervention, which according to the commenter are the two of the most polarizing topics addressed by states.
The commenter (11903) anticipates that the EPA will receive comments from all interested parties  -  i.e.., repeat players (State, regulated companies, etc.) familiar with administrative proceeding or the legislative process, submitters that prefer one option over another and provide data to support their position, and stakeholders that offer predictions that very depending on the commenter type  -  e.g., industry group, environmental group, etc. The commenter notes that the current Administration has created a political environment with polarizing effect amongst federal and state agencies; and heightened political polarization has created obstacles to good governmental decision-making. According to the commenter collectively, an administrative Agency's or legislator's response to competing analyses is far from unpredictable, because these individuals have track records. The commenter adds that it is a government official's predictability that allows interest groups to make and use scorecards and rankings of those officials, to identify those officials mostly likely to be receptive to the group's analyses, and to identify those most likely to be skeptical of them. The commenter claims that a quick scan of the scorecards and rankings confirms the conclusion that virtually any participant in this rulemaking has already reached  -  i.e., predictions given to agencies and legislatures are heard by different audiences depending on whether the Administration or legislature is Republican- or Democratic- controlled; and those differences have consequences. Based on this assumption, the commenter proclaims that the analyses offered by the proponents of EPA's current proposal will be found to be more persuasive, and will receive greater weight, by officials in states with Republican administrations and Republican-controlled legislatures. The commenter adds that we should not be surprised that predictions indicating that such a proposal inadequately protects public health and the environment will be viewed more skeptically and given lesser weight by the officials in those States.
Commenter (11903) contends that when evaluating a prediction, it is unreasonable and unwise to ignore the predictor's past performance. The commenter poses the question whether the differences between Red States' and Blue States' methods for evaluating predictions regarding public health and environmental protection has gotten too large. Regardless of the answer, the commenter states that the fact that there are differences should not be ignored. According to the commenter the choices that each state made when it determined to rely on a prediction that appeared to be persuasive, and rejected or gave less weight to conflicting predictions because they seemed inferior, when coupled with the subsequent outcomes to the state's citizens, provide evidence that determines whose predictions were, in fact, more accurate. 
Accordingly, the commenter (11903) believes that if there are substantial differences in the public health outcomes of Americans, and those differences can be predicted by living in a "Red State" versus a "Blue State", those differences suggest that one of those groups of States is better at recognizing accurate predictions, and acting upon them, than the other group. The commenter further asserts that if there are substantial differences in the public health outcomes of Americans, and those differences can be predicted by living in a "Red State" versus a "Blue State," the practices of the more successful states should be emulated, not ignored.
When evaluating conflicting predictions in the comments for this rulemaking, the commenter (11903) urges the EPA not to ignore useful evidence for identifying those predictions which are reliable vs. those predictions which are not reliable  -  i.e., based on past performance. The commenter recommends that the EPA not inutility or blindly guess which predictor will do a better job of predicting actions that will benefit the public. The commenter maintains that identifying those actions and selecting among them is EPA's, and each state's, core responsibility; each of them are repeat players, and their past performance can be compared.
The commenter (11903) states that scientific literature has been able to predict the public health and welfare outcomes of the people living within a particular state and based on the literature Democratic states produce better outcomes. The commenter offers a few questions which they believe, when answered, would help inform EPA on how much, and upon whom, it should rely when evaluating the conflicting predictions in this rulemaking  -  e.g., What happened to the citizens in [Red] States as a result of their governmental officials' reliance on those predictions [from repeat player supporting this proposal]: Did the public at large share in that success, as those interest groups surely predicted as they sought to influence those officials? and What has happened to the citizens in Blue States, the jurisdictions more skeptical of conservative interest groups' rosy predictions? The commenter provides a list of studies with summaries that covers more than 50 pages and includes, for example: 
 Fenelon A, Geographic divergence in mortality in the United States, Population and Development Review 39(4):611-634 (2013);
 Mokdad AH, Ballestros K, Echko M, Glenn S, Olsen HE, Mullany E, Lee A, Khan AR, Ahmadi A, Ferrari AJ, Kasaeian A, The state of US health, 1990-2016: Burden of diseases, injuries, and risk factors among US states, JAMA 319(14):1444-1472 (2018); and 
 Vierboom YC, Preston SH, Hendi AS, Rising geographic inequality in mortality in the United States, SSM-Population Health 17:100478 (2019). 
The commenter (11903) urges the EPA to consider the referenced studies which quantify the disparate performance of the states' performance on a variety of health and welfare outcomes, while remaining mindful of the degree to which a state's performance can be predicted by whether it is a "Red State" or a "Blue State", and the speed by which the gap between the two groups of states is widening. The commenter recommends that the EPA pay particular attention to studies that examine the relationship between the differing outcomes among the states and the governmental decisions that contributed to those differences. 
Response: EPA evaluated all relevant comments and accompanying information received as appropriate when developing this final rule.
21.2.2.3 Previously-Submitted Comments
Comment: Commenters (10861, 11191) provided comments previously-submitted on the initial proposal as part of their comments on the supplemental proposal.
Response: Comments submitted on the NPRM, as well as comments submitted on the SNPRM, were considered during the development of the final rule. Comments previous submitted on the NPRM are addressed in the sections related to those comments.
 21.2.2.4 Interagency Review
Comment: Commenter (13788) questioned the Executive .Order 12866 interagency review process for the supplemental proposed rule.
Response: Comments on deliberative processes, including interagency review and Executive Order 12866, are out of the scope of this rulemaking.
Comment: Commenter (13788) asserted that OMB added in provisions related to the IQA during Executive Order 12866 review.
Response: Comments on deliberative processes, including interagency review and Executive Order 12866, are out of the scope of this rulemaking.
 21.2.2.5 Federal Guidelines
Comment: Commenter (12453) provides that the definition for "influential scientific information" is drawn directly from OMB's December 16, 2004 "Final Information Quality Bulletin for Peer Review" and offered an interpretation of exemptions under OMB M-05-03. Commenter (11360) provides an interpretation of the intention of OMB M-19-15.
Response: OMB documents are not open to public comments and are out of the scope of this rulemaking.
Comment: Commenter (11479) asserts that greater clarity is needed in the process for determining which actions are identified as "significant regulatory actions" under Executive Order 12866.
Response: The determination of significant regulatory actions under Executive Order 12866 is not open to public comments and is outside the scope of this rulemaking.